From owner-svn-src-all@FreeBSD.ORG Mon Jun 8 21:01:14 2009 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD1131065680; Mon, 8 Jun 2009 21:01:14 +0000 (UTC) (envelope-from gnn@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 9762C8FC19; Mon, 8 Jun 2009 21:01:14 +0000 (UTC) (envelope-from gnn@FreeBSD.org) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n58L1ErI043337; Mon, 8 Jun 2009 21:01:14 GMT (envelope-from gnn@svn.freebsd.org) Received: (from gnn@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n58L1EHi043331; Mon, 8 Jun 2009 21:01:14 GMT (envelope-from gnn@svn.freebsd.org) Message-Id: <200906082101.n58L1EHi043331@svn.freebsd.org> From: "George V. Neville-Neil" Date: Mon, 8 Jun 2009 21:01:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-7@freebsd.org X-SVN-Group: stable-7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r193754 - in stable/7/sys: . contrib/pf dev/ath/ath_hal dev/cxgb dev/cxgb/common X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 21:01:16 -0000 Author: gnn Date: Mon Jun 8 21:01:14 2009 New Revision: 193754 URL: http://svn.freebsd.org/changeset/base/193754 Log: MFC of 190206 190330 192537 192 540 192584 192593 192933 Bring the 7.x version of the cxgb driver up to date with respect to HEAD as of 8 June 2009 192933 Rework interrupt bringup and teardown. Calculate the exact number of vectors we'll use before calling pci_alloc_msix. Don't grab nine all the time. Call cxgb_setup_interrupts once per T3, not once per port. Ditto for cxgb_teardown_interrupts. Don't leak resources when interrupt setup fails in the middle. 192593 Partial reversion of previous commit. The CXGB_SHUTDOWN flag does NOT need to be inverted when doing an ifconfig down of an interface. 192584 Fix a possible panic cxgb_controller_attach() routine that would occur only if prepping the adapter failed. Slight adjustment to comments. Fix a bug whereby downing the interface didn't preven it from processing packets. 192540 Integrate three changes from Chelsio. 1) Add a sysctl that will say what type of PHYs exist on the card. 2) Fix a bug that occurs when an AEL 2005 PHY resets without a transciever in the card. 3) Unify the PHY link detection code. 192537 Modified the attach and detach routines to handle bringing ports up and down more cleanly. This addresses a problem where if we have the link flap during boot the driver would lock up the system. 190330 Minor updates to the Chelsio driver, including removing an LOR. 190206 Fix a bug in the recent update to the Chelsio driver. The tick routine was not being restarted in the init_locked routine which could resulted in loss of carrier when updating the MTU. Modified: stable/7/sys/ (props changed) stable/7/sys/contrib/pf/ (props changed) stable/7/sys/dev/ath/ath_hal/ (props changed) stable/7/sys/dev/cxgb/ (props changed) stable/7/sys/dev/cxgb/common/cxgb_ael1002.c stable/7/sys/dev/cxgb/common/cxgb_common.h stable/7/sys/dev/cxgb/common/cxgb_t3_hw.c stable/7/sys/dev/cxgb/cxgb_adapter.h stable/7/sys/dev/cxgb/cxgb_main.c stable/7/sys/dev/cxgb/cxgb_sge.c Modified: stable/7/sys/dev/cxgb/common/cxgb_ael1002.c ============================================================================== --- stable/7/sys/dev/cxgb/common/cxgb_ael1002.c Mon Jun 8 20:50:38 2009 (r193753) +++ stable/7/sys/dev/cxgb/common/cxgb_ael1002.c Mon Jun 8 21:01:14 2009 (r193754) @@ -1160,6 +1160,7 @@ static int get_module_type(struct cphy * v = ael_i2c_rd(phy, MODULE_DEV_ADDR, 131); if (v < 0) return v; + v &= 0xf0; if (v == 0x10) return phy_modtype_lrm; if (v == 0x40) @@ -1249,7 +1250,9 @@ static int ael2005_reset(struct cphy *ph return err; phy->modtype = (u8)err; - if (err == phy_modtype_twinax || err == phy_modtype_twinax_long) + if (err == phy_modtype_none || err == phy_modtype_unknown) + err = 0; + else if (err == phy_modtype_twinax || err == phy_modtype_twinax_long) err = ael2005_setup_twinax_edc(phy, err); else err = ael2005_setup_sr_edc(phy); Modified: stable/7/sys/dev/cxgb/common/cxgb_common.h ============================================================================== --- stable/7/sys/dev/cxgb/common/cxgb_common.h Mon Jun 8 20:50:38 2009 (r193753) +++ stable/7/sys/dev/cxgb/common/cxgb_common.h Mon Jun 8 21:01:14 2009 (r193754) @@ -709,7 +709,6 @@ int t3_slow_intr_handler(adapter_t *adap int t3_phy_intr_handler(adapter_t *adapter); void t3_link_changed(adapter_t *adapter, int port_id); -void t3_link_fault(adapter_t *adapter, int port_id); int t3_link_start(struct cphy *phy, struct cmac *mac, struct link_config *lc); const struct adapter_info *t3_get_adapter_info(unsigned int board_id); int t3_seeprom_read(adapter_t *adapter, u32 addr, u32 *data); Modified: stable/7/sys/dev/cxgb/common/cxgb_t3_hw.c ============================================================================== --- stable/7/sys/dev/cxgb/common/cxgb_t3_hw.c Mon Jun 8 20:50:38 2009 (r193753) +++ stable/7/sys/dev/cxgb/common/cxgb_t3_hw.c Mon Jun 8 21:01:14 2009 (r193754) @@ -1288,6 +1288,49 @@ static void t3_open_rx_traffic(struct cm t3_write_reg(mac->adapter, A_XGM_RX_HASH_LOW, rx_hash_low); } +static int t3_detect_link_fault(adapter_t *adapter, int port_id) +{ + struct port_info *pi = adap2pinfo(adapter, port_id); + struct cmac *mac = &pi->mac; + uint32_t rx_cfg, rx_hash_high, rx_hash_low; + int link_fault; + + /* stop rx */ + t3_gate_rx_traffic(mac, &rx_cfg, &rx_hash_high, &rx_hash_low); + t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, 0); + + /* clear status and make sure intr is enabled */ + (void) t3_read_reg(adapter, A_XGM_INT_STATUS + mac->offset); + t3_xgm_intr_enable(adapter, port_id); + + /* restart rx */ + t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, F_RXEN); + t3_open_rx_traffic(mac, rx_cfg, rx_hash_high, rx_hash_low); + + link_fault = t3_read_reg(adapter, A_XGM_INT_STATUS + mac->offset); + return (link_fault & F_LINKFAULTCHANGE ? 1 : 0); +} + +static void t3_clear_faults(adapter_t *adapter, int port_id) +{ + struct port_info *pi = adap2pinfo(adapter, port_id); + struct cmac *mac = &pi->mac; + + t3_set_reg_field(adapter, A_XGM_TXFIFO_CFG + mac->offset, + F_ENDROPPKT, 0); + t3_mac_enable(mac, MAC_DIRECTION_TX | MAC_DIRECTION_RX); + t3_set_reg_field(adapter, A_XGM_STAT_CTRL + mac->offset, F_CLRSTATS, 1); + + if (adapter->params.nports <= 2) { + t3_xgm_intr_disable(adapter, pi->port_id); + t3_read_reg(adapter, A_XGM_INT_STATUS + mac->offset); + t3_write_reg(adapter, A_XGM_INT_CAUSE + mac->offset, F_XGM_INT); + t3_set_reg_field(adapter, A_XGM_INT_ENABLE + mac->offset, + F_XGM_INT, F_XGM_INT); + t3_xgm_intr_enable(adapter, pi->port_id); + } +} + /** * t3_link_changed - handle interface link changes * @adapter: the adapter @@ -1299,34 +1342,47 @@ static void t3_open_rx_traffic(struct cm */ void t3_link_changed(adapter_t *adapter, int port_id) { - int link_ok, speed, duplex, fc; + int link_ok, speed, duplex, fc, link_fault, link_change; struct port_info *pi = adap2pinfo(adapter, port_id); struct cphy *phy = &pi->phy; struct cmac *mac = &pi->mac; struct link_config *lc = &pi->link_config; - int force_link_down = 0; + + link_ok = lc->link_ok; + speed = lc->speed; + duplex = lc->duplex; + fc = lc->fc; + link_fault = 0; phy->ops->get_link_status(phy, &link_ok, &speed, &duplex, &fc); - if (!lc->link_ok && link_ok && adapter->params.nports <= 2) { - u32 rx_cfg, rx_hash_high, rx_hash_low; - u32 status; + /* + * Check for link faults if any of these is true: + * a) A link fault is suspected, and PHY says link ok + * b) PHY link transitioned from down -> up + */ + if (adapter->params.nports <= 2 && + ((pi->link_fault && link_ok) || (!lc->link_ok && link_ok))) { + + link_fault = t3_detect_link_fault(adapter, port_id); + if (link_fault) { + if (pi->link_fault != LF_YES) { + mac->stats.link_faults++; + pi->link_fault = LF_YES; + } - t3_xgm_intr_enable(adapter, port_id); - t3_gate_rx_traffic(mac, &rx_cfg, &rx_hash_high, &rx_hash_low); - t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, 0); - t3_mac_enable(mac, MAC_DIRECTION_RX); + /* Don't report link up or any other change */ + link_ok = 0; + speed = lc->speed; + duplex = lc->duplex; + fc = lc->fc; + } else { + /* clear faults here if this was a false alarm. */ + if (pi->link_fault == LF_MAYBE && + link_ok && lc->link_ok) + t3_clear_faults(adapter, port_id); - status = t3_read_reg(adapter, A_XGM_INT_STATUS + mac->offset); - if (status & F_LINKFAULTCHANGE) { - mac->stats.link_faults++; - force_link_down = 1; - } - t3_open_rx_traffic(mac, rx_cfg, rx_hash_high, rx_hash_low); - - if (force_link_down) { - t3_os_link_fault_handler(adapter, port_id); - return; + pi->link_fault = LF_NO; } } @@ -1339,75 +1395,65 @@ void t3_link_changed(adapter_t *adapter, duplex == lc->duplex && fc == lc->fc) return; /* nothing changed */ - if (link_ok != lc->link_ok && adapter->params.rev > 0 && - uses_xaui(adapter)) { - if (link_ok) - t3b_pcs_reset(mac); - t3_write_reg(adapter, A_XGM_XAUI_ACT_CTRL + mac->offset, - link_ok ? F_TXACTENABLE | F_RXEN : 0); - } + link_change = link_ok != lc->link_ok; lc->link_ok = (unsigned char)link_ok; lc->speed = speed < 0 ? SPEED_INVALID : speed; lc->duplex = duplex < 0 ? DUPLEX_INVALID : duplex; - if (link_ok && speed >= 0 && lc->autoneg == AUTONEG_ENABLE) { - /* Set MAC speed, duplex, and flow control to match PHY. */ - t3_mac_set_speed_duplex_fc(mac, speed, duplex, fc); - lc->fc = (unsigned char)fc; - } + if (link_ok) { - t3_os_link_changed(adapter, port_id, link_ok, speed, duplex, fc); -} - -void t3_link_fault(adapter_t *adapter, int port_id) -{ - struct port_info *pi = adap2pinfo(adapter, port_id); - struct cmac *mac = &pi->mac; - struct cphy *phy = &pi->phy; - struct link_config *lc = &pi->link_config; - int link_ok, speed, duplex, fc, link_fault; - u32 rx_cfg, rx_hash_high, rx_hash_low; + /* down -> up, or up -> up with changed settings */ - if (!pi->link_fault) - return; /* nothing to do */ + if (link_change && adapter->params.rev > 0 && + uses_xaui(adapter)) { + t3b_pcs_reset(mac); + t3_write_reg(adapter, A_XGM_XAUI_ACT_CTRL + mac->offset, + F_TXACTENABLE | F_RXEN); + } - t3_gate_rx_traffic(mac, &rx_cfg, &rx_hash_high, &rx_hash_low); + if (speed >= 0 && lc->autoneg == AUTONEG_ENABLE) { + /* Set MAC settings to match PHY. */ + t3_mac_set_speed_duplex_fc(mac, speed, duplex, fc); + lc->fc = (unsigned char)fc; + } - if (adapter->params.rev > 0 && uses_xaui(adapter)) - t3_write_reg(adapter, A_XGM_XAUI_ACT_CTRL + mac->offset, 0); + t3_clear_faults(adapter, port_id); - t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, 0); - t3_mac_enable(mac, MAC_DIRECTION_RX); + } else { - t3_open_rx_traffic(mac, rx_cfg, rx_hash_high, rx_hash_low); + /* up -> down */ - link_fault = t3_read_reg(adapter, - A_XGM_INT_STATUS + mac->offset); - link_fault &= F_LINKFAULTCHANGE; + if (adapter->params.rev > 0 && uses_xaui(adapter)) { + t3_write_reg(adapter, + A_XGM_XAUI_ACT_CTRL + mac->offset, 0); + } - phy->ops->get_link_status(phy, &link_ok, &speed, &duplex, &fc); + t3_xgm_intr_disable(adapter, pi->port_id); + if (adapter->params.nports <= 2) { + t3_set_reg_field(adapter, + A_XGM_INT_ENABLE + mac->offset, + F_XGM_INT, 0); + } - if (link_fault) { - lc->link_ok = 0; - lc->speed = SPEED_INVALID; - lc->duplex = DUPLEX_INVALID; - - t3_os_link_fault(adapter, port_id, 0); - - /* Account link faults only when the phy reports a link up */ - if (link_ok) - mac->stats.link_faults++; - } else { - if (link_ok) - t3_write_reg(adapter, A_XGM_XAUI_ACT_CTRL + mac->offset, - F_TXACTENABLE | F_RXEN); + if (!link_fault) { + if (is_10G(adapter)) + pi->phy.ops->power_down(&pi->phy, 1); + t3_mac_disable(mac, MAC_DIRECTION_RX); + t3_link_start(phy, mac, lc); + } - pi->link_fault = 0; - lc->link_ok = (unsigned char)link_ok; - lc->speed = speed < 0 ? SPEED_INVALID : speed; - lc->duplex = duplex < 0 ? DUPLEX_INVALID : duplex; - t3_os_link_fault(adapter, port_id, link_ok); + /* + * Make sure Tx FIFO continues to drain, even as rxen is left + * high to help detect and indicate remote faults. + */ + t3_set_reg_field(adapter, A_XGM_TXFIFO_CFG + mac->offset, 0, + F_ENDROPPKT); + t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, 0); + t3_write_reg(adapter, A_XGM_TX_CTRL + mac->offset, F_TXEN); + t3_write_reg(adapter, A_XGM_RX_CTRL + mac->offset, F_RXEN); } + + t3_os_link_changed(adapter, port_id, link_ok, speed, duplex, fc); } /** @@ -1906,10 +1952,12 @@ static void mc7_intr_handler(struct mc7 static int mac_intr_handler(adapter_t *adap, unsigned int idx) { u32 cause; + struct port_info *pi; struct cmac *mac; idx = idx == 0 ? 0 : adapter_info(adap)->nports0; /* MAC idx -> port */ - mac = &adap2pinfo(adap, idx)->mac; + pi = adap2pinfo(adap, idx); + mac = &pi->mac; /* * We mask out interrupt causes for which we're not taking interrupts. @@ -1942,9 +1990,9 @@ static int mac_intr_handler(adapter_t *a t3_set_reg_field(adap, A_XGM_INT_ENABLE + mac->offset, F_XGM_INT, 0); - mac->stats.link_faults++; - t3_os_link_fault_handler(adap, idx); + /* link fault suspected */ + pi->link_fault = LF_MAYBE; } t3_write_reg(adap, A_XGM_INT_CAUSE + mac->offset, cause); Modified: stable/7/sys/dev/cxgb/cxgb_adapter.h ============================================================================== --- stable/7/sys/dev/cxgb/cxgb_adapter.h Mon Jun 8 20:50:38 2009 (r193753) +++ stable/7/sys/dev/cxgb/cxgb_adapter.h Mon Jun 8 21:01:14 2009 (r193754) @@ -107,6 +107,12 @@ extern int cxgb_debug; #define SX_DESTROY sx_destroy #endif +enum { + LF_NO = 0, + LF_MAYBE, + LF_YES +}; + struct port_info { struct adapter *adapter; struct ifnet *ifp; @@ -130,7 +136,6 @@ struct port_info { uint8_t hw_addr[ETHER_ADDR_LEN]; struct task timer_reclaim_task; - struct task link_fault_task; struct cdev *port_cdev; #define PORT_LOCK_NAME_LEN 32 @@ -394,6 +399,7 @@ struct adapter { device_t portdev[MAX_NPORTS]; struct t3cdev tdev; char fw_version[64]; + char port_types[MAX_NPORTS + 1]; uint32_t open_device_map; uint32_t registered_device_map; #ifdef USE_SX @@ -436,6 +442,7 @@ struct t3_rx_mode { #define ADAPTER_LOCK_INIT(adap, name) SX_INIT(&(adap)->lock, name) #define ADAPTER_LOCK_DEINIT(adap) SX_DESTROY(&(adap)->lock) #define ADAPTER_LOCK_ASSERT_NOTOWNED(adap) sx_assert(&(adap)->lock, SA_UNLOCKED) +#define ADAPTER_LOCK_ASSERT_OWNED(adap) sx_assert(&(adap)->lock, SA_LOCKED) #else #define PORT_LOCK(port) mtx_lock(&(port)->lock); #define PORT_UNLOCK(port) mtx_unlock(&(port)->lock); @@ -447,7 +454,8 @@ struct t3_rx_mode { #define ADAPTER_UNLOCK(adap) mtx_unlock(&(adap)->lock); #define ADAPTER_LOCK_INIT(adap, name) mtx_init(&(adap)->lock, name, 0, MTX_DEF) #define ADAPTER_LOCK_DEINIT(adap) mtx_destroy(&(adap)->lock) -#define ADAPTER_LOCK_ASSERT_NOTOWNED(adap) mtx_assert(&(adap)->lock, MO_NOTOWNED) +#define ADAPTER_LOCK_ASSERT_NOTOWNED(adap) mtx_assert(&(adap)->lock, MA_NOTOWNED) +#define ADAPTER_LOCK_ASSERT_OWNED(adap) mtx_assert(&(adap)->lock, MA_OWNED) #endif @@ -531,8 +539,6 @@ int t3_os_pci_restore_state(struct adapt void t3_os_link_changed(adapter_t *adapter, int port_id, int link_status, int speed, int duplex, int fc); void t3_os_phymod_changed(struct adapter *adap, int port_id); -void t3_os_link_fault(adapter_t *adapter, int port_id, int state); -void t3_os_link_fault_handler(adapter_t *adapter, int port_id); void t3_sge_err_intr_handler(adapter_t *adapter); int t3_offload_tx(struct t3cdev *, struct mbuf *); void t3_os_ext_intr_handler(adapter_t *adapter); Modified: stable/7/sys/dev/cxgb/cxgb_main.c ============================================================================== --- stable/7/sys/dev/cxgb/cxgb_main.c Mon Jun 8 20:50:38 2009 (r193753) +++ stable/7/sys/dev/cxgb/cxgb_main.c Mon Jun 8 21:01:14 2009 (r193754) @@ -76,22 +76,14 @@ __FBSDID("$FreeBSD$"); #include #include -#ifdef CONFIG_DEFINED #include -#else -#include -#endif #ifdef PRIV_SUPPORTED #include #endif -#ifdef IFNET_MULTIQUEUE -#include -#endif - -static int cxgb_setup_msix(adapter_t *, int); -static void cxgb_teardown_msix(adapter_t *); +static int cxgb_setup_interrupts(adapter_t *); +static void cxgb_teardown_interrupts(adapter_t *); static void cxgb_init(void *); static void cxgb_init_locked(struct port_info *); static void cxgb_stop_locked(struct port_info *); @@ -123,7 +115,7 @@ static int offload_open(struct port_info static void touch_bars(device_t dev); static int offload_close(struct t3cdev *tdev); static void cxgb_link_start(struct port_info *p); -static void cxgb_link_fault(void *arg, int ncount); +int t3_detect_link_fault(adapter_t *adapter, int port_id); static device_method_t cxgb_controller_methods[] = { DEVMETHOD(device_probe, cxgb_controller_probe), @@ -183,8 +175,6 @@ static struct cdevsw cxgb_cdevsw = { static devclass_t cxgb_port_devclass; DRIVER_MODULE(cxgb, cxgbc, cxgb_port_driver, cxgb_port_devclass, 0, 0); -#define SGE_MSIX_COUNT (SGE_QSETS + 1) - /* * The driver uses the best interrupt scheme available on a platform in the * order MSI-X, MSI, legacy pin interrupts. This parameter determines which @@ -212,17 +202,17 @@ SYSCTL_UINT(_hw_cxgb, OID_AUTO, ofld_dis /* * The driver uses an auto-queue algorithm by default. - * To disable it and force a single queue-set per port, use singleq = 1. + * To disable it and force a single queue-set per port, use multiq = 0 */ -static int singleq = 0; -TUNABLE_INT("hw.cxgb.singleq", &singleq); -SYSCTL_UINT(_hw_cxgb, OID_AUTO, singleq, CTLFLAG_RDTUN, &singleq, 0, - "use a single queue-set per port"); - +static int multiq = 1; +TUNABLE_INT("hw.cxgb.multiq", &multiq); +SYSCTL_UINT(_hw_cxgb, OID_AUTO, multiq, CTLFLAG_RDTUN, &multiq, 0, + "use min(ncpus/ports, 8) queue-sets per port"); /* - * The driver uses an auto-queue algorithm by default. - * To disable it and force a single queue-set per port, use singleq = 1. + * By default the driver will not update the firmware unless + * it was compiled against a newer version + * */ static int force_fw_update = 0; TUNABLE_INT("hw.cxgb.force_fw_update", &force_fw_update); @@ -234,15 +224,6 @@ TUNABLE_INT("hw.cxgb.use_16k_clusters", SYSCTL_UINT(_hw_cxgb, OID_AUTO, use_16k_clusters, CTLFLAG_RDTUN, &cxgb_use_16k_clusters, 0, "use 16kB clusters for the jumbo queue "); -/* - * Tune the size of the output queue. - */ -int cxgb_snd_queue_len = IFQ_MAXLEN; -TUNABLE_INT("hw.cxgb.snd_queue_len", &cxgb_snd_queue_len); -SYSCTL_UINT(_hw_cxgb, OID_AUTO, snd_queue_len, CTLFLAG_RDTUN, - &cxgb_snd_queue_len, 0, "send queue size "); - - enum { MAX_TXQ_ENTRIES = 16384, MAX_CTRL_TXQ_ENTRIES = 1024, @@ -307,31 +288,6 @@ struct cxgb_ident { static int set_eeprom(struct port_info *pi, const uint8_t *data, int len, int offset); -void -cxgb_log_tcb(struct adapter *sc, unsigned int tid) -{ - char buf[TCB_SIZE]; - uint64_t *tcb = (uint64_t *)buf; - int i, error; - struct mc7 *mem = &sc->cm; - - error = t3_mc7_bd_read(mem, tid*TCB_SIZE/8, TCB_SIZE/8, tcb); - if (error) - printf("cxgb_tcb_log failed\n"); - - CTR1(KTR_CXGB, "TCB tid=%u", tid); - for (i = 0; i < TCB_SIZE / 32; i++) { - CTR5(KTR_CXGB, "%1d: %08x %08x %08x %08x", - i, (uint32_t)tcb[1], (uint32_t)(tcb[1] >> 32), - (uint32_t)tcb[0], (uint32_t)(tcb[0] >> 32)); - tcb += 2; - CTR4(KTR_CXGB, " %08x %08x %08x %08x", - (uint32_t)tcb[1], (uint32_t)(tcb[1] >> 32), - (uint32_t)tcb[0], (uint32_t)(tcb[0] >> 32)); - tcb += 2; - } -} - static __inline char t3rev2char(struct adapter *adapter) { @@ -371,7 +327,7 @@ cxgb_get_adapter_info(device_t dev) { struct cxgb_ident *id; const struct adapter_info *ai; - + id = cxgb_get_ident(dev); if (id == NULL) return (NULL); @@ -400,15 +356,15 @@ cxgb_controller_probe(device_t dev) ports = "ports"; snprintf(buf, sizeof(buf), "%s %sNIC, rev: %d nports: %d %s", - ai->desc, is_offload(sc) ? "R" : "", - sc->params.rev, nports, ports); + ai->desc, is_offload(sc) ? "R" : "", + sc->params.rev, nports, ports); device_set_desc_copy(dev, buf); return (BUS_PROBE_DEFAULT); } #define FW_FNAME "cxgb_t3fw" -#define TPEEPROM_NAME "t3%c_tp_eeprom" -#define TPSRAM_NAME "t3%c_protocol_sram" +#define TPEEPROM_NAME "cxgb_t3%c_tp_eeprom" +#define TPSRAM_NAME "cxgb_t3%c_protocol_sram" static int upgrade_fw(adapter_t *sc) @@ -434,6 +390,32 @@ upgrade_fw(adapter_t *sc) return (status); } +/* + * The cxgb_controller_attach function is responsible for the initial + * bringup of the device. Its responsibilities include: + * + * 1. Determine if the device supports MSI or MSI-X. + * 2. Allocate bus resources so that we can access the Base Address Register + * 3. Create and initialize mutexes for the controller and its control + * logic such as SGE and MDIO. + * 4. Call hardware specific setup routine for the adapter as a whole. + * 5. Allocate the BAR for doing MSI-X. + * 6. Setup the line interrupt iff MSI-X is not supported. + * 7. Create the driver's taskq. + * 8. Start one task queue service thread. + * 9. Check if the firmware and SRAM are up-to-date. They will be + * auto-updated later (before FULL_INIT_DONE), if required. + * 10. Create a child device for each MAC (port) + * 11. Initialize T3 private state. + * 12. Trigger the LED + * 13. Setup offload iff supported. + * 14. Reset/restart the tick callout. + * 15. Attach sysctls + * + * NOTE: Any modification or deviation from this list MUST be reflected in + * the above comment. Failure to do so will result in problems on various + * error conditions including link flapping. + */ static int cxgb_controller_attach(device_t dev) { @@ -533,46 +515,52 @@ cxgb_controller_attach(device_t dev) (sc->msix_regs_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->msix_regs_rid, RF_ACTIVE)) != NULL) { - msi_needed = sc->msi_count = SGE_MSIX_COUNT; - - if (((error = pci_alloc_msix(dev, &sc->msi_count)) != 0) || - (sc->msi_count != msi_needed)) { - device_printf(dev, "msix allocation failed - msi_count = %d" - " msi_needed=%d will try msi err=%d\n", sc->msi_count, - msi_needed, error); + if (multiq) + port_qsets = min(SGE_QSETS/sc->params.nports, mp_ncpus); + msi_needed = sc->msi_count = sc->params.nports * port_qsets + 1; + + if (pci_msix_count(dev) == 0 || + (error = pci_alloc_msix(dev, &sc->msi_count)) != 0 || + sc->msi_count != msi_needed) { + device_printf(dev, "alloc msix failed - " + "msi_count=%d, msi_needed=%d, err=%d; " + "will try MSI\n", sc->msi_count, + msi_needed, error); sc->msi_count = 0; + port_qsets = 1; pci_release_msi(dev); bus_release_resource(dev, SYS_RES_MEMORY, sc->msix_regs_rid, sc->msix_regs_res); sc->msix_regs_res = NULL; } else { sc->flags |= USING_MSIX; - sc->cxgb_intr = t3_intr_msix; + sc->cxgb_intr = cxgb_async_intr; + device_printf(dev, + "using MSI-X interrupts (%u vectors)\n", + sc->msi_count); } } if ((msi_allowed >= 1) && (sc->msi_count == 0)) { sc->msi_count = 1; - if (pci_alloc_msi(dev, &sc->msi_count)) { - device_printf(dev, "alloc msi failed - will try INTx\n"); + if ((error = pci_alloc_msi(dev, &sc->msi_count)) != 0) { + device_printf(dev, "alloc msi failed - " + "err=%d; will try INTx\n", error); sc->msi_count = 0; + port_qsets = 1; pci_release_msi(dev); } else { sc->flags |= USING_MSI; - sc->irq_rid = 1; sc->cxgb_intr = t3_intr_msi; + device_printf(dev, "using MSI interrupts\n"); } } #endif if (sc->msi_count == 0) { device_printf(dev, "using line interrupts\n"); - sc->irq_rid = 0; sc->cxgb_intr = t3b_intr; } - if ((sc->flags & USING_MSIX) && !singleq) - port_qsets = min((SGE_QSETS/(sc)->params.nports), mp_ncpus); - /* Create a private taskqueue thread for handling driver events */ #ifdef TASKQUEUE_CURRENT sc->tq = taskqueue_create("cxgb_taskq", M_NOWAIT, @@ -667,6 +655,10 @@ cxgb_controller_attach(device_t dev) sc->params.vpd.ec, sc->params.vpd.sn); device_set_desc_copy(dev, buf); + snprintf(&sc->port_types[0], sizeof(sc->port_types), "%x%x%x%x", + sc->params.vpd.port_type[0], sc->params.vpd.port_type[1], + sc->params.vpd.port_type[2], sc->params.vpd.port_type[3]); + device_printf(sc->dev, "Firmware Version %s\n", &sc->fw_version[0]); callout_reset(&sc->cxgb_tick_ch, CXGB_TICKS(sc), cxgb_tick, sc); t3_add_attach_sysctls(sc); @@ -677,6 +669,11 @@ out: return (error); } +/* + * The cxgb_controller_detach routine is called with the device is + * unloaded from the system. + */ + static int cxgb_controller_detach(device_t dev) { @@ -689,6 +686,24 @@ cxgb_controller_detach(device_t dev) return (0); } +/* + * The cxgb_free() is called by the cxgb_controller_detach() routine + * to tear down the structures that were built up in + * cxgb_controller_attach(), and should be the final piece of work + * done when fully unloading the driver. + * + * + * 1. Shutting down the threads started by the cxgb_controller_attach() + * routine. + * 2. Stopping the lower level device and all callouts (cxgb_down_locked()). + * 3. Detaching all of the port devices created during the + * cxgb_controller_attach() routine. + * 4. Removing the device children created via cxgb_controller_attach(). + * 5. Releasing PCI resources associated with the device. + * 6. Turning off the offload support, iff it was turned on. + * 7. Destroying the mutexes created in cxgb_controller_attach(). + * + */ static void cxgb_free(struct adapter *sc) { @@ -697,14 +712,30 @@ cxgb_free(struct adapter *sc) ADAPTER_LOCK(sc); sc->flags |= CXGB_SHUTDOWN; ADAPTER_UNLOCK(sc); + cxgb_pcpu_shutdown_threads(sc); - ADAPTER_LOCK(sc); -/* - * drops the lock - */ + ADAPTER_LOCK(sc); cxgb_down_locked(sc); + ADAPTER_UNLOCK(sc); + + t3_sge_deinit_sw(sc); + /* + * Wait for last callout + */ + DELAY(hz*100); + + bus_generic_detach(sc->dev); + + for (i = 0; i < (sc)->params.nports; i++) { + if (sc->portdev[i] && + device_delete_child(sc->dev, sc->portdev[i]) != 0) + device_printf(sc->dev, "failed to delete child port\n"); + } + + cxgb_teardown_interrupts(sc); + #ifdef MSI_SUPPORTED if (sc->flags & (USING_MSI | USING_MSIX)) { device_printf(sc->dev, "releasing msi message(s)\n"); @@ -712,25 +743,13 @@ cxgb_free(struct adapter *sc) } else { device_printf(sc->dev, "no msi message to release\n"); } -#endif + if (sc->msix_regs_res != NULL) { bus_release_resource(sc->dev, SYS_RES_MEMORY, sc->msix_regs_rid, sc->msix_regs_res); } +#endif - t3_sge_deinit_sw(sc); - /* - * Wait for last callout - */ - - DELAY(hz*100); - - for (i = 0; i < (sc)->params.nports; ++i) { - if (sc->portdev[i] != NULL) - device_delete_child(sc->dev, sc->portdev[i]); - } - - bus_generic_detach(sc->dev); if (sc->tq != NULL) { taskqueue_free(sc->tq); sc->tq = NULL; @@ -809,88 +828,116 @@ setup_sge_qsets(adapter_t *sc) } static void -cxgb_teardown_msix(adapter_t *sc) +cxgb_teardown_interrupts(adapter_t *sc) { - int i, nqsets; - - for (nqsets = i = 0; i < (sc)->params.nports; i++) - nqsets += sc->port[i].nqsets; + int i; - for (i = 0; i < nqsets; i++) { - if (sc->msix_intr_tag[i] != NULL) { - bus_teardown_intr(sc->dev, sc->msix_irq_res[i], - sc->msix_intr_tag[i]); - sc->msix_intr_tag[i] = NULL; - } - if (sc->msix_irq_res[i] != NULL) { - bus_release_resource(sc->dev, SYS_RES_IRQ, - sc->msix_irq_rid[i], sc->msix_irq_res[i]); - sc->msix_irq_res[i] = NULL; + for (i = 0; i < SGE_QSETS; i++) { + if (sc->msix_intr_tag[i] == NULL) { + + /* Should have been setup fully or not at all */ + KASSERT(sc->msix_irq_res[i] == NULL && + sc->msix_irq_rid[i] == 0, + ("%s: half-done interrupt (%d).", __func__, i)); + + continue; } + + bus_teardown_intr(sc->dev, sc->msix_irq_res[i], + sc->msix_intr_tag[i]); + bus_release_resource(sc->dev, SYS_RES_IRQ, sc->msix_irq_rid[i], + sc->msix_irq_res[i]); + + sc->msix_irq_res[i] = sc->msix_intr_tag[i] = NULL; + sc->msix_irq_rid[i] = 0; } -} -static int -cxgb_setup_msix(adapter_t *sc, int msix_count) -{ - int i, j, k, nqsets, rid; + if (sc->intr_tag) { + KASSERT(sc->irq_res != NULL, + ("%s: half-done interrupt.", __func__)); - /* The first message indicates link changes and error conditions */ - sc->irq_rid = 1; - if ((sc->irq_res = bus_alloc_resource_any(sc->dev, SYS_RES_IRQ, - &sc->irq_rid, RF_SHAREABLE | RF_ACTIVE)) == NULL) { - device_printf(sc->dev, "Cannot allocate msix interrupt\n"); - return (EINVAL); + bus_teardown_intr(sc->dev, sc->irq_res, sc->intr_tag); + bus_release_resource(sc->dev, SYS_RES_IRQ, sc->irq_rid, + sc->irq_res); + + sc->irq_res = sc->intr_tag = NULL; + sc->irq_rid = 0; } +} - if (bus_setup_intr(sc->dev, sc->irq_res, INTR_MPSAFE|INTR_TYPE_NET, +static int +cxgb_setup_interrupts(adapter_t *sc) +{ + struct resource *res; + void *tag; + int i, rid, err, intr_flag = sc->flags & (USING_MSI | USING_MSIX); + + sc->irq_rid = intr_flag ? 1 : 0; + sc->irq_res = bus_alloc_resource_any(sc->dev, SYS_RES_IRQ, &sc->irq_rid, + RF_SHAREABLE | RF_ACTIVE); + if (sc->irq_res == NULL) { + device_printf(sc->dev, "Cannot allocate interrupt (%x, %u)\n", + intr_flag, sc->irq_rid); + err = EINVAL; + sc->irq_rid = 0; + } else { + err = bus_setup_intr(sc->dev, sc->irq_res, + INTR_MPSAFE | INTR_TYPE_NET, #ifdef INTR_FILTERS - NULL, + NULL, #endif - cxgb_async_intr, sc, &sc->intr_tag)) { - device_printf(sc->dev, "Cannot set up interrupt\n"); - return (EINVAL); + sc->cxgb_intr, sc, &sc->intr_tag); + + if (err) { + device_printf(sc->dev, + "Cannot set up interrupt (%x, %u, %d)\n", + intr_flag, sc->irq_rid, err); + bus_release_resource(sc->dev, SYS_RES_IRQ, sc->irq_rid, + sc->irq_res); + sc->irq_res = sc->intr_tag = NULL; + sc->irq_rid = 0; + } } - for (i = k = 0; i < (sc)->params.nports; i++) { - nqsets = sc->port[i].nqsets; - for (j = 0; j < nqsets; j++, k++) { - struct sge_qset *qs = &sc->sge.qs[k]; - - rid = k + 2; - if (cxgb_debug) - printf("rid=%d ", rid); - if ((sc->msix_irq_res[k] = bus_alloc_resource_any( - sc->dev, SYS_RES_IRQ, &rid, - RF_SHAREABLE | RF_ACTIVE)) == NULL) { - device_printf(sc->dev, "Cannot allocate " - "interrupt for message %d\n", rid); - return (EINVAL); - } - sc->msix_irq_rid[k] = rid; - if (bus_setup_intr(sc->dev, sc->msix_irq_res[k], - INTR_MPSAFE|INTR_TYPE_NET, + + /* That's all for INTx or MSI */ + if (!(intr_flag & USING_MSIX) || err) + return (err); + + for (i = 0; i < sc->msi_count - 1; i++) { + rid = i + 2; + res = bus_alloc_resource_any(sc->dev, SYS_RES_IRQ, &rid, + RF_SHAREABLE | RF_ACTIVE); + if (res == NULL) { + device_printf(sc->dev, "Cannot allocate interrupt " + "for message %d\n", rid); + err = EINVAL; + break; + } + + err = bus_setup_intr(sc->dev, res, INTR_MPSAFE | INTR_TYPE_NET, #ifdef INTR_FILTERS - NULL, + NULL, #endif - t3_intr_msix, qs, &sc->msix_intr_tag[k])) { - device_printf(sc->dev, "Cannot set up " - "interrupt for message %d\n", rid); - return (EINVAL); - } -#ifdef IFNET_MULTIQUEUE - if (singleq == 0) { - int vector = rman_get_start(sc->msix_irq_res[k]); - if (bootverbose) - device_printf(sc->dev, "binding vector=%d to cpu=%d\n", vector, k % mp_ncpus); - intr_bind(vector, k % mp_ncpus); - } -#endif + t3_intr_msix, &sc->sge.qs[i], &tag); + if (err) { + device_printf(sc->dev, "Cannot set up interrupt " + "for message %d (%d)\n", rid, err); + bus_release_resource(sc->dev, SYS_RES_IRQ, rid, res); + break; } + + sc->msix_irq_rid[i] = rid; + sc->msix_irq_res[i] = res; + sc->msix_intr_tag[i] = tag; } - return (0); + if (err) + cxgb_teardown_interrupts(sc); + + return (err); } + static int cxgb_port_probe(device_t dev) { @@ -921,6 +968,12 @@ cxgb_makedev(struct port_info *pi) return (0); } +#ifndef LRO_SUPPORTED +#ifdef IFCAP_LRO +#undef IFCAP_LRO +#endif +#define IFCAP_LRO 0x0 +#endif #ifdef TSO_SUPPORTED #define CXGB_CAP (IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_MTU | IFCAP_HWCSUM | IFCAP_VLAN_HWCSUM | IFCAP_TSO | IFCAP_JUMBO_MTU | IFCAP_LRO) @@ -968,16 +1021,11 @@ cxgb_port_attach(device_t dev) ifp->if_ioctl = cxgb_ioctl; ifp->if_start = cxgb_start; -#if 0 -#ifdef IFNET_MULTIQUEUE - ifp->if_flags |= IFF_MULTIQ; - ifp->if_mq_start = cxgb_pcpu_start; -#endif -#endif + ifp->if_timer = 0; /* Disable ifnet watchdog */ ifp->if_watchdog = NULL; - ifp->if_snd.ifq_drv_maxlen = cxgb_snd_queue_len; + ifp->if_snd.ifq_drv_maxlen = IFQ_MAXLEN; IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen); IFQ_SET_READY(&ifp->if_snd); @@ -995,6 +1043,10 @@ cxgb_port_attach(device_t dev) } ether_ifattach(ifp, p->hw_addr); + +#ifdef IFNET_MULTIQUEUE + ifp->if_transmit = cxgb_pcpu_transmit; +#endif /* * Only default to jumbo frames on 10GigE */ @@ -1055,40 +1107,60 @@ cxgb_port_attach(device_t dev) bcopy(IF_LLADDR(p->ifp), p->hw_addr, ETHER_ADDR_LEN); t3_sge_init_port(p); - TASK_INIT(&p->link_fault_task, 0, cxgb_link_fault, p); - #if defined(LINK_ATTACH) cxgb_link_start(p); t3_link_changed(sc, p->port_id); #endif - return (0); + + return (err); } +/* + * cxgb_port_detach() is called via the device_detach methods when + * cxgb_free() calls the bus_generic_detach. It is responsible for + * removing the device from the view of the kernel, i.e. from all + * interfaces lists etc. This routine is only called when the driver is + * being unloaded, not when the link goes down. + * + */ static int cxgb_port_detach(device_t dev) { struct port_info *p; + struct adapter *sc; p = device_get_softc(dev); + sc = p->adapter; + + if (p->port_cdev != NULL) + destroy_dev(p->port_cdev); + + ether_ifdetach(p->ifp); PORT_LOCK(p); if (p->ifp->if_drv_flags & IFF_DRV_RUNNING) cxgb_stop_locked(p); PORT_UNLOCK(p); - ether_ifdetach(p->ifp); - printf("waiting for callout to stop ..."); - DELAY(1000000); - printf("done\n"); + callout_drain(&sc->cxgb_tick_ch); + callout_drain(&sc->sge_timer_ch); + + if (sc->tq != NULL) { + printf("draining slow intr\n"); + + taskqueue_drain(sc->tq, &sc->slow_intr_task); + printf("draining ext intr\n"); + taskqueue_drain(sc->tq, &sc->ext_intr_task); + printf("draining tick task\n"); + taskqueue_drain(sc->tq, &sc->tick_task); + } + /* * the lock may be acquired in ifdetach */ *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***