Date: Mon, 3 Oct 2016 23:49:06 +0000 (UTC) From: John Baldwin <jhb@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r306664 - in stable/11: share/man/man4 sys/arm/conf sys/conf sys/dev/cxgbe sys/dev/cxgbe/common sys/dev/cxgbe/iw_cxgbe sys/modules/cxgbe sys/modules/cxgbe/if_cxgbev sys/modules/cxgbe/if... Message-ID: <201610032349.u93Nn6Ld068983@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: jhb Date: Mon Oct 3 23:49:05 2016 New Revision: 306664 URL: https://svnweb.freebsd.org/changeset/base/306664 Log: MFC 303522,303647,303860,303880,304168,304169,304170,304479,304485,305549: Chelsio T4/T5 VF driver. 303522: Various fixes to the t4/5nex character device. - Remove null open/close methods. - Don't set d_flags to 0 explicitly. - Remove t5_cdevsw as the .d_name member isn't really used and doesn't warrant a separate cdevsw just for the name. - Use ENOTTY as the error value for an unknown ioctl request. - Use make_dev_s() to close race with setting si_drv1. 303647: Store the offset of the KDOORBELL and GTS registers in the softc. VF devices use a different register layout than PF devices. Storing the offset in a value in the softc allows code to be shared between the PF and VF drivers. 303860: Reserve an adapter flag IS_VF to mark VF devices vs PF devices. 303880: Track the base absolute ID of ingress and egress queues. Use this to map an absolute queue ID to a logical queue ID in interrupt handlers. For the regular cxgbe/cxl drivers this should be a no-op as the base absolute ID should be zero. VF devices have a non-zero base absolute ID and require this change. While here, export the absolute ID of egress queues via a sysctl. 304168: Make SGE parameter handling more VF-friendly. Add fields to hold the SGE control register and free list buffer sizes to the sge_params structure. Populate these new fields in t4_init_sge_params() for PF devices and change t4_read_chip_settings() to pull these values out of the params structure instead of reading registers directly. This will permit t4_read_chip_settings() to be reused for VF devices which cannot read SGE registers directly. While here, move the call to t4_init_sge_params() to get_params__post_init(). The VF driver will populate the SGE parameters structure via a different method before calling t4_read_chip_settings(). 304169: Update mailbox writes to work with VF devices. - Use alternate register locations for the data and control registers for VFs. - Do a dummy read to force the writes to the mailbox data registers to post before the write to the control register on VFs. - Do not check the PCI-e firmware register for errors on VFs. 304170: Add support for register dumps on VF devices. - Add handling of VF register sets to t4_get_regs_len() and t4_get_regs(). - While here, use t4_get_regs_len() in the ioctl handler for regdump instead of inlining it. 304479: Add structures for VF-specific adapter parameters. While here, mark which parameters are PF-specific and which are VF-specific. 304485: Reorder sysctls so that nodes shared with the VF driver are added first. This permits a single early return for VF devices in the routines that add sysctl nodes. 305549: Chelsio T4/T5 VF driver. The cxgbev/cxlv driver supports Virtual Function devices for Chelsio T4 and T4 adapters. The VF devices share most of their code with the existing PF4 driver (cxgbe/cxl) and as such the VF device driver currently depends on the PF4 driver. Similar to the cxgbe/cxl drivers, the VF driver includes a t4vf/t5vf PCI device driver that attaches to the VF device. It then creates child cxgbev/cxlv devices representing ports assigned to the VF. By default, the PF driver assigns a single port to each VF. t4vf_hw.c contains VF-specific routines from the shared code used to fetch VF-specific parameters from the firmware. t4_vf.c contains the VF-specific PCI device driver and includes its own attach routine. VF devices are required to use a different firmware request when transmitting packets (which in turn requires a different CPL message to encapsulate messages). This alternate firmware request does not permit chaining multiple packets in a single message, so each packet results in a firmware request. In addition, the different CPL message requires more detailed information when enabling hardware checksums, so parse_pkt() on VF devices must examine L2 and L3 headers for all packets (not just TSO packets) for VF devices. Finally, L2 checksums on non-UDP/non-TCP packets do not work reliably (the firmware trashes the IPv4 fragment field), so IPv4 checksums for such packets are calculated in software. Most of the other changes in the non-VF-specific code are to expose various variables and functions private to the PF driver so that they can be used by the VF driver. Note that a limited subset of cxgbetool functions are supported on VF devices including register dumps, scheduler classes, and clearing of statistics. In addition, TOE is not supported on VF devices, only for the PF interfaces. Sponsored by: Chelsio Communications Added: stable/11/share/man/man4/cxgbev.4 - copied unchanged from r305549, head/share/man/man4/cxgbev.4 stable/11/sys/dev/cxgbe/common/t4vf_hw.c - copied unchanged from r305549, head/sys/dev/cxgbe/common/t4vf_hw.c stable/11/sys/dev/cxgbe/if_cxlv.c - copied unchanged from r305549, head/sys/dev/cxgbe/if_cxlv.c stable/11/sys/dev/cxgbe/t4_vf.c - copied unchanged from r305549, head/sys/dev/cxgbe/t4_vf.c stable/11/sys/modules/cxgbe/if_cxgbev/ - copied from r305549, head/sys/modules/cxgbe/if_cxgbev/ stable/11/sys/modules/cxgbe/if_cxlv/ - copied from r305549, head/sys/modules/cxgbe/if_cxlv/ Modified: stable/11/share/man/man4/Makefile stable/11/share/man/man4/cxgbe.4 stable/11/sys/arm/conf/NOTES stable/11/sys/conf/NOTES stable/11/sys/conf/files stable/11/sys/dev/cxgbe/adapter.h stable/11/sys/dev/cxgbe/common/common.h stable/11/sys/dev/cxgbe/common/t4_hw.c stable/11/sys/dev/cxgbe/iw_cxgbe/cq.c stable/11/sys/dev/cxgbe/iw_cxgbe/qp.c stable/11/sys/dev/cxgbe/t4_main.c stable/11/sys/dev/cxgbe/t4_netmap.c stable/11/sys/dev/cxgbe/t4_sge.c stable/11/sys/modules/cxgbe/Makefile stable/11/sys/powerpc/conf/NOTES Directory Properties: stable/11/ (props changed) Modified: stable/11/share/man/man4/Makefile ============================================================================== --- stable/11/share/man/man4/Makefile Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/share/man/man4/Makefile Mon Oct 3 23:49:05 2016 (r306664) @@ -113,6 +113,7 @@ MAN= aac.4 \ cue.4 \ cxgb.4 \ cxgbe.4 \ + cxgbev.4 \ cy.4 \ cyapa.4 \ da.4 \ @@ -607,6 +608,9 @@ MLINKS+=cxgb.4 if_cxgb.4 MLINKS+=cxgbe.4 if_cxgbe.4 \ cxgbe.4 cxl.4 \ cxgbe.4 if_cxl.4 +MLINKS+=cxgbev.4 if_cxgbev.4 \ + cxgbev.4 cxlv.4 \ + cxgbev.4 if_cxlv.4 MLINKS+=dc.4 if_dc.4 MLINKS+=de.4 if_de.4 MLINKS+=disc.4 if_disc.4 Modified: stable/11/share/man/man4/cxgbe.4 ============================================================================== --- stable/11/share/man/man4/cxgbe.4 Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/share/man/man4/cxgbe.4 Mon Oct 3 23:49:05 2016 (r306664) @@ -77,8 +77,7 @@ For more information on configuring this .Sh HARDWARE The .Nm -driver supports 40Gb, 10Gb and 1Gb Ethernet adapters based on the T5 ASIC -(ports will be named cxl): +driver supports 40Gb, 10Gb and 1Gb Ethernet adapters based on the T5 ASIC: .Pp .Bl -bullet -compact .It @@ -320,6 +319,7 @@ email all the specific information relat .Xr altq 4 , .Xr arp 4 , .Xr cxgb 4 , +.Xr cxgbev 4 , .Xr netintro 4 , .Xr ng_ether 4 , .Xr ifconfig 8 Copied: stable/11/share/man/man4/cxgbev.4 (from r305549, head/share/man/man4/cxgbev.4) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/share/man/man4/cxgbev.4 Mon Oct 3 23:49:05 2016 (r306664, copy of r305549, head/share/man/man4/cxgbev.4) @@ -0,0 +1,290 @@ +.\" Copyright (c) 2011-2016, Chelsio Inc +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are met: +.\" +.\" 1. Redistributions of source code must retain the above copyright notice, +.\" this list of conditions and the following disclaimer. +.\" +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" 3. Neither the name of the Chelsio Inc nor the names of its +.\" contributors may be used to endorse or promote products derived from +.\" this software without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +.\" POSSIBILITY OF SUCH DAMAGE. +.\" +.\" * Other names and brands may be claimed as the property of others. +.\" +.\" $FreeBSD$ +.\" +.Dd August 22, 2016 +.Dt CXGBEV 4 +.Os +.Sh NAME +.Nm cxgbev +.Nd "Chelsio T4 and T5 based 40Gb, 10Gb, and 1Gb Ethernet VF driver" +.Sh SYNOPSIS +To compile this driver into the kernel, +place the following lines in your +kernel configuration file: +.Bd -ragged -offset indent +.Cd "device cxgbe" +.Cd "device cxgbev" +.Ed +.Pp +To load the driver as a +module at boot time, place the following line in +.Xr loader.conf 5 : +.Bd -literal -offset indent +if_cxgbev_load="YES" +.Ed +.Sh DESCRIPTION +The +.Nm +driver provides support for Virtual Functions on PCI Express Ethernet adapters +based on the Chelsio Terminator 4 and Terminator 5 ASICs (T4 and T5). +The driver supports Jumbo Frames, Transmit/Receive checksum offload, +TCP segmentation offload (TSO), Large Receive Offload (LRO), VLAN +tag insertion/extraction, VLAN checksum offload, VLAN TSO, and +Receive Side Steering (RSS). +For further hardware information and questions related to hardware +requirements, see +.Pa http://www.chelsio.com/ . +.Pp +Note that ports of T5 VFs are named cxlv and attach to a t5vf parent device +(in contrast to ports named cxgbev that attach to a t4vf parent for a T4 VF). +Loader tunables with the hw.cxgbe prefix apply to both T4 and T5 VFs. +The Physical Function driver for T4 and T5 adapters shares these tunables. +The sysctl MIBs are at dev.t5vf and dev.cxlv for T5 cards and at dev.t4vf and +dev.cxgbev for T4 cards. +.Pp +For more information on configuring this device, see +.Xr ifconfig 8 . +.Sh HARDWARE +The +.Nm +driver supports Virtual Functions on 40Gb, 10Gb and 1Gb Ethernet adapters +based on the T5 ASIC: +.Pp +.Bl -bullet -compact +.It +Chelsio T580-CR +.It +Chelsio T580-LP-CR +.It +Chelsio T580-LP-SO-CR +.It +Chelsio T560-CR +.It +Chelsio T540-CR +.It +Chelsio T540-LP-CR +.It +Chelsio T522-CR +.It +Chelsio T520-LL-CR +.It +Chelsio T520-CR +.It +Chelsio T520-SO +.It +Chelsio T520-BT +.It +Chelsio T504-BT +.El +.Pp +The +.Nm +driver supports Virtual Functions on 10Gb and 1Gb Ethernet adapters based +on the T4 ASIC: +.Pp +.Bl -bullet -compact +.It +Chelsio T420-CR +.It +Chelsio T422-CR +.It +Chelsio T440-CR +.It +Chelsio T420-BCH +.It +Chelsio T440-BCH +.It +Chelsio T440-CH +.It +Chelsio T420-SO +.It +Chelsio T420-CX +.It +Chelsio T420-BT +.It +Chelsio T404-BT +.El +.Sh LOADER TUNABLES +Tunables can be set at the +.Xr loader 8 +prompt before booting the kernel or stored in +.Xr loader.conf 5 . +.Bl -tag -width indent +.It Va hw.cxgbe.ntxq10g +The number of tx queues to use for a 10Gb or 40Gb port. +The default is 16 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nrxq10g +The number of rx queues to use for a 10Gb or 40Gb port. +The default is 8 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.ntxq1g +The number of tx queues to use for a 1Gb port. +The default is 4 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nrxq1g +The number of rx queues to use for a 1Gb port. +The default is 2 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.holdoff_timer_idx_10G +.It Va hw.cxgbe.holdoff_timer_idx_1G +The timer index value to use to delay interrupts. +The holdoff timer list has the values 1, 5, 10, 50, 100, and 200 +by default (all values are in microseconds) and the index selects a +value from this list. +The default value is 1 which means the timer value is 5us. +Different interfaces can be assigned different values at any time via the +dev.cxgbev.X.holdoff_tmr_idx or dev.cxlv.X.holdoff_tmr_idx sysctl. +.It Va hw.cxgbe.holdoff_pktc_idx_10G +.It Va hw.cxgbe.holdoff_pktc_idx_1G +The packet-count index value to use to delay interrupts. +The packet-count list has the values 1, 8, 16, and 32 by default +and the index selects a value from this list. +The default value is -1 which means packet counting is disabled and interrupts +are generated based solely on the holdoff timer value. +Different interfaces can be assigned different values via the +dev.cxgbev.X.holdoff_pktc_idx or dev.cxlv.X.holdoff_pktc_idx sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). +.It Va hw.cxgbe.qsize_txq +The size, in number of entries, of the descriptor ring used for a tx +queue. +A buf_ring of the same size is also allocated for additional +software queuing. +See +.Xr ifnet 9 . +The default value is 1024. +Different interfaces can be assigned different values via the +dev.cxgbev.X.qsize_txq sysctl or dev.cxlv.X.qsize_txq sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). +.It Va hw.cxgbe.qsize_rxq +The size, in number of entries, of the descriptor ring used for an +rx queue. +The default value is 1024. +Different interfaces can be assigned different values via the +dev.cxgbev.X.qsize_rxq or dev.cxlv.X.qsize_rxq sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). +.It Va hw.cxgbe.interrupt_types +The interrupt types that the driver is allowed to use. +Bit 0 represents INTx (line interrupts), bit 1 MSI, bit 2 MSI-X. +The default is 7 (all allowed). +The driver will select the best possible type out of the allowed types by +itself. +.It Va hw.cxgbe.fl_pktshift +The number of bytes of padding inserted before the beginning of an Ethernet +frame in the receive buffer. +The default value of 2 ensures that the Ethernet payload (usually the IP header) +is at a 4 byte aligned address. +0-7 are all valid values. +.It Va hw.cxgbe.fl_pad +A non-zero value ensures that writes from the hardware to a receive buffer are +padded up to the specified boundary. +The default is -1 which lets the driver pick a pad boundary. +0 disables trailer padding completely. +.It Va hw.cxgbe.buffer_packing +Allow the hardware to deliver multiple frames in the same receive buffer +opportunistically. +The default is -1 which lets the driver decide. +0 or 1 explicitly disable or enable this feature. +.It Va hw.cxgbe.allow_mbufs_in_cluster +1 allows the driver to lay down one or more mbufs within the receive buffer +opportunistically. +This is the default. +0 prohibits the driver from doing so. +.It Va hw.cxgbe.largest_rx_cluster +.It Va hw.cxgbe.safest_rx_cluster +Sizes of rx clusters. +Each of these must be set to one of the sizes available +(usually 2048, 4096, 9216, and 16384) and largest_rx_cluster must be greater +than or equal to safest_rx_cluster. +The defaults are 16384 and 4096 respectively. +The driver will never attempt to allocate a receive buffer larger than +largest_rx_cluster and will fall back to allocating buffers of +safest_rx_cluster size if an allocation larger than safest_rx_cluster fails. +Note that largest_rx_cluster merely establishes a ceiling -- the driver is +allowed to allocate buffers of smaller sizes. +.El +.Pp +Certain settings and resources for Virtual Functions are dictated +by the parent Physical Function driver. +For example, the Physical Function driver limits the number of queues a +Virtual Function is permitted to use. +Some of these limits can be adjusted in the firmware configuration file +used with the Physical Function driver. +.Pp +The PAUSE settings on the port of a Virtual Function are inherited from +the settings of the same port on the Physical Function. +Virtual Functions cannot modify the setting and track changes made to +the associated port's setting by the Physical Function driver. +.Pp +Receive queues on a Virtual Function always drop packets in response to +congestion +.Po +equivalent to setting +.Va hw.cxgbe.cong_drop +to 1 +.Pc . +.Pp +The VF driver currently depends on the PF driver. +As a result, loading the VF driver will also load the PF driver as a +dependency. +.Sh SUPPORT +For general information and support, +go to the Chelsio support website at: +.Pa http://www.chelsio.com/ . +.Pp +If an issue is identified with this driver with a supported adapter, +email all the specific information related to the issue to +.Aq Mt support@chelsio.com . +.Sh SEE ALSO +.Xr altq 4 , +.Xr arp 4 , +.Xr cxgbe 4 , +.Xr netintro 4 , +.Xr ng_ether 4 , +.Xr ifconfig 8 +.Sh HISTORY +The +.Nm +device driver first appeared in +.Fx 12.0 . +.Sh AUTHORS +.An -nosplit +The +.Nm +driver was written by +.An Navdeep Parhar Aq Mt np@FreeBSD.org +and +.An John Baldwin Aq Mt jhb@FreeBSD.org . Modified: stable/11/sys/arm/conf/NOTES ============================================================================== --- stable/11/sys/arm/conf/NOTES Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/arm/conf/NOTES Mon Oct 3 23:49:05 2016 (r306664) @@ -80,6 +80,7 @@ nodevice star_saver nodevice warp_saver nodevice cxgbe +nodevice cxgbev nodevice snd_cmi # Modified: stable/11/sys/conf/NOTES ============================================================================== --- stable/11/sys/conf/NOTES Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/conf/NOTES Mon Oct 3 23:49:05 2016 (r306664) @@ -1949,6 +1949,7 @@ device xmphy # XaQti XMAC II # (and SMC COM90c66 in '56 compatibility mode) adapters. # cxgb: Chelsio T3 based 1GbE/10GbE PCIe Ethernet adapters. # cxgbe:Chelsio T4 and T5 based 1GbE/10GbE/40GbE PCIe Ethernet adapters. +# cxgbev: Chelsio T4 and T5 based PCIe Virtual Functions. # dc: Support for PCI fast ethernet adapters based on the DEC/Intel 21143 # and various workalikes including: # the ADMtek AL981 Comet and AN985 Centaur, the ASIX Electronics @@ -2132,6 +2133,7 @@ device wb # Winbond W89C840F device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # PCI Ethernet NICs. +device cxgbev # Chelsio T4 and T5 1GbE/10GbE/40GbE VF device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel Pro/1000 Gigabit Ethernet device igb # Intel Pro/1000 PCIE Gigabit Ethernet Modified: stable/11/sys/conf/files ============================================================================== --- stable/11/sys/conf/files Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/conf/files Mon Oct 3 23:49:05 2016 (r306664) @@ -1258,8 +1258,12 @@ dev/cxgbe/t4_l2t.c optional cxgbe pci \ compile-with "${NORMAL_C} -I$S/dev/cxgbe" dev/cxgbe/t4_tracer.c optional cxgbe pci \ compile-with "${NORMAL_C} -I$S/dev/cxgbe" +dev/cxgbe/t4_vf.c optional cxgbev pci \ + compile-with "${NORMAL_C} -I$S/dev/cxgbe" dev/cxgbe/common/t4_hw.c optional cxgbe pci \ compile-with "${NORMAL_C} -I$S/dev/cxgbe" +dev/cxgbe/common/t4vf_hw.c optional cxgbev pci \ + compile-with "${NORMAL_C} -I$S/dev/cxgbe" t4fw_cfg.c optional cxgbe \ compile-with "${AWK} -f $S/tools/fw_stub.awk t4fw_cfg.fw:t4fw_cfg t4fw_cfg_uwire.fw:t4fw_cfg_uwire t4fw.fw:t4fw -mt4fw_cfg -c${.TARGET}" \ no-implicit-rule before-depend local \ Modified: stable/11/sys/dev/cxgbe/adapter.h ============================================================================== --- stable/11/sys/dev/cxgbe/adapter.h Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/dev/cxgbe/adapter.h Mon Oct 3 23:49:05 2016 (r306664) @@ -195,6 +195,7 @@ enum { ADAP_SYSCTL_CTX = (1 << 4), /* TOM_INIT_DONE= (1 << 5), No longer used */ BUF_PACKING_OK = (1 << 6), + IS_VF = (1 << 7), CXGBE_BUSY = (1 << 9), @@ -429,6 +430,7 @@ enum {DOORBELL_UDB, DOORBELL_WCWR, DOORB struct sge_eq { unsigned int flags; /* MUST be first */ unsigned int cntxt_id; /* SGE context id for the eq */ + unsigned int abs_id; /* absolute SGE id for the eq */ struct mtx eq_lock; struct tx_desc *desc; /* KVA of descriptor ring */ @@ -737,8 +739,10 @@ struct sge { struct sge_nm_txq *nm_txq; /* netmap tx queues */ struct sge_nm_rxq *nm_rxq; /* netmap rx queues */ - uint16_t iq_start; - int eq_start; + uint16_t iq_start; /* first cntxt_id */ + uint16_t iq_base; /* first abs_id */ + int eq_start; /* first cntxt_id */ + int eq_base; /* first abs_id */ struct sge_iq **iqmap; /* iq->cntxt_id to iq mapping */ struct sge_eq **eqmap; /* eq->cntxt_id to eq mapping */ @@ -781,6 +785,8 @@ struct adapter { struct sge_rxq *rxq; struct sge_nm_rxq *nm_rxq; } __aligned(CACHE_LINE_SIZE) *irq; + int sge_gts_reg; + int sge_kdoorbell_reg; bus_dma_tag_t dmat; /* Parent DMA tag */ @@ -938,6 +944,9 @@ struct adapter { /* One for errors, one for firmware events */ #define T4_EXTRA_INTR 2 +/* One for firmware events */ +#define T4VF_EXTRA_INTR 1 + static inline uint32_t t4_read_reg(struct adapter *sc, uint32_t reg) { @@ -1072,13 +1081,34 @@ t4_use_ldst(struct adapter *sc) } /* t4_main.c */ +extern int t4_ntxq10g; +extern int t4_nrxq10g; +extern int t4_ntxq1g; +extern int t4_nrxq1g; +extern int t4_intr_types; +extern int t4_tmr_idx_10g; +extern int t4_pktc_idx_10g; +extern int t4_tmr_idx_1g; +extern int t4_pktc_idx_1g; +extern unsigned int t4_qsize_rxq; +extern unsigned int t4_qsize_txq; +extern device_method_t cxgbe_methods[]; + int t4_os_find_pci_capability(struct adapter *, int); int t4_os_pci_save_state(struct adapter *); int t4_os_pci_restore_state(struct adapter *); void t4_os_portmod_changed(const struct adapter *, int); void t4_os_link_changed(struct adapter *, int, int, int); void t4_iterate(void (*)(struct adapter *, void *), void *); +void t4_add_adapter(struct adapter *); +int t4_detach_common(device_t); int t4_filter_rpl(struct sge_iq *, const struct rss_header *, struct mbuf *); +int t4_map_bars_0_and_4(struct adapter *); +int t4_map_bar_2(struct adapter *); +int t4_set_sched_class(struct adapter *, struct t4_sched_params *); +int t4_set_sched_queue(struct adapter *, struct t4_sched_queue *); +int t4_setup_intr_handlers(struct adapter *); +void t4_sysctls(struct adapter *); int begin_synchronized_op(struct adapter *, struct vi_info *, int, char *); void doom_vi(struct adapter *, struct vi_info *); void end_synchronized_op(struct adapter *, int); @@ -1119,7 +1149,7 @@ void t4_intr_err(void *); void t4_intr_evt(void *); void t4_wrq_tx_locked(struct adapter *, struct sge_wrq *, struct wrqe *); void t4_update_fl_bufsize(struct ifnet *); -int parse_pkt(struct mbuf **); +int parse_pkt(struct adapter *, struct mbuf **); void *start_wrq_wr(struct sge_wrq *, int, struct wrq_cookie *); void commit_wrq_wr(struct sge_wrq *, void *, struct wrq_cookie *); int tnl_cong(struct port_info *, int); Modified: stable/11/sys/dev/cxgbe/common/common.h ============================================================================== --- stable/11/sys/dev/cxgbe/common/common.h Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/dev/cxgbe/common/common.h Mon Oct 3 23:49:05 2016 (r306664) @@ -215,6 +215,8 @@ struct sge_params { int pad_boundary; int pack_boundary; int fl_pktshift; + u32 sge_control; + u32 sge_fl_buffer_size[SGE_FLBUF_SIZES]; }; struct tp_params { @@ -278,12 +280,52 @@ struct chip_params { u16 mps_tcam_size; }; +/* VF-only parameters. */ + +/* + * Global Receive Side Scaling (RSS) parameters in host-native format. + */ +struct rss_params { + unsigned int mode; /* RSS mode */ + union { + struct { + u_int synmapen:1; /* SYN Map Enable */ + u_int syn4tupenipv6:1; /* enable hashing 4-tuple IPv6 SYNs */ + u_int syn2tupenipv6:1; /* enable hashing 2-tuple IPv6 SYNs */ + u_int syn4tupenipv4:1; /* enable hashing 4-tuple IPv4 SYNs */ + u_int syn2tupenipv4:1; /* enable hashing 2-tuple IPv4 SYNs */ + u_int ofdmapen:1; /* Offload Map Enable */ + u_int tnlmapen:1; /* Tunnel Map Enable */ + u_int tnlalllookup:1; /* Tunnel All Lookup */ + u_int hashtoeplitz:1; /* use Toeplitz hash */ + } basicvirtual; + } u; +}; + +/* + * Maximum resources provisioned for a PCI VF. + */ +struct vf_resources { + unsigned int nvi; /* N virtual interfaces */ + unsigned int neq; /* N egress Qs */ + unsigned int nethctrl; /* N egress ETH or CTRL Qs */ + unsigned int niqflint; /* N ingress Qs/w free list(s) & intr */ + unsigned int niq; /* N ingress Qs */ + unsigned int tc; /* PCI-E traffic class */ + unsigned int pmask; /* port access rights mask */ + unsigned int nexactf; /* N exact MPS filters */ + unsigned int r_caps; /* read capabilities */ + unsigned int wx_caps; /* write/execute capabilities */ +}; + struct adapter_params { struct sge_params sge; - struct tp_params tp; + struct tp_params tp; /* PF-only */ struct vpd_params vpd; struct pci_params pci; - struct devlog_params devlog; + struct devlog_params devlog; /* PF-only */ + struct rss_params rss; /* VF-only */ + struct vf_resources vfres; /* VF-only */ unsigned int sf_size; /* serial flash size in bytes */ unsigned int sf_nsec; /* # of flash sectors */ @@ -509,6 +551,7 @@ int t4_get_fw_version(struct adapter *ad int t4_get_tp_version(struct adapter *adapter, u32 *vers); int t4_get_exprom_version(struct adapter *adapter, u32 *vers); int t4_init_hw(struct adapter *adapter, u32 fw_params); +const struct chip_params *t4_get_chip_params(int chipid); int t4_prep_adapter(struct adapter *adapter, u8 *buf); int t4_shutdown_adapter(struct adapter *adapter); int t4_init_devlog_params(struct adapter *adapter, int fw_attach); @@ -716,4 +759,32 @@ int t4_config_watchdog(struct adapter *a int t4_get_devlog_level(struct adapter *adapter, unsigned int *level); int t4_set_devlog_level(struct adapter *adapter, unsigned int level); void t4_sge_decode_idma_state(struct adapter *adapter, int state); + +static inline int t4vf_query_params(struct adapter *adapter, + unsigned int nparams, const u32 *params, + u32 *vals) +{ + return t4_query_params(adapter, 0, 0, 0, nparams, params, vals); +} + +static inline int t4vf_set_params(struct adapter *adapter, + unsigned int nparams, const u32 *params, + const u32 *vals) +{ + return t4_set_params(adapter, 0, 0, 0, nparams, params, vals); +} + +static inline int t4vf_wr_mbox(struct adapter *adap, const void *cmd, + int size, void *rpl) +{ + return t4_wr_mbox(adap, adap->mbox, cmd, size, rpl); +} + +int t4vf_wait_dev_ready(struct adapter *adapter); +int t4vf_fw_reset(struct adapter *adapter); +int t4vf_get_sge_params(struct adapter *adapter); +int t4vf_get_rss_glb_config(struct adapter *adapter); +int t4vf_get_vfres(struct adapter *adapter); +int t4vf_prep_adapter(struct adapter *adapter); + #endif /* __CHELSIO_COMMON_H */ Modified: stable/11/sys/dev/cxgbe/common/t4_hw.c ============================================================================== --- stable/11/sys/dev/cxgbe/common/t4_hw.c Mon Oct 3 23:17:57 2016 (r306663) +++ stable/11/sys/dev/cxgbe/common/t4_hw.c Mon Oct 3 23:49:05 2016 (r306664) @@ -289,6 +289,14 @@ int t4_wr_mbox_meat_timeout(struct adapt if ((size & 15) || size > MBOX_LEN) return -EINVAL; + if (adap->flags & IS_VF) { + if (is_t6(adap)) + data_reg = FW_T6VF_MBDATA_BASE_ADDR; + else + data_reg = FW_T4VF_MBDATA_BASE_ADDR; + ctl_reg = VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_CTRL); + } + /* * If we have a negative timeout, that implies that we can't sleep. */ @@ -343,6 +351,22 @@ int t4_wr_mbox_meat_timeout(struct adapt for (i = 0; i < size; i += 8, p++) t4_write_reg64(adap, data_reg + i, be64_to_cpu(*p)); + if (adap->flags & IS_VF) { + /* + * For the VFs, the Mailbox Data "registers" are + * actually backed by T4's "MA" interface rather than + * PL Registers (as is the case for the PFs). Because + * these are in different coherency domains, the write + * to the VF's PL-register-backed Mailbox Control can + * race in front of the writes to the MA-backed VF + * Mailbox Data "registers". So we need to do a + * read-back on at least one byte of the VF Mailbox + * Data registers before doing the write to the VF + * Mailbox Control register. + */ + t4_read_reg(adap, data_reg); + } + CH_DUMP_MBOX(adap, mbox, data_reg); t4_write_reg(adap, ctl_reg, F_MBMSGVALID | V_MBOWNER(X_MBOWNER_FW)); @@ -355,10 +379,13 @@ int t4_wr_mbox_meat_timeout(struct adapt * Loop waiting for the reply; bail out if we time out or the firmware * reports an error. */ - for (i = 0; - !((pcie_fw = t4_read_reg(adap, A_PCIE_FW)) & F_PCIE_FW_ERR) && - i < timeout; - i += ms) { + pcie_fw = 0; + for (i = 0; i < timeout; i += ms) { + if (!(adap->flags & IS_VF)) { + pcie_fw = t4_read_reg(adap, A_PCIE_FW); + if (pcie_fw & F_PCIE_FW_ERR) + break; + } if (sleep_ok) { ms = delay[delay_idx]; /* last element may repeat */ if (delay_idx < ARRAY_SIZE(delay) - 1) @@ -698,10 +725,14 @@ unsigned int t4_get_regs_len(struct adap switch (chip_version) { case CHELSIO_T4: + if (adapter->flags & IS_VF) + return FW_T4VF_REGMAP_SIZE; return T4_REGMAP_SIZE; case CHELSIO_T5: case CHELSIO_T6: + if (adapter->flags & IS_VF) + return FW_T4VF_REGMAP_SIZE; return T5_REGMAP_SIZE; } @@ -1180,6 +1211,18 @@ void t4_get_regs(struct adapter *adap, u 0x27e00, 0x27e04, }; + static const unsigned int t4vf_reg_ranges[] = { + VF_SGE_REG(A_SGE_VF_KDOORBELL), VF_SGE_REG(A_SGE_VF_GTS), + VF_MPS_REG(A_MPS_VF_CTL), + VF_MPS_REG(A_MPS_VF_STAT_RX_VF_ERR_FRAMES_H), + VF_PL_REG(A_PL_VF_WHOAMI), VF_PL_REG(A_PL_VF_WHOAMI), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_CTRL), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_STATUS), + FW_T4VF_MBDATA_BASE_ADDR, + FW_T4VF_MBDATA_BASE_ADDR + + ((NUM_CIM_PF_MAILBOX_DATA_INSTANCES - 1) * 4), + }; + static const unsigned int t5_reg_ranges[] = { 0x1008, 0x10c0, 0x10cc, 0x10f8, @@ -1955,6 +1998,18 @@ void t4_get_regs(struct adapter *adap, u 0x51300, 0x51308, }; + static const unsigned int t5vf_reg_ranges[] = { + VF_SGE_REG(A_SGE_VF_KDOORBELL), VF_SGE_REG(A_SGE_VF_GTS), + VF_MPS_REG(A_MPS_VF_CTL), + VF_MPS_REG(A_MPS_VF_STAT_RX_VF_ERR_FRAMES_H), + VF_PL_REG(A_PL_VF_WHOAMI), VF_PL_REG(A_PL_VF_REVISION), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_CTRL), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_STATUS), + FW_T4VF_MBDATA_BASE_ADDR, + FW_T4VF_MBDATA_BASE_ADDR + + ((NUM_CIM_PF_MAILBOX_DATA_INSTANCES - 1) * 4), + }; + static const unsigned int t6_reg_ranges[] = { 0x1008, 0x101c, 0x1024, 0x10a8, @@ -2532,6 +2587,18 @@ void t4_get_regs(struct adapter *adap, u 0x51300, 0x51324, }; + static const unsigned int t6vf_reg_ranges[] = { + VF_SGE_REG(A_SGE_VF_KDOORBELL), VF_SGE_REG(A_SGE_VF_GTS), + VF_MPS_REG(A_MPS_VF_CTL), + VF_MPS_REG(A_MPS_VF_STAT_RX_VF_ERR_FRAMES_H), + VF_PL_REG(A_PL_VF_WHOAMI), VF_PL_REG(A_PL_VF_REVISION), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_CTRL), + VF_CIM_REG(A_CIM_VF_EXT_MAILBOX_STATUS), + FW_T6VF_MBDATA_BASE_ADDR, + FW_T6VF_MBDATA_BASE_ADDR + + ((NUM_CIM_PF_MAILBOX_DATA_INSTANCES - 1) * 4), + }; + u32 *buf_end = (u32 *)(buf + buf_size); const unsigned int *reg_ranges; int reg_ranges_size, range; @@ -2543,18 +2610,33 @@ void t4_get_regs(struct adapter *adap, u */ switch (chip_version) { case CHELSIO_T4: - reg_ranges = t4_reg_ranges; - reg_ranges_size = ARRAY_SIZE(t4_reg_ranges); + if (adap->flags & IS_VF) { + reg_ranges = t4vf_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t4vf_reg_ranges); + } else { + reg_ranges = t4_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t4_reg_ranges); + } break; case CHELSIO_T5: - reg_ranges = t5_reg_ranges; - reg_ranges_size = ARRAY_SIZE(t5_reg_ranges); + if (adap->flags & IS_VF) { + reg_ranges = t5vf_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t5vf_reg_ranges); + } else { + reg_ranges = t5_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t5_reg_ranges); + } break; case CHELSIO_T6: - reg_ranges = t6_reg_ranges; - reg_ranges_size = ARRAY_SIZE(t6_reg_ranges); + if (adap->flags & IS_VF) { + reg_ranges = t6vf_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t6vf_reg_ranges); + } else { + reg_ranges = t6_reg_ranges; + reg_ranges_size = ARRAY_SIZE(t6_reg_ranges); + } break; default: @@ -7416,7 +7498,7 @@ static void set_pcie_completion_timeout( } } -static const struct chip_params *get_chip_params(int chipid) +const struct chip_params *t4_get_chip_params(int chipid) { static const struct chip_params chip_params[] = { { @@ -7495,7 +7577,7 @@ int t4_prep_adapter(struct adapter *adap } } - adapter->chip_params = get_chip_params(chip_id(adapter)); + adapter->chip_params = t4_get_chip_params(chip_id(adapter)); if (adapter->chip_params == NULL) return -EINVAL; @@ -7644,6 +7726,7 @@ int t4_init_sge_params(struct adapter *a { u32 r; struct sge_params *sp = &adapter->params.sge; + unsigned i; r = t4_read_reg(adapter, A_SGE_INGRESS_RX_THRESHOLD); sp->counter_val[0] = G_THRESHOLD_0(r); @@ -7686,6 +7769,7 @@ int t4_init_sge_params(struct adapter *a sp->page_shift = (r & M_HOSTPAGESIZEPF0) + 10; r = t4_read_reg(adapter, A_SGE_CONTROL); + sp->sge_control = r; sp->spg_len = r & F_EGRSTATUSPAGESIZE ? 128 : 64; sp->fl_pktshift = G_PKTSHIFT(r); sp->pad_boundary = 1 << (G_INGPADBOUNDARY(r) + 5); @@ -7698,6 +7782,9 @@ int t4_init_sge_params(struct adapter *a else sp->pack_boundary = 1 << (G_INGPACKBOUNDARY(r) + 5); } + for (i = 0; i < SGE_FLBUF_SIZES; i++) + sp->sge_fl_buffer_size[i] = t4_read_reg(adapter, + A_SGE_FL_BUFFER_SIZE0 + (4 * i)); return 0; } Copied: stable/11/sys/dev/cxgbe/common/t4vf_hw.c (from r305549, head/sys/dev/cxgbe/common/t4vf_hw.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/sys/dev/cxgbe/common/t4vf_hw.c Mon Oct 3 23:49:05 2016 (r306664, copy of r305549, head/sys/dev/cxgbe/common/t4vf_hw.c) @@ -0,0 +1,376 @@ +/*- + * Copyright (c) 2016 Chelsio Communications, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include "common.h" +#include "t4_regs.h" + +#undef msleep +#define msleep(x) do { \ + if (cold) \ + DELAY((x) * 1000); \ + else \ + pause("t4hw", (x) * hz / 1000); \ +} while (0) + +/* + * Wait for the device to become ready (signified by our "who am I" register + * returning a value other than all 1's). Return an error if it doesn't + * become ready ... + */ +int t4vf_wait_dev_ready(struct adapter *adapter) +{ + const u32 whoami = VF_PL_REG(A_PL_VF_WHOAMI); + const u32 notready1 = 0xffffffff; + const u32 notready2 = 0xeeeeeeee; + u32 val; + + val = t4_read_reg(adapter, whoami); + if (val != notready1 && val != notready2) + return 0; + msleep(500); + val = t4_read_reg(adapter, whoami); + if (val != notready1 && val != notready2) + return 0; + else + return -EIO; +} + + +/** + * t4vf_fw_reset - issue a reset to FW + * @adapter: the adapter + * + * Issues a reset command to FW. For a Physical Function this would + * result in the Firmware reseting all of its state. For a Virtual + * Function this just resets the state associated with the VF. + */ +int t4vf_fw_reset(struct adapter *adapter) +{ + struct fw_reset_cmd cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.op_to_write = cpu_to_be32(V_FW_CMD_OP(FW_RESET_CMD) | + F_FW_CMD_WRITE); + cmd.retval_len16 = cpu_to_be32(V_FW_CMD_LEN16(FW_LEN16(cmd))); + return t4vf_wr_mbox(adapter, &cmd, sizeof(cmd), NULL); +} + +/** + * t4vf_get_sge_params - retrieve adapter Scatter gather Engine parameters + * @adapter: the adapter + * + * Retrieves various core SGE parameters in the form of hardware SGE + * register values. The caller is responsible for decoding these as + * needed. The SGE parameters are stored in @adapter->params.sge. + */ +int t4vf_get_sge_params(struct adapter *adapter) +{ + struct sge_params *sp = &adapter->params.sge; + u32 params[7], vals[7]; + u32 whoami; + unsigned int pf, s_hps; + int i, v; + + params[0] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_CONTROL)); + params[1] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_HOST_PAGE_SIZE)); + params[2] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_TIMER_VALUE_0_AND_1)); + params[3] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_TIMER_VALUE_2_AND_3)); + params[4] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_TIMER_VALUE_4_AND_5)); + params[5] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_CONM_CTRL)); + params[6] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_INGRESS_RX_THRESHOLD)); + v = t4vf_query_params(adapter, 7, params, vals); + if (v != FW_SUCCESS) + return v; + + sp->sge_control = vals[0]; + sp->counter_val[0] = G_THRESHOLD_0(vals[6]); + sp->counter_val[1] = G_THRESHOLD_1(vals[6]); + sp->counter_val[2] = G_THRESHOLD_2(vals[6]); + sp->counter_val[3] = G_THRESHOLD_3(vals[6]); + sp->timer_val[0] = core_ticks_to_us(adapter, G_TIMERVALUE0(vals[2])); + sp->timer_val[1] = core_ticks_to_us(adapter, G_TIMERVALUE1(vals[2])); + sp->timer_val[2] = core_ticks_to_us(adapter, G_TIMERVALUE2(vals[3])); + sp->timer_val[3] = core_ticks_to_us(adapter, G_TIMERVALUE3(vals[3])); + sp->timer_val[4] = core_ticks_to_us(adapter, G_TIMERVALUE4(vals[4])); + sp->timer_val[5] = core_ticks_to_us(adapter, G_TIMERVALUE5(vals[4])); + + sp->fl_starve_threshold = G_EGRTHRESHOLD(vals[5]) * 2 + 1; + if (is_t4(adapter)) + sp->fl_starve_threshold2 = sp->fl_starve_threshold; + else + sp->fl_starve_threshold2 = G_EGRTHRESHOLDPACKING(vals[5]) * 2 + + 1; + + /* + * We need the Queues/Page and Host Page Size for our VF. + * This is based on the PF from which we're instantiated. + */ + whoami = t4_read_reg(adapter, VF_PL_REG(A_PL_VF_WHOAMI)); + pf = G_SOURCEPF(whoami); + + s_hps = (S_HOSTPAGESIZEPF0 + + (S_HOSTPAGESIZEPF1 - S_HOSTPAGESIZEPF0) * pf); + sp->page_shift = ((vals[1] >> s_hps) & M_HOSTPAGESIZEPF0) + 10; + + for (i = 0; i < SGE_FLBUF_SIZES; i++) { + params[0] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_FL_BUFFER_SIZE0 + (4 * i))); + v = t4vf_query_params(adapter, 1, params, vals); + if (v != FW_SUCCESS) + return v; + + sp->sge_fl_buffer_size[i] = vals[0]; + } + + /* + * T4 uses a single control field to specify both the PCIe Padding and + * Packing Boundary. T5 introduced the ability to specify these + * separately with the Padding Boundary in SGE_CONTROL and and Packing + * Boundary in SGE_CONTROL2. So for T5 and later we need to grab + * SGE_CONTROL in order to determine how ingress packet data will be + * laid out in Packed Buffer Mode. Unfortunately, older versions of + * the firmware won't let us retrieve SGE_CONTROL2 so if we get a + * failure grabbing it we throw an error since we can't figure out the + * right value. + */ + sp->spg_len = sp->sge_control & F_EGRSTATUSPAGESIZE ? 128 : 64; + sp->fl_pktshift = G_PKTSHIFT(sp->sge_control); + sp->pad_boundary = 1 << (G_INGPADBOUNDARY(sp->sge_control) + 5); + if (is_t4(adapter)) + sp->pack_boundary = sp->pad_boundary; + else { + params[0] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_CONTROL2)); + v = t4vf_query_params(adapter, 1, params, vals); + if (v != FW_SUCCESS) { + CH_ERR(adapter, "Unable to get SGE Control2; " + "probably old firmware.\n"); + return v; + } + if (G_INGPACKBOUNDARY(vals[0]) == 0) + sp->pack_boundary = 16; + else + sp->pack_boundary = 1 << (G_INGPACKBOUNDARY(vals[0]) + + 5); + } + + /* + * For T5 and later we want to use the new BAR2 Doorbells. + * Unfortunately, older firmware didn't allow the this register to be + * read. + */ + if (!is_t4(adapter)) { + unsigned int s_qpp; + + params[0] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_EGRESS_QUEUES_PER_PAGE_VF)); + params[1] = (V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_REG) | + V_FW_PARAMS_PARAM_XYZ(A_SGE_INGRESS_QUEUES_PER_PAGE_VF)); + v = t4vf_query_params(adapter, 2, params, vals); + if (v != FW_SUCCESS) { + CH_WARN(adapter, "Unable to get VF SGE Queues/Page; " + "probably old firmware.\n"); + return v; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201610032349.u93Nn6Ld068983>