From owner-svn-src-head@freebsd.org Sat Jun 29 00:52:22 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7254615D07A5; Sat, 29 Jun 2019 00:52:22 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10028906F7; Sat, 29 Jun 2019 00:52:22 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 001676E75; Sat, 29 Jun 2019 00:52:21 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5T0qLmJ037624; Sat, 29 Jun 2019 00:52:21 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5T0qL66037621; Sat, 29 Jun 2019 00:52:21 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201906290052.x5T0qL66037621@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Sat, 29 Jun 2019 00:52:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r349533 - in head/sys/dev/cxgbe: . tom X-SVN-Group: head X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: in head/sys/dev/cxgbe: . tom X-SVN-Commit-Revision: 349533 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 10028906F7 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jun 2019 00:52:22 -0000 Author: jhb Date: Sat Jun 29 00:52:21 2019 New Revision: 349533 URL: https://svnweb.freebsd.org/changeset/base/349533 Log: Add support for IFCAP_NOMAP to cxgbe(4). Since cxgbe(4) uses sglist instead of bus_dma, this required updates to the code that generates scatter/gather lists for packets. Also, unmapped mbufs are always sent via DMA and never as immediate data in the payload of a work request. Submitted by: gallatin (earlier version) Reviewed by: gallatin, hselasky, rrs Discussed with: np Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20616 Modified: head/sys/dev/cxgbe/t4_main.c head/sys/dev/cxgbe/t4_sge.c head/sys/dev/cxgbe/tom/t4_cpl_io.c Modified: head/sys/dev/cxgbe/t4_main.c ============================================================================== --- head/sys/dev/cxgbe/t4_main.c Sat Jun 29 00:51:38 2019 (r349532) +++ head/sys/dev/cxgbe/t4_main.c Sat Jun 29 00:52:21 2019 (r349533) @@ -1623,7 +1623,7 @@ cxgbe_probe(device_t dev) #define T4_CAP (IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_MTU | IFCAP_HWCSUM | \ IFCAP_VLAN_HWCSUM | IFCAP_TSO | IFCAP_JUMBO_MTU | IFCAP_LRO | \ IFCAP_VLAN_HWTSO | IFCAP_LINKSTATE | IFCAP_HWCSUM_IPV6 | IFCAP_HWSTATS | \ - IFCAP_HWRXTSTMP) + IFCAP_HWRXTSTMP | IFCAP_NOMAP) #define T4_CAP_ENABLE (T4_CAP) static int @@ -1986,6 +1986,8 @@ cxgbe_ioctl(struct ifnet *ifp, unsigned long cmd, cadd rxq->iq.flags &= ~IQ_RX_TIMESTAMP; } } + if (mask & IFCAP_NOMAP) + ifp->if_capenable ^= IFCAP_NOMAP; #ifdef VLAN_CAPABILITIES VLAN_CAPABILITIES(ifp); Modified: head/sys/dev/cxgbe/t4_sge.c ============================================================================== --- head/sys/dev/cxgbe/t4_sge.c Sat Jun 29 00:51:38 2019 (r349532) +++ head/sys/dev/cxgbe/t4_sge.c Sat Jun 29 00:52:21 2019 (r349533) @@ -83,6 +83,7 @@ __FBSDID("$FreeBSD$"); #endif /* Internal mbuf flags stored in PH_loc.eight[1]. */ +#define MC_NOMAP 0x01 #define MC_RAW_WR 0x02 /* @@ -2434,15 +2435,78 @@ m_advance(struct mbuf **pm, int *poffset, int len) return ((void *)p); } +static inline int +count_mbuf_ext_pgs(struct mbuf *m, int skip, vm_paddr_t *nextaddr) +{ + struct mbuf_ext_pgs *ext_pgs; + vm_paddr_t paddr; + int i, len, off, pglen, pgoff, seglen, segoff; + int nsegs = 0; + + MBUF_EXT_PGS_ASSERT(m); + ext_pgs = m->m_ext.ext_pgs; + off = mtod(m, vm_offset_t); + len = m->m_len; + off += skip; + len -= skip; + + if (ext_pgs->hdr_len != 0) { + if (off >= ext_pgs->hdr_len) { + off -= ext_pgs->hdr_len; + } else { + seglen = ext_pgs->hdr_len - off; + segoff = off; + seglen = min(seglen, len); + off = 0; + len -= seglen; + paddr = pmap_kextract( + (vm_offset_t)&ext_pgs->hdr[segoff]); + if (*nextaddr != paddr) + nsegs++; + *nextaddr = paddr + seglen; + } + } + pgoff = ext_pgs->first_pg_off; + for (i = 0; i < ext_pgs->npgs && len > 0; i++) { + pglen = mbuf_ext_pg_len(ext_pgs, i, pgoff); + if (off >= pglen) { + off -= pglen; + pgoff = 0; + continue; + } + seglen = pglen - off; + segoff = pgoff + off; + off = 0; + seglen = min(seglen, len); + len -= seglen; + paddr = ext_pgs->pa[i] + segoff; + if (*nextaddr != paddr) + nsegs++; + *nextaddr = paddr + seglen; + pgoff = 0; + }; + if (len != 0) { + seglen = min(len, ext_pgs->trail_len - off); + len -= seglen; + paddr = pmap_kextract((vm_offset_t)&ext_pgs->trail[off]); + if (*nextaddr != paddr) + nsegs++; + *nextaddr = paddr + seglen; + } + + return (nsegs); +} + + /* * Can deal with empty mbufs in the chain that have m_len = 0, but the chain * must have at least one mbuf that's not empty. It is possible for this * routine to return 0 if skip accounts for all the contents of the mbuf chain. */ static inline int -count_mbuf_nsegs(struct mbuf *m, int skip) +count_mbuf_nsegs(struct mbuf *m, int skip, uint8_t *cflags) { - vm_paddr_t lastb, next; + vm_paddr_t nextaddr, paddr; vm_offset_t va; int len, nsegs; @@ -2451,9 +2515,8 @@ count_mbuf_nsegs(struct mbuf *m, int skip) MPASS(m->m_pkthdr.len >= skip); nsegs = 0; - lastb = 0; + nextaddr = 0; for (; m; m = m->m_next) { - len = m->m_len; if (__predict_false(len == 0)) continue; @@ -2461,14 +2524,20 @@ count_mbuf_nsegs(struct mbuf *m, int skip) skip -= len; continue; } + if ((m->m_flags & M_NOMAP) != 0) { + *cflags |= MC_NOMAP; + nsegs += count_mbuf_ext_pgs(m, skip, &nextaddr); + skip = 0; + continue; + } va = mtod(m, vm_offset_t) + skip; len -= skip; skip = 0; - next = pmap_kextract(va); + paddr = pmap_kextract(va); nsegs += sglist_count((void *)(uintptr_t)va, len); - if (lastb + 1 == next) + if (paddr == nextaddr) nsegs--; - lastb = pmap_kextract(va + len - 1); + nextaddr = pmap_kextract(va + len - 1) + 1; } return (nsegs); @@ -2490,7 +2559,9 @@ parse_pkt(struct adapter *sc, struct mbuf **mp) struct tcphdr *tcp; #endif uint16_t eh_type; + uint8_t cflags; + cflags = 0; M_ASSERTPKTHDR(m0); if (__predict_false(m0->m_pkthdr.len < ETHER_HDR_LEN)) { rc = EINVAL; @@ -2506,7 +2577,7 @@ restart: */ M_ASSERTPKTHDR(m0); MPASS(m0->m_pkthdr.len > 0); - nsegs = count_mbuf_nsegs(m0, 0); + nsegs = count_mbuf_nsegs(m0, 0, &cflags); if (nsegs > (needs_tso(m0) ? TX_SGL_SEGS_TSO : TX_SGL_SEGS)) { if (defragged++ > 0 || (m = m_defrag(m0, M_NOWAIT)) == NULL) { rc = EFBIG; @@ -2516,7 +2587,8 @@ restart: goto restart; } - if (__predict_false(nsegs > 2 && m0->m_pkthdr.len <= MHLEN)) { + if (__predict_false(nsegs > 2 && m0->m_pkthdr.len <= MHLEN && + !(cflags & MC_NOMAP))) { m0 = m_pullup(m0, m0->m_pkthdr.len); if (m0 == NULL) { /* Should have left well enough alone. */ @@ -2527,7 +2599,7 @@ restart: goto restart; } set_mbuf_nsegs(m0, nsegs); - set_mbuf_cflags(m0, 0); + set_mbuf_cflags(m0, cflags); if (sc->flags & IS_VF) set_mbuf_len16(m0, txpkt_vm_len16(nsegs, needs_tso(m0))); else @@ -2616,7 +2688,9 @@ restart: /* EO WRs have the headers in the WR and not the GL. */ immhdrs = m0->m_pkthdr.l2hlen + m0->m_pkthdr.l3hlen + m0->m_pkthdr.l4hlen; - nsegs = count_mbuf_nsegs(m0, immhdrs); + cflags = 0; + nsegs = count_mbuf_nsegs(m0, immhdrs, &cflags); + MPASS(cflags == mbuf_cflags(m0)); set_mbuf_eo_nsegs(m0, nsegs); set_mbuf_eo_len16(m0, txpkt_eo_len16(nsegs, immhdrs, needs_tso(m0))); @@ -4723,7 +4797,8 @@ write_txpkt_wr(struct sge_txq *txq, struct fw_eth_tx_p ctrl = sizeof(struct cpl_tx_pkt_core); if (needs_tso(m0)) ctrl += sizeof(struct cpl_tx_pkt_lso_core); - else if (pktlen <= imm_payload(2) && available >= 2) { + else if (!(mbuf_cflags(m0) & MC_NOMAP) && pktlen <= imm_payload(2) && + available >= 2) { /* Immediate data. Recalculate len16 and set nsegs to 0. */ ctrl += pktlen; len16 = howmany(sizeof(struct fw_eth_tx_pkt_wr) + Modified: head/sys/dev/cxgbe/tom/t4_cpl_io.c ============================================================================== --- head/sys/dev/cxgbe/tom/t4_cpl_io.c Sat Jun 29 00:51:38 2019 (r349532) +++ head/sys/dev/cxgbe/tom/t4_cpl_io.c Sat Jun 29 00:52:21 2019 (r349533) @@ -650,6 +650,8 @@ write_tx_sgl(void *dst, struct mbuf *start, struct mbu if (IS_AIOTX_MBUF(m)) rc = sglist_append_vmpages(&sg, aiotx_mbuf_pages(m), aiotx_mbuf_pgoff(m), m->m_len); + else if (m->m_flags & M_NOMAP) + rc = sglist_append_mb_ext_pgs(&sg, m); else rc = sglist_append(&sg, mtod(m, void *), m->m_len); if (__predict_false(rc != 0)) @@ -771,6 +773,8 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep if (IS_AIOTX_MBUF(m)) n = sglist_count_vmpages(aiotx_mbuf_pages(m), aiotx_mbuf_pgoff(m), m->m_len); + else if (m->m_flags & M_NOMAP) + n = sglist_count_mb_ext_pgs(m); else n = sglist_count(mtod(m, void *), m->m_len);