From owner-svn-src-all@FreeBSD.ORG Wed May 7 04:21:08 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73175D3; Wed, 7 May 2014 04:21:08 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 55B71D51; Wed, 7 May 2014 04:21:08 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s474L8gr032692; Wed, 7 May 2014 04:21:08 GMT (envelope-from np@svn.freebsd.org) Received: (from np@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s474L7Q4032688; Wed, 7 May 2014 04:21:07 GMT (envelope-from np@svn.freebsd.org) Message-Id: <201405070421.s474L7Q4032688@svn.freebsd.org> From: Navdeep Parhar Date: Wed, 7 May 2014 04:21:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org Subject: svn commit: r265482 - stable/9/sys/dev/cxgbe/tom X-SVN-Group: stable-9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 04:21:08 -0000 Author: np Date: Wed May 7 04:21:07 2014 New Revision: 265482 URL: http://svnweb.freebsd.org/changeset/base/265482 Log: MFC r255198, r255410, and r255411. r255198: For TOE connections, the window scale factor in CPL_PASS_ACCEPT_REQ is set to 15 to indicate that the peer did not send a window scale option with its SYN. Do not send a window scale option in the SYN|ACK reply in that case. r255410: Fix a miscalculation that caused cxgbe/tom to auto-increment a TOE socket's tx buffer size too aggressively. r255411: Rework the tx credit mechanism between the cxgbe/tom driver and the card. This helps smooth out some burstiness in the exchange. Modified: stable/9/sys/dev/cxgbe/tom/t4_cpl_io.c stable/9/sys/dev/cxgbe/tom/t4_listen.c stable/9/sys/dev/cxgbe/tom/t4_tom.c stable/9/sys/dev/cxgbe/tom/t4_tom.h Directory Properties: stable/9/sys/ (props changed) stable/9/sys/dev/ (props changed) Modified: stable/9/sys/dev/cxgbe/tom/t4_cpl_io.c ============================================================================== --- stable/9/sys/dev/cxgbe/tom/t4_cpl_io.c Wed May 7 04:00:05 2014 (r265481) +++ stable/9/sys/dev/cxgbe/tom/t4_cpl_io.c Wed May 7 04:21:07 2014 (r265482) @@ -444,22 +444,12 @@ max_dsgl_nsegs(int tx_credits) static inline void write_tx_wr(void *dst, struct toepcb *toep, unsigned int immdlen, - unsigned int plen, uint8_t credits, int more_to_come) + unsigned int plen, uint8_t credits, int shove) { struct fw_ofld_tx_data_wr *txwr = dst; - int shove = !more_to_come; - int compl = 1; - - /* - * We always request completion notifications from the firmware. The - * only exception is when we know we'll get more data to send shortly - * and that we'll have some tx credits remaining to transmit that data. - */ - if (more_to_come && toep->tx_credits - credits >= MIN_OFLD_TX_CREDITS) - compl = 0; txwr->op_to_immdlen = htobe32(V_WR_OP(FW_OFLD_TX_DATA_WR) | - V_FW_WR_COMPL(compl) | V_FW_WR_IMMDLEN(immdlen)); + V_FW_WR_IMMDLEN(immdlen)); txwr->flowid_len16 = htobe32(V_FW_WR_FLOWID(toep->tid) | V_FW_WR_LEN16(credits)); txwr->tunnel_to_proxy = @@ -529,19 +519,26 @@ write_tx_sgl(void *dst, struct mbuf *sta * The socket's so_snd buffer consists of a stream of data starting with sb_mb * and linked together with m_next. sb_sndptr, if set, is the last mbuf that * was transmitted. + * + * drop indicates the number of bytes that should be dropped from the head of + * the send buffer. It is an optimization that lets do_fw4_ack avoid creating + * contention on the send buffer lock (before this change it used to do + * sowwakeup and then t4_push_frames right after that when recovering from tx + * stalls). When drop is set this function MUST drop the bytes and wake up any + * writers. */ static void -t4_push_frames(struct adapter *sc, struct toepcb *toep) +t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) { struct mbuf *sndptr, *m, *sb_sndptr; struct fw_ofld_tx_data_wr *txwr; struct wrqe *wr; - unsigned int plen, nsegs, credits, max_imm, max_nsegs, max_nsegs_1mbuf; + u_int plen, nsegs, credits, max_imm, max_nsegs, max_nsegs_1mbuf; struct inpcb *inp = toep->inp; struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; - int tx_credits; + int tx_credits, shove, compl, space, sowwakeup; struct ofld_tx_sdesc *txsd = &toep->txsd[toep->txsd_pidx]; INP_WLOCK_ASSERT(inp); @@ -556,8 +553,11 @@ t4_push_frames(struct adapter *sc, struc * This function doesn't resume by itself. Someone else must clear the * flag and call this function. */ - if (__predict_false(toep->flags & TPF_TX_SUSPENDED)) + if (__predict_false(toep->flags & TPF_TX_SUSPENDED)) { + KASSERT(drop == 0, + ("%s: drop (%d) != 0 but tx is suspended", __func__, drop)); return; + } do { tx_credits = min(toep->tx_credits, MAX_OFLD_TX_CREDITS); @@ -565,6 +565,11 @@ t4_push_frames(struct adapter *sc, struc max_nsegs = max_dsgl_nsegs(tx_credits); SOCKBUF_LOCK(sb); + sowwakeup = drop; + if (drop) { + sbdrop_locked(sb, drop); + drop = 0; + } sb_sndptr = sb->sb_sndptr; sndptr = sb_sndptr ? sb_sndptr->m_next : sb->sb_mb; plen = 0; @@ -583,7 +588,11 @@ t4_push_frames(struct adapter *sc, struc if (plen == 0) { /* Too few credits */ toep->flags |= TPF_TX_SUSPENDED; - SOCKBUF_UNLOCK(sb); + if (sowwakeup) + sowwakeup_locked(so); + else + SOCKBUF_UNLOCK(sb); + SOCKBUF_UNLOCK_ASSERT(sb); return; } break; @@ -600,23 +609,32 @@ t4_push_frames(struct adapter *sc, struc } } + shove = m == NULL && !(tp->t_flags & TF_MORETOCOME); + space = sbspace(sb); + + if (space <= sb->sb_hiwat * 3 / 8 && + toep->plen_nocompl + plen >= sb->sb_hiwat / 4) + compl = 1; + else + compl = 0; + if (sb->sb_flags & SB_AUTOSIZE && V_tcp_do_autosndbuf && sb->sb_hiwat < V_tcp_autosndbuf_max && - sbspace(sb) < sb->sb_hiwat / 8 * 7) { + space < sb->sb_hiwat / 8) { int newsize = min(sb->sb_hiwat + V_tcp_autosndbuf_inc, V_tcp_autosndbuf_max); if (!sbreserve_locked(sb, newsize, so, NULL)) sb->sb_flags &= ~SB_AUTOSIZE; - else { - sowwakeup_locked(so); /* room available */ - SOCKBUF_UNLOCK_ASSERT(sb); - goto unlocked; - } + else + sowwakeup = 1; /* room available */ } - SOCKBUF_UNLOCK(sb); -unlocked: + if (sowwakeup) + sowwakeup_locked(so); + else + SOCKBUF_UNLOCK(sb); + SOCKBUF_UNLOCK_ASSERT(sb); /* nothing to send */ if (plen == 0) { @@ -641,9 +659,9 @@ unlocked: } txwr = wrtod(wr); credits = howmany(wr->wr_len, 16); - write_tx_wr(txwr, toep, plen, plen, credits, - tp->t_flags & TF_MORETOCOME); + write_tx_wr(txwr, toep, plen, plen, credits, shove); m_copydata(sndptr, 0, plen, (void *)(txwr + 1)); + nsegs = 0; } else { int wr_len; @@ -659,8 +677,7 @@ unlocked: } txwr = wrtod(wr); credits = howmany(wr_len, 16); - write_tx_wr(txwr, toep, 0, plen, credits, - tp->t_flags & TF_MORETOCOME); + write_tx_wr(txwr, toep, 0, plen, credits, shove); write_tx_sgl(txwr + 1, sndptr, m, nsegs, max_nsegs_1mbuf); if (wr_len & 0xf) { @@ -674,6 +691,17 @@ unlocked: ("%s: not enough credits", __func__)); toep->tx_credits -= credits; + toep->tx_nocompl += credits; + toep->plen_nocompl += plen; + if (toep->tx_credits <= toep->tx_total * 3 / 8 && + toep->tx_nocompl >= toep->tx_total / 4) + compl = 1; + + if (compl) { + txwr->op_to_immdlen |= htobe32(F_FW_WR_COMPL); + toep->tx_nocompl = 0; + toep->plen_nocompl = 0; + } tp->snd_nxt += plen; tp->snd_max += plen; @@ -684,6 +712,8 @@ unlocked: SOCKBUF_UNLOCK(sb); toep->flags |= TPF_TX_DATA_SENT; + if (toep->tx_credits < MIN_OFLD_TX_CREDITS) + toep->flags |= TPF_TX_SUSPENDED; KASSERT(toep->txsd_avail > 0, ("%s: no txsd", __func__)); txsd->plen = plen; @@ -717,7 +747,7 @@ t4_tod_output(struct toedev *tod, struct ("%s: inp %p dropped.", __func__, inp)); KASSERT(toep != NULL, ("%s: toep is NULL", __func__)); - t4_push_frames(sc, toep); + t4_push_frames(sc, toep, 0); return (0); } @@ -737,7 +767,8 @@ t4_send_fin(struct toedev *tod, struct t KASSERT(toep != NULL, ("%s: toep is NULL", __func__)); toep->flags |= TPF_SEND_FIN; - t4_push_frames(sc, toep); + if (tp->t_state >= TCPS_ESTABLISHED) + t4_push_frames(sc, toep, 0); return (0); } @@ -1366,7 +1397,16 @@ do_fw4_ack(struct sge_iq *iq, const stru } } - if (plen > 0) { + if (toep->tx_credits == toep->tx_total) { + toep->tx_nocompl = 0; + toep->plen_nocompl = 0; + } + + if (toep->flags & TPF_TX_SUSPENDED && + toep->tx_credits >= toep->tx_total / 4) { + toep->flags &= ~TPF_TX_SUSPENDED; + t4_push_frames(sc, toep, plen); + } else if (plen > 0) { struct sockbuf *sb = &so->so_snd; SOCKBUF_LOCK(sb); @@ -1375,14 +1415,6 @@ do_fw4_ack(struct sge_iq *iq, const stru SOCKBUF_UNLOCK_ASSERT(sb); } - /* XXX */ - if ((toep->flags & TPF_TX_SUSPENDED && - toep->tx_credits >= MIN_OFLD_TX_CREDITS) || - toep->tx_credits == toep->txsd_total * - howmany((sizeof(struct fw_ofld_tx_data_wr) + 1), 16)) { - toep->flags &= ~TPF_TX_SUSPENDED; - t4_push_frames(sc, toep); - } INP_WUNLOCK(inp); return (0); Modified: stable/9/sys/dev/cxgbe/tom/t4_listen.c ============================================================================== --- stable/9/sys/dev/cxgbe/tom/t4_listen.c Wed May 7 04:00:05 2014 (r265481) +++ stable/9/sys/dev/cxgbe/tom/t4_listen.c Wed May 7 04:21:07 2014 (r265482) @@ -1007,7 +1007,7 @@ calc_opt2p(struct adapter *sc, struct po opt2 |= F_TSTAMPS_EN; if (tcpopt->sack) opt2 |= F_SACK_EN; - if (tcpopt->wsf > 0) + if (tcpopt->wsf <= 14) opt2 |= F_WND_SCALE_EN; } Modified: stable/9/sys/dev/cxgbe/tom/t4_tom.c ============================================================================== --- stable/9/sys/dev/cxgbe/tom/t4_tom.c Wed May 7 04:00:05 2014 (r265481) +++ stable/9/sys/dev/cxgbe/tom/t4_tom.c Wed May 7 04:21:07 2014 (r265482) @@ -148,6 +148,7 @@ alloc_toepcb(struct port_info *pi, int t toep->td = sc->tom_softc; toep->port = pi; + toep->tx_total = tx_credits; toep->tx_credits = tx_credits; toep->ofld_txq = &sc->sge.ofld_txq[txqid]; toep->ofld_rxq = &sc->sge.ofld_rxq[rxqid]; Modified: stable/9/sys/dev/cxgbe/tom/t4_tom.h ============================================================================== --- stable/9/sys/dev/cxgbe/tom/t4_tom.h Wed May 7 04:00:05 2014 (r265481) +++ stable/9/sys/dev/cxgbe/tom/t4_tom.h Wed May 7 04:21:07 2014 (r265482) @@ -99,7 +99,7 @@ struct ddp_buffer { struct toepcb { TAILQ_ENTRY(toepcb) link; /* toep_list */ - unsigned int flags; /* miscellaneous flags */ + u_int flags; /* miscellaneous flags */ struct tom_data *td; struct inpcb *inp; /* backpointer to host stack's PCB */ struct port_info *port; /* physical port */ @@ -109,13 +109,20 @@ struct toepcb { struct l2t_entry *l2te; /* L2 table entry used by this connection */ struct clip_entry *ce; /* CLIP table entry used by this tid */ int tid; /* Connection identifier */ - unsigned int tx_credits;/* tx WR credits (in 16 byte units) remaining */ - unsigned int sb_cc; /* last noted value of so_rcv->sb_cc */ + + /* tx credit handling */ + u_int tx_total; /* total tx WR credits (in 16B units) */ + u_int tx_credits; /* tx WR credits (in 16B units) available */ + u_int tx_nocompl; /* tx WR credits since last compl request */ + u_int plen_nocompl; /* payload since last compl request */ + + /* rx credit handling */ + u_int sb_cc; /* last noted value of so_rcv->sb_cc */ int rx_credits; /* rx credits (in bytes) to be returned to hw */ - unsigned int ulp_mode; /* ULP mode */ + u_int ulp_mode; /* ULP mode */ - unsigned int ddp_flags; + u_int ddp_flags; struct ddp_buffer *db[2]; time_t ddp_disabled; uint8_t ddp_score;