From owner-svn-src-stable@FreeBSD.ORG Wed Mar 25 13:18:54 2015 Return-Path: Delivered-To: svn-src-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 720C259F; Wed, 25 Mar 2015 13:18:54 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5AF9A51; Wed, 25 Mar 2015 13:18:54 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t2PDIsRE070896; Wed, 25 Mar 2015 13:18:54 GMT (envelope-from arybchik@FreeBSD.org) Received: (from arybchik@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t2PDIqM1070879; Wed, 25 Mar 2015 13:18:52 GMT (envelope-from arybchik@FreeBSD.org) Message-Id: <201503251318.t2PDIqM1070879@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: arybchik set sender to arybchik@FreeBSD.org using -f From: Andrew Rybchenko Date: Wed, 25 Mar 2015 13:18:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r280596 - stable/10/sys/dev/sfxge X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2015 13:18:54 -0000 Author: arybchik Date: Wed Mar 25 13:18:51 2015 New Revision: 280596 URL: https://svnweb.freebsd.org/changeset/base/280596 Log: MFC: 279351 sfxge: expect required init_state on data path and in periodic calls With the patch applied the number of instruction events is 1% less and number of mispredicted branch events is 5% less under multistream TCP traffic load close to line rate. Sponsored by: Solarflare Communications, Inc. Approved by: gnn (mentor) Modified: stable/10/sys/dev/sfxge/sfxge_ev.c stable/10/sys/dev/sfxge/sfxge_intr.c stable/10/sys/dev/sfxge/sfxge_port.c stable/10/sys/dev/sfxge/sfxge_rx.c stable/10/sys/dev/sfxge/sfxge_tx.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/dev/sfxge/sfxge_ev.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_ev.c Wed Mar 25 13:18:36 2015 (r280595) +++ stable/10/sys/dev/sfxge/sfxge_ev.c Wed Mar 25 13:18:51 2015 (r280596) @@ -98,7 +98,7 @@ sfxge_ev_rx(void *arg, uint32_t label, u KASSERT(evq->index == rxq->index, ("evq->index != rxq->index")); - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) goto done; expected = rxq->pending++ & rxq->ptr_mask; @@ -251,7 +251,7 @@ sfxge_ev_tx(void *arg, uint32_t label, u KASSERT(evq->index == txq->evq_index, ("evq->index != txq->evq_index")); - if (txq->init_state != SFXGE_TXQ_STARTED) + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) goto done; stop = (id + 1) & txq->ptr_mask; @@ -428,7 +428,7 @@ sfxge_ev_stat_update(struct sfxge_softc SFXGE_ADAPTER_LOCK(sc); - if (sc->evq[0]->init_state != SFXGE_EVQ_STARTED) + if (__predict_false(sc->evq[0]->init_state != SFXGE_EVQ_STARTED)) goto out; now = ticks; @@ -593,8 +593,8 @@ sfxge_ev_qpoll(struct sfxge_evq *evq) SFXGE_EVQ_LOCK(evq); - if (evq->init_state != SFXGE_EVQ_STARTING && - evq->init_state != SFXGE_EVQ_STARTED) { + if (__predict_false(evq->init_state != SFXGE_EVQ_STARTING && + evq->init_state != SFXGE_EVQ_STARTED)) { rc = EINVAL; goto fail; } Modified: stable/10/sys/dev/sfxge/sfxge_intr.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_intr.c Wed Mar 25 13:18:36 2015 (r280595) +++ stable/10/sys/dev/sfxge/sfxge_intr.c Wed Mar 25 13:18:51 2015 (r280596) @@ -130,7 +130,7 @@ sfxge_intr_message(void *arg) KASSERT(intr->type == EFX_INTR_MESSAGE, ("intr->type != EFX_INTR_MESSAGE")); - if (intr->state != SFXGE_INTR_STARTED) + if (__predict_false(intr->state != SFXGE_INTR_STARTED)) return; (void)efx_intr_status_message(enp, index, &fatal); Modified: stable/10/sys/dev/sfxge/sfxge_port.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_port.c Wed Mar 25 13:18:36 2015 (r280595) +++ stable/10/sys/dev/sfxge/sfxge_port.c Wed Mar 25 13:18:51 2015 (r280596) @@ -50,7 +50,7 @@ sfxge_mac_stat_update(struct sfxge_softc SFXGE_PORT_LOCK_ASSERT_OWNED(port); - if (port->init_state != SFXGE_PORT_STARTED) { + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) { rc = 0; goto out; } @@ -179,7 +179,7 @@ sfxge_port_wanted_fc_handler(SYSCTL_HAND SFXGE_PORT_LOCK(port); if (port->wanted_fc != fcntl) { - if (port->init_state == SFXGE_PORT_STARTED) + if (__predict_false(port->init_state == SFXGE_PORT_STARTED)) error = efx_mac_fcntl_set(sc->enp, port->wanted_fc, B_TRUE); @@ -210,7 +210,8 @@ sfxge_port_link_fc_handler(SYSCTL_HANDLE port = &sc->port; SFXGE_PORT_LOCK(port); - if (port->init_state == SFXGE_PORT_STARTED && SFXGE_LINK_UP(sc)) + if (__predict_true(port->init_state == SFXGE_PORT_STARTED) && + SFXGE_LINK_UP(sc)) efx_mac_fcntl_get(sc->enp, &wanted_fc, &link_fc); else link_fc = 0; @@ -265,7 +266,7 @@ sfxge_mac_poll_work(void *arg, int npend SFXGE_PORT_LOCK(port); - if (port->init_state != SFXGE_PORT_STARTED) + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) goto done; /* This may sleep waiting for MCDI completion */ @@ -332,7 +333,7 @@ sfxge_mac_filter_set(struct sfxge_softc * lock is held in sleeping thread. Both problems are repeatable * on LAG with LACP proto bring up. */ - if (port->init_state == SFXGE_PORT_STARTED) + if (__predict_true(port->init_state == SFXGE_PORT_STARTED)) rc = sfxge_mac_filter_set_locked(sc); else rc = 0; @@ -456,7 +457,7 @@ sfxge_phy_stat_update(struct sfxge_softc SFXGE_PORT_LOCK_ASSERT_OWNED(port); - if (port->init_state != SFXGE_PORT_STARTED) { + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) { rc = 0; goto out; } Modified: stable/10/sys/dev/sfxge/sfxge_rx.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_rx.c Wed Mar 25 13:18:36 2015 (r280595) +++ stable/10/sys/dev/sfxge/sfxge_rx.c Wed Mar 25 13:18:51 2015 (r280596) @@ -209,7 +209,7 @@ sfxge_rx_qfill(struct sfxge_rxq *rxq, un SFXGE_EVQ_LOCK_ASSERT_OWNED(evq); - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) return; rxfill = rxq->added - rxq->completed; @@ -269,7 +269,7 @@ void sfxge_rx_qrefill(struct sfxge_rxq *rxq) { - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) return; /* Make sure the queue is full */ @@ -760,7 +760,7 @@ sfxge_rx_qcomplete(struct sfxge_rxq *rxq rx_desc = &rxq->queue[id]; m = rx_desc->mbuf; - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) goto discard; if (rx_desc->flags & (EFX_ADDR_MISMATCH | EFX_DISCARD)) Modified: stable/10/sys/dev/sfxge/sfxge_tx.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_tx.c Wed Mar 25 13:18:36 2015 (r280595) +++ stable/10/sys/dev/sfxge/sfxge_tx.c Wed Mar 25 13:18:51 2015 (r280596) @@ -311,7 +311,7 @@ static int sfxge_tx_queue_mbuf(struct sf if (mbuf->m_pkthdr.csum_flags & CSUM_TSO) prefetch_read_many(mbuf->m_data); - if (txq->init_state != SFXGE_TXQ_STARTED) { + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) { rc = EINTR; goto reject; } @@ -1142,7 +1142,7 @@ sfxge_tx_qunblock(struct sfxge_txq *txq) SFXGE_EVQ_LOCK_ASSERT_OWNED(evq); - if (txq->init_state != SFXGE_TXQ_STARTED) + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) return; SFXGE_TXQ_LOCK(txq);