From owner-freebsd-stable@FreeBSD.ORG Thu Apr 14 14:38:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E37B106564A; Thu, 14 Apr 2011 14:38:20 +0000 (UTC) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: from emmi.physik-pool.tu-berlin.de (emmi.physik-pool.tu-berlin.de [130.149.58.146]) by mx1.freebsd.org (Postfix) with ESMTP id BF63C8FC08; Thu, 14 Apr 2011 14:38:19 +0000 (UTC) Received: from emmi.physik-pool.tu-berlin.de (localhost.physik-pool.tu-berlin.de [127.0.0.1]) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4) with ESMTP id p3EEcIoA079519; Thu, 14 Apr 2011 16:38:18 +0200 (CEST) (envelope-from elon@emmi.physik-pool.tu-berlin.de) Received: (from elon@localhost) by emmi.physik-pool.tu-berlin.de (8.14.4/8.14.4/Submit) id p3EEcIF8079518; Thu, 14 Apr 2011 16:38:18 +0200 (CEST) (envelope-from elon) Date: Thu, 14 Apr 2011 16:38:18 +0200 From: Leon =?iso-8859-15?Q?Me=DFner?= To: "K. Macy" Message-ID: <20110414143818.GF9116@emmi.physik-pool.tu-berlin.de> Mail-Followup-To: "K. Macy" , freebsd-stable@freebsd.org, Leon =?iso-8859-15?Q?Me=DFner?= References: <20110414130540.GD9116@emmi.physik-pool.tu-berlin.de> <20110414141817.GE9116@emmi.physik-pool.tu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Leon =?iso-8859-15?Q?Me=DFner?= , freebsd-stable@freebsd.org Subject: Re: ixgbe(4) and "Could not setup receive structures" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Apr 2011 14:38:20 -0000 On Thu, Apr 14, 2011 at 04:20:19PM +0200, K. Macy wrote: > That should be plenty, but how large are your receive queues? > How do i show that ? Random sysctl output following ;) # sysctl net.inet.tcp.recvspace net.inet.tcp.recvspace: 65536 # sysctl -a | grep ix.0.queue\* dev.ix.0.queue0.interrupt_rate: 1000000 dev.ix.0.queue0.txd_head: 59 dev.ix.0.queue0.txd_tail: 59 dev.ix.0.queue0.no_desc_avail: 0 dev.ix.0.queue0.tx_packets: 32 dev.ix.0.queue0.rxd_head: 942 dev.ix.0.queue0.rxd_tail: 941 dev.ix.0.queue0.rx_packets: 942 dev.ix.0.queue0.rx_bytes: 60114 dev.ix.0.queue0.lro_queued: 0 dev.ix.0.queue0.lro_flushed: 0 dev.ix.0.queue1.interrupt_rate: 1000000 dev.ix.0.queue1.txd_head: 0 dev.ix.0.queue1.txd_tail: 0 dev.ix.0.queue1.no_desc_avail: 0 dev.ix.0.queue1.tx_packets: 0 dev.ix.0.queue1.rxd_head: 106 dev.ix.0.queue1.rxd_tail: 105 dev.ix.0.queue1.rx_packets: 106 dev.ix.0.queue1.rx_bytes: 12215 dev.ix.0.queue1.lro_queued: 0 dev.ix.0.queue1.lro_flushed: 0 dev.ix.0.queue2.interrupt_rate: 1000000 dev.ix.0.queue2.txd_head: 0 dev.ix.0.queue2.txd_tail: 0 dev.ix.0.queue2.no_desc_avail: 0 dev.ix.0.queue2.tx_packets: 0 dev.ix.0.queue2.rxd_head: 68 dev.ix.0.queue2.rxd_tail: 67 dev.ix.0.queue2.rx_packets: 68 dev.ix.0.queue2.rx_bytes: 17965 dev.ix.0.queue2.lro_queued: 0 dev.ix.0.queue2.lro_flushed: 0 dev.ix.0.queue3.interrupt_rate: 111111 dev.ix.0.queue3.txd_head: 817 dev.ix.0.queue3.txd_tail: 821 dev.ix.0.queue3.no_desc_avail: 0 dev.ix.0.queue3.tx_packets: 1323 dev.ix.0.queue3.rxd_head: 1396 dev.ix.0.queue3.rxd_tail: 1395 dev.ix.0.queue3.rx_packets: 1396 dev.ix.0.queue3.rx_bytes: 145665 dev.ix.0.queue3.lro_queued: 0 dev.ix.0.queue3.lro_flushed: 0 > On Thu, Apr 14, 2011 at 4:18 PM, Leon Meßner > wrote: > > On Thu, Apr 14, 2011 at 03:44:23PM +0200, K. Macy wrote: > >> How many 9k jumbo clusters are available? > > > > Does this output suffice as information ? > > > > # netstat -m > > 8194/1031/9225 mbufs in use (current/cache/total) > > 8192/518/8710/25600 mbuf clusters in use (current/cache/total/max) > > 8192/512 mbuf+clusters out of packet secondary zone in use > > (current/cache) > > 0/5/5/12800 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > > 18432K/1313K/19746K bytes allocated to network (current/cache/total) > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > > >> On Thu, Apr 14, 2011 at 3:05 PM, Leon Meßner > >> wrote: > >> > Hi, > >> > > >> > i tried setting the mtu on one of my ixgbe(4) intel NICs to support > >> > jumbo frames. This is on a box with RELENG_8 from today. > >> > > >> > # ifconfig ix0 mtu 9198 > >> > > >> > I then get the following error: > >> > > >> > # tail -n 1 /var/log/messages > >> > Apr 14 12:48:43 siloneu kernel: ix0: Could not setup receive structures > >> > > >> > I already tried the following patch because of Jack Vogel's advice given > >> > in the following thread on -stable in Oct. last year, which still > >> > produces the same error message and leaves the box unpingable: > >> > > >> > http://lists.freebsd.org/pipermail/freebsd-stable/2010-October/059541.html > >> > > >> > # cat ~/patches/ixgbe.num_queues_to_4.patch > >> > --- /root/.vimbackup/ixgbe.c~   2011-04-12 22:14:27.000000000 +0000 > >> > +++ sys/dev/ixgbe/ixgbe.c       2011-04-12 22:14:27.000000000 +0000 > >> > @@ -273,7 +273,7 @@ TUNABLE_INT("hw.ixgbe.hdr_split", &ixgbe > >> >  * number of cpus. Each queue is a pair > >> >    * of RX and TX rings with a msix vector > >> >      */ > >> >      -static int ixgbe_num_queues = 0; > >> >      +static int ixgbe_num_queues = 4; > >> >       TUNABLE_INT("hw.ixgbe.num_queues", &ixgbe_num_queues); > >> > > >> >         /* > >> > > >> > _______________________________________________ > >> > freebsd-stable@freebsd.org mailing list > >> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >> > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >