Date: Thu, 14 Apr 2011 13:28:15 -0700 From: Jack Vogel <jfvogel@gmail.com> To: "K. Macy" <kmacy@freebsd.org>, Jack Vogel <jfvogel@gmail.com>, =?ISO-8859-1?Q?Leon_Me=DFner?= <l.messner@physik.tu-berlin.de>, freebsd-stable@freebsd.org Subject: Re: ixgbe(4) and "Could not setup receive structures" Message-ID: <BANLkTinRmQ2BCWJQt0H0DjgPE0UzryK78g@mail.gmail.com> In-Reply-To: <20110414194448.GG9116@emmi.physik-pool.tu-berlin.de> References: <20110414130540.GD9116@emmi.physik-pool.tu-berlin.de> <BANLkTikVVGPb3k74Xdty%2BH_tjjWD3SZ_Ng@mail.gmail.com> <BANLkTinw8PcTZ4A3VQXKQs7=zZWL6bt8dA@mail.gmail.com> <20110414194448.GG9116@emmi.physik-pool.tu-berlin.de>
next in thread | previous in thread | raw e-mail | index | archive | help
If you are using the latest code, then the RX ring size is set to be 2K descriptors, so you will use that many 9k jumbos per queue to initialize things. Having a spare amount free to use as you clean/refresh is needed also. I upped the ring size for performance reasons on 10G, its possible to try dropping it to 1K. But, for 10G, I don't think its unreasonable to have enough memory around t= o handle this. Cheers, Jack On Thu, Apr 14, 2011 at 12:44 PM, Leon Me=DFner <l.messner@physik.tu-berlin= .de > wrote: > On Thu, Apr 14, 2011 at 08:55:17PM +0200, K. Macy wrote: > > That isn't guaranteed to work if he is KVA limited. > > > > On Thu, Apr 14, 2011 at 6:44 PM, Jack Vogel <jfvogel@gmail.com> wrote: > > > If you get this message its only for one reason, you don't have enoug= h > mbufs > > > to > > > fill your rings. You must do one of two things, either reduce the > number of > > > queues, > > > or increase the relevant mbuf pool. > > > > > > Increase the 9K mbuf cluster pool. > > I did set it to twice the default, and now it works and netstat -m > shows: > > 8192/391/8583/12800 9k jumbo clusters in use (current/cache/total/max) > > Whats a reasonable amount to set kern.ipc.nmbjumbo9 to and is there > any > form of auto-tuning (i have absolutely no load on this machine and > mbufs > are higher than default pool size). > > Thanks to all, > Leon > > > > On Thu, Apr 14, 2011 at 6:05 AM, Leon Me=DFner > > > <l.messner@physik.tu-berlin.de>wrote: > > > > > >> Hi, > > >> > > >> i tried setting the mtu on one of my ixgbe(4) intel NICs to support > > >> jumbo frames. This is on a box with RELENG_8 from today. > > >> > > >> # ifconfig ix0 mtu 9198 > > >> > > >> I then get the following error: > > >> > > >> # tail -n 1 /var/log/messages > > >> Apr 14 12:48:43 siloneu kernel: ix0: Could not setup receive > structures > > >> > > >> I already tried the following patch because of Jack Vogel's advice > given > > >> in the following thread on -stable in Oct. last year, which still > > >> produces the same error message and leaves the box unpingable: > > >> > > >> > http://lists.freebsd.org/pipermail/freebsd-stable/2010-October/059541.htm= l > > >> > > >> # cat ~/patches/ixgbe.num_queues_to_4.patch > > >> --- /root/.vimbackup/ixgbe.c~ 2011-04-12 22:14:27.000000000 +0000 > > >> +++ sys/dev/ixgbe/ixgbe.c 2011-04-12 22:14:27.000000000 +0000 > > >> @@ -273,7 +273,7 @@ TUNABLE_INT("hw.ixgbe.hdr_split", &ixgbe > > >> * number of cpus. Each queue is a pair > > >> * of RX and TX rings with a msix vector > > >> */ > > >> -static int ixgbe_num_queues =3D 0; > > >> +static int ixgbe_num_queues =3D 4; > > >> TUNABLE_INT("hw.ixgbe.num_queues", &ixgbe_num_queues); > > >> > > >> /* > > >> > > >> _______________________________________________ > > >> freebsd-stable@freebsd.org mailing list > > >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > >> To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org" > > >> > > > _______________________________________________ > > > freebsd-stable@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org" > > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BANLkTinRmQ2BCWJQt0H0DjgPE0UzryK78g>