Date: Wed, 8 Sep 2010 07:34:44 -0700 From: "Mahlon E. Smith" <mahlon@martini.nu> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: Pyun YongHyeon <pyunyh@gmail.com>, freebsd-stable@freebsd.org Subject: Re: Network memory allocation failures Message-ID: <20100908143444.GB27923@martini.nu> In-Reply-To: <20100908043834.GA27124@icarus.home.lan> References: <20100907210813.GI49065@martini.nu> <20100907222403.GA18595@icarus.home.lan> <20100907233257.GA94092@martini.nu> <20100908002917.GO1439@michelle.cdnetworks.com> <20100908043834.GA27124@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
--zx4FCpZtqtKETZ7O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 07, 2010, Jeremy Chadwick wrote: >=20 > I figured there might memory exhaustion of sorts, possibly in the bce(4) > driver itself, that could cause the OP's problem. bce(4) might not be > the problem at all. But the OP's issue seems to only occur when > transmitting data, not receiving: >=20 > http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058708.h= tml More information: Looks like 100M wasn't enough of a test burst to tickle the problem in my original message... 10G is, though. It's definitely happening in both directions. Upgraded to -STABLE on one of the two machines last night, running GENERIC. FreeBSD obb 8.1-STABLE FreeBSD 8.1-STABLE #0: Tue Sep 7 19:48:55 PDT 2010 = root@obb:/usr/obj/usr/src/sys/GENERIC amd64 Outgoing: obb# scp testfile root@holp:/usr/local/tmp/ testfile 8% 856MB 37.6MB/s 04:09 ETA Write failed: Cannot allocate memory lost connection obb# scp testfile root@holp:/usr/local/tmp/ testfile 0% 72MB 34.3MB/s 04:56 ETA Write failed: Cannot allocate memory lost connection Incoming: obb# scp root@holp:/usr/local/tmp/testfile . testfile 6% 670MB 31.9MB/s 04:59 ETA Write failed: Cannot allocate memory lost connection obb# scp root@holp:/usr/local/tmp/testfile . testfile 1% 118MB 39.3MB/s 04:17 ETA Write failed: Cannot allocate memory lost connection obb# scp root@holp:/usr/local/tmp/testfile . testfile 15% 1613MB 29.0MB/s 04:57 ETA Write failed: Cannot allocate memory lost connection > The 2nd-to-last paragraph there is worth noting, specifically how > limiting maximum addressable memory to 32GB via loader.conf seems to > work around the issue. I'd no longer consider this a coincidence, limiting the memory to 16G eliminates the issue completely. I'll retest with 32G today. Incoming: obb# scp root@holp:/usr/local/tmp/testfile testfile2 testfile 100% 10GB 17.8MB/s 09:35 obb# scp root@holp:/usr/local/tmp/testfile testfile2 testfile 100% 10GB 17.0MB/s 10:02 Outgoing: obb# scp testfile root@holp:/usr/local/tmp/testfile2 testfile 100% 10GB 35.7MB/s 04:47 obb# scp testfile root@holp:/usr/local/tmp/testfile2 testfile 100% 10GB 35.4MB/s 04:49 =20 > There were other problems with the systems in question back in July, it > seems. I assume these got hammered out somehow: >=20 > http://www.mail-archive.com/freebsd-stable@freebsd.org/msg111408.html To a degree -- the initial install and cpu count problems are all fixed up, thanks to help from the list. The Intel 10G panics were stifled with a newer driver from Intel's site, but I ran out of time to do any serious testing with it, and just ended up using the broadcoms to satisfy my time constraint. -- Mahlon E. Smith =20 http://www.martini.nu/contact.html --zx4FCpZtqtKETZ7O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFMh58E1bsjBDapbeMRAg7qAJ4pGqjom+X+D0G06KzaLqR/d7gn5ACglvjL LBf1URS7U4R1TcZ9O4PrBCE= =Nus7 -----END PGP SIGNATURE----- --zx4FCpZtqtKETZ7O--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100908143444.GB27923>