From owner-freebsd-stable@FreeBSD.ORG Wed Sep 8 14:34:45 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBBB71065694 for ; Wed, 8 Sep 2010 14:34:45 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: from martini.nu (martini.nu [198.145.180.83]) by mx1.freebsd.org (Postfix) with SMTP id A96C08FC08 for ; Wed, 8 Sep 2010 14:34:45 +0000 (UTC) Received: (qmail 55879 invoked by uid 1000); 8 Sep 2010 14:34:45 -0000 Date: Wed, 8 Sep 2010 07:34:44 -0700 From: "Mahlon E. Smith" To: Jeremy Chadwick Message-ID: <20100908143444.GB27923@martini.nu> Mail-Followup-To: "Mahlon E. Smith" , Jeremy Chadwick , Pyun YongHyeon , freebsd-stable@freebsd.org References: <20100907210813.GI49065@martini.nu> <20100907222403.GA18595@icarus.home.lan> <20100907233257.GA94092@martini.nu> <20100908002917.GO1439@michelle.cdnetworks.com> <20100908043834.GA27124@icarus.home.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zx4FCpZtqtKETZ7O" Content-Disposition: inline In-Reply-To: <20100908043834.GA27124@icarus.home.lan> X-GPG-Fingerprint: 19B8 DDB3 0156 3A03 FA80 8278 C0BE 6BFB 3606 B267 X-Sysinfo: FreeBSD 7.0-RELEASE-p1 i386 User-Agent: Mutt/1.5.17 (2007-11-01) Cc: Pyun YongHyeon , freebsd-stable@freebsd.org Subject: Re: Network memory allocation failures X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Sep 2010 14:34:46 -0000 --zx4FCpZtqtKETZ7O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 07, 2010, Jeremy Chadwick wrote: >=20 > I figured there might memory exhaustion of sorts, possibly in the bce(4) > driver itself, that could cause the OP's problem. bce(4) might not be > the problem at all. But the OP's issue seems to only occur when > transmitting data, not receiving: >=20 > http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058708.h= tml More information: Looks like 100M wasn't enough of a test burst to tickle the problem in my original message... 10G is, though. It's definitely happening in both directions. Upgraded to -STABLE on one of the two machines last night, running GENERIC. FreeBSD obb 8.1-STABLE FreeBSD 8.1-STABLE #0: Tue Sep 7 19:48:55 PDT 2010 = root@obb:/usr/obj/usr/src/sys/GENERIC amd64 Outgoing: obb# scp testfile root@holp:/usr/local/tmp/ testfile 8% 856MB 37.6MB/s 04:09 ETA Write failed: Cannot allocate memory lost connection obb# scp testfile root@holp:/usr/local/tmp/ testfile 0% 72MB 34.3MB/s 04:56 ETA Write failed: Cannot allocate memory lost connection Incoming: obb# scp root@holp:/usr/local/tmp/testfile . testfile 6% 670MB 31.9MB/s 04:59 ETA Write failed: Cannot allocate memory lost connection obb# scp root@holp:/usr/local/tmp/testfile . testfile 1% 118MB 39.3MB/s 04:17 ETA Write failed: Cannot allocate memory lost connection obb# scp root@holp:/usr/local/tmp/testfile . testfile 15% 1613MB 29.0MB/s 04:57 ETA Write failed: Cannot allocate memory lost connection > The 2nd-to-last paragraph there is worth noting, specifically how > limiting maximum addressable memory to 32GB via loader.conf seems to > work around the issue. I'd no longer consider this a coincidence, limiting the memory to 16G eliminates the issue completely. I'll retest with 32G today. Incoming: obb# scp root@holp:/usr/local/tmp/testfile testfile2 testfile 100% 10GB 17.8MB/s 09:35 obb# scp root@holp:/usr/local/tmp/testfile testfile2 testfile 100% 10GB 17.0MB/s 10:02 Outgoing: obb# scp testfile root@holp:/usr/local/tmp/testfile2 testfile 100% 10GB 35.7MB/s 04:47 obb# scp testfile root@holp:/usr/local/tmp/testfile2 testfile 100% 10GB 35.4MB/s 04:49 =20 > There were other problems with the systems in question back in July, it > seems. I assume these got hammered out somehow: >=20 > http://www.mail-archive.com/freebsd-stable@freebsd.org/msg111408.html To a degree -- the initial install and cpu count problems are all fixed up, thanks to help from the list. The Intel 10G panics were stifled with a newer driver from Intel's site, but I ran out of time to do any serious testing with it, and just ended up using the broadcoms to satisfy my time constraint. -- Mahlon E. Smith =20 http://www.martini.nu/contact.html --zx4FCpZtqtKETZ7O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFMh58E1bsjBDapbeMRAg7qAJ4pGqjom+X+D0G06KzaLqR/d7gn5ACglvjL LBf1URS7U4R1TcZ9O4PrBCE= =Nus7 -----END PGP SIGNATURE----- --zx4FCpZtqtKETZ7O--