From owner-freebsd-stable@FreeBSD.ORG Thu Aug 19 20:27:03 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B67410657DA for ; Thu, 19 Aug 2010 20:27:03 +0000 (UTC) (envelope-from drosih@rpi.edu) Received: from smtp5.server.rpi.edu (smtp5.server.rpi.edu [128.113.2.225]) by mx1.freebsd.org (Postfix) with ESMTP id 4E28C8FC21 for ; Thu, 19 Aug 2010 20:27:03 +0000 (UTC) Received: from smtp5.server.rpi.edu (localhost.localdomain [127.0.0.1]) by smtp5.server.rpi.edu (8.13.1/8.13.1) with ESMTP id o7JJJt8B014926 for ; Thu, 19 Aug 2010 15:20:15 -0400 Received: (from defang@localhost) by smtp5.server.rpi.edu (8.13.1/8.13.0/Submit) id o7JJJM4k014758 for freebsd-stable@freebsd.org; Thu, 19 Aug 2010 15:19:22 -0400 Received: from [128.113.124.121] (gilead.netel.rpi.edu [128.113.124.121]) by smtp5.server.rpi.edu (envelope-sender ) (MIMEDefang) with ESMTP id o7JJJKwl014750; Thu, 19 Aug 2010 15:19:22 -0400 (EDT) Mime-Version: 1.0 Message-Id: In-Reply-To: <4C659B4C.50207@rpi.edu> References: <4C659B4C.50207@rpi.edu> Date: Thu, 19 Aug 2010 15:19:20 -0400 To: Robert Healey , freebsd-stable@freebsd.org From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Bayes-Prob: 0.0001 (Score 0) X-RPI-SA-Score: 0.00 () [Hold at 15.00] 22490(-25) X-CanItPRO-Stream: outgoing X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.225 Cc: Subject: Re: Problem with mxge on RELENG_8_1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Aug 2010 20:27:03 -0000 At 3:21 PM -0400 8/13/10, Robert Healey wrote: > >I recently updated the central file server/router systems for a >pair of research clusters from RELENG_8_0 to RELENG_8_1. After >following the proper procedures, the network throughput when >pulling files from both machines via mxge0 is 200KB/s or less. >Before the update, 50MB/s was the normal rate. > >Doing some testing, with the 8.1 kernel booted, I can upload >files to the server over the 10G link with scp or NFS at the >expected rates. I can fetch files from the Internet using >this server as the NAT gateway also at the expected rates. >Retrieving files from the server over mxge, the throughput >falls to 200KB/s. Retrieving files from the server from the >onboard Broadcom NIC, rates are as to be expected from gigabit. >With 8.0, everything works as expected. > >Hardware Config 1: >Dell Poweredge R610 with 2 Xeon E5530 @ 2.4 GHz, hyperthread >disabled and 24GB RAM. Onboard interface is bce. Disk is >attached via a PERC 6/E. Internal cluster switch is Dell >Powerconnect 6248P. This switch sees excessive large packets >on the 10G connection on 8.1, but not 8.0. > >Hardware Config 2: >HP Proliant DL 320G6 with 1 Xeon E5540 @ 2.53GHz, hyperthread >enabled and 8GB RAM. Onboard interface is bge. Disk is >attached via a HP Smart Array P212. Internal cluster switch >is an HP Procurve 2910al. It does not see any packet errors >from the 10G link. You're seeing the performance problems with both scp and NFS? Could you get some tcpdumps of a file transfer with both setups (8.0 vs 8.1), just to see if something very odd jumps out? If I'm reading this right, file transfers work at expected speeds if the file is going *to* the server, but you see the slowdown when you're pulling files *from* the server. That sounds similar to a problem with a duplex-mismatch, except of course that you shouldn't be getting into duplex issues on a gigabit network. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu