From owner-freebsd-smp@FreeBSD.ORG Sun Oct 2 20:09:46 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BFD4316A41F for ; Sun, 2 Oct 2005 20:09:46 +0000 (GMT) (envelope-from robert@orcafat.com) Received: from slay.orcafat.com (1-1-6-11a.mfb.mlm.bostream.se [82.182.195.254]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37B7A43D48 for ; Sun, 2 Oct 2005 20:09:45 +0000 (GMT) (envelope-from robert@orcafat.com) Received: from localhost (localhost [127.0.0.1]) by slay.orcafat.com (Postfix) with ESMTP id CB1EB1FC for ; Sun, 2 Oct 2005 22:09:42 +0200 (CEST) Received: from slay.orcafat.com ([127.0.0.1]) by localhost (slay [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 22535-06 for ; Sun, 2 Oct 2005 22:09:41 +0200 (CEST) Received: from ht.orcafat.com (ht.orcafat.com [192.168.1.2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by slay.orcafat.com (Postfix) with ESMTP id A49BFDC for ; Sun, 2 Oct 2005 22:09:41 +0200 (CEST) From: Robert Lindgren To: freebsd-smp@freebsd.org Date: Sun, 2 Oct 2005 22:09:38 +0200 User-Agent: KMail/1.8.2 MIME-Version: 1.0 Content-Disposition: inline X-UID: 78 X-Length: 1063 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <200510022209.38594.robert@orcafat.com> X-Virus-Scanned: by amavisd-maia-1.0.0-rc5 (Debian) at orcafat.com Subject: FreeBSD 5.4 network performance with SMP X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2005 20:09:46 -0000 Hi All, I have a strange little problem on Dell 1800's. If HT,SMP is enabled in the kernel (5.4p7), performance on the integrated em0 is not especially good, half the speed of the current 100Mbit connection. This is true from the server and a little better to the server. On a pci rl0 card in the same machine performance is even worse. About 15Mbit from the server! Then I read a lot of people having poor performance from the rl0 cards and a lot of FreeBSD people bashing it for being crap, even in the source code for the module it's stated that these cards aren't so good. I guess that is right to some extent but not 15Mbit/sec. So when I rebuilt the kernel without HT and SMP I get full wirespeed from the em0 and the rl0 (100Mbit/sec). Is there anything I've been doing wrong to get hit by this poor performance with the SMP kernel. I read the disabling USB could improve performance but tried that with the HT kernel without any luck. The server has 1 3Ghz cpu, and I would prefer too run a HT enabled kernel, but not with that kind of performance, so please advice. The same machines (Dell 1800) run linux with HT enabled without any network performance degradation. Best regards Robert Lindgren From owner-freebsd-smp@FreeBSD.ORG Sun Oct 2 22:47:31 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6C46E16A41F for ; Sun, 2 Oct 2005 22:47:31 +0000 (GMT) (envelope-from mchartzell@getdts.com) Received: from smtp102.sbc.mail.re2.yahoo.com (smtp102.sbc.mail.re2.yahoo.com [68.142.229.103]) by mx1.FreeBSD.org (Postfix) with SMTP id F005443D45 for ; Sun, 2 Oct 2005 22:47:30 +0000 (GMT) (envelope-from mchartzell@getdts.com) Received: (qmail 18482 invoked from network); 2 Oct 2005 22:47:30 -0000 Received: from unknown (HELO ?192.168.0.3?) (mchartzell@sbcglobal.net@67.163.122.242 with plain) by smtp102.sbc.mail.re2.yahoo.com with SMTP; 2 Oct 2005 22:47:30 -0000 Message-ID: <4340636D.3090408@getdts.com> Date: Sun, 02 Oct 2005 17:47:09 -0500 From: Matt Hartzell User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Lindgren References: <200510022209.38594.robert@orcafat.com> In-Reply-To: <200510022209.38594.robert@orcafat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-smp@freebsd.org Subject: Re: FreeBSD 5.4 network performance with SMP X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mchartzell@getdts.com List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2005 22:47:31 -0000 In my experience I had to disable the USB in the bios. Do a quick vmstat -i and note how many interrupts are on the uhci device - with no devices even connected to the usb bus. Also, what sysctl settings are you using? If your network supports it, how about using jumbo frames? Robert Lindgren wrote: >Hi All, > >I have a strange little problem on Dell 1800's. If HT,SMP is enabled in the >kernel (5.4p7), performance on the integrated em0 is not especially good, >half the speed of the current 100Mbit connection. This is true from the >server and a little better to the server. On a pci rl0 card in the same >machine performance is even worse. About 15Mbit from the server! > >Then I read a lot of people having poor performance from the rl0 cards and a >lot of FreeBSD people bashing it for being crap, even in the source code for >the module it's stated that these cards aren't so good. I guess that is right >to some extent but not 15Mbit/sec. > >So when I rebuilt the kernel without HT and SMP I get full wirespeed from the >em0 and the rl0 (100Mbit/sec). Is there anything I've been doing wrong to get >hit by this poor performance with the SMP kernel. I read the disabling USB >could improve performance but tried that with the HT kernel without any luck. > >The server has 1 3Ghz cpu, and I would prefer too run a HT enabled kernel, but >not with that kind of performance, so please advice. The same machines (Dell >1800) run linux with HT enabled without any network performance degradation. > >Best regards >Robert Lindgren > >_______________________________________________ >freebsd-smp@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-smp >To unsubscribe, send any mail to "freebsd-smp-unsubscribe@freebsd.org" > > > From owner-freebsd-smp@FreeBSD.ORG Sun Oct 2 23:57:09 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6EB9516A41F for ; Sun, 2 Oct 2005 23:57:09 +0000 (GMT) (envelope-from ohartman@mail.uni-mainz.de) Received: from mailgate1.zdv.Uni-Mainz.DE (mailgate1.zdv.Uni-Mainz.DE [134.93.178.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B8A543D55 for ; Sun, 2 Oct 2005 23:57:06 +0000 (GMT) (envelope-from ohartman@mail.uni-mainz.de) Received: from [84.245.182.115] (84-245-182-115.ipool.celox.de [84.245.182.115]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mailgate1.zdv.Uni-Mainz.DE (Postfix) with ESMTP id 851EB3000750; Mon, 3 Oct 2005 01:57:03 +0200 (CEST) Message-ID: <434073C8.2000608@mail.uni-mainz.de> Date: Mon, 03 Oct 2005 01:56:56 +0200 From: "O. Hartmann" Organization: Institut =?ISO-8859-1?Q?f=FCr_Geophysik?= User-Agent: Mozilla Thunderbird 1.0.7 (X11/20050930) X-Accept-Language: en-us, en MIME-Version: 1.0 To: mchartzell@getdts.com References: <200510022209.38594.robert@orcafat.com> <4340636D.3090408@getdts.com> In-Reply-To: <4340636D.3090408@getdts.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-mainz.de Cc: freebsd-smp@freebsd.org Subject: Re: FreeBSD 5.4 network performance with SMP X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2005 23:57:09 -0000 Matt Hartzell wrote: > In my experience I had to disable the USB in the bios. > > Do a quick vmstat -i and note how many interrupts are on the uhci > device - with no devices even connected to the usb bus. > > Also, what sysctl settings are you using? > > If your network supports it, how about using jumbo frames? > > Robert Lindgren wrote: > >> Hi All, >> >> I have a strange little problem on Dell 1800's. If HT,SMP is enabled >> in the kernel (5.4p7), performance on the integrated em0 is not >> especially good, half the speed of the current 100Mbit connection. >> This is true from the server and a little better to the server. On a >> pci rl0 card in the same machine performance is even worse. About >> 15Mbit from the server! >> >> Then I read a lot of people having poor performance from the rl0 >> cards and a lot of FreeBSD people bashing it for being crap, even in >> the source code for the module it's stated that these cards aren't so >> good. I guess that is right to some extent but not 15Mbit/sec. >> >> So when I rebuilt the kernel without HT and SMP I get full wirespeed >> from the em0 and the rl0 (100Mbit/sec). Is there anything I've been >> doing wrong to get hit by this poor performance with the SMP kernel. >> I read the disabling USB could improve performance but tried that >> with the HT kernel without any luck. >> >> The server has 1 3Ghz cpu, and I would prefer too run a HT enabled >> kernel, but not with that kind of performance, so please advice. The >> same machines (Dell 1800) run linux with HT enabled without any >> network performance degradation. >> >> Best regards >> Robert Lindgren >> >> _______________________________________________ >> freebsd-smp@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-smp >> To unsubscribe, send any mail to "freebsd-smp-unsubscribe@freebsd.org" >> >> >> > > _______________________________________________ > freebsd-smp@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-smp > To unsubscribe, send any mail to "freebsd-smp-unsubscribe@freebsd.org" I see something similar in FreeBSD 6.0-BETA5, no SMP, but with both sk0 and nve0 NICs. This is what my vmstat -i shows up. I do not have attached any USB device, but there seems to be a lot of interrupt requests. nve0 seems to 'hang' sometimes. As I can see, nve0 and the sound device ocm0 (snd_ich) shares an IRQ. Can this be changed anyway? interrupt total rate irq1: atkbd0 15130 3 irq4: sio0 6178 1 irq6: fdc0 27 0 irq12: psm0 42158 10 irq14: ata0 13047 3 irq16: fwohci0 1 0 irq17: skc0 4 0 irq22: ehci0+ 11525 2 irq23: pcm0 nve0 27 0 lapic0: timer 8350298 1999 Total 8438395 2020 Oliver From owner-freebsd-smp@FreeBSD.ORG Mon Oct 3 07:39:23 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D6C8716A41F for ; Mon, 3 Oct 2005 07:39:23 +0000 (GMT) (envelope-from robert@orcafat.com) Received: from slay.orcafat.com (1-1-6-11a.mfb.mlm.bostream.se [82.182.195.254]) by mx1.FreeBSD.org (Postfix) with ESMTP id 238CD43D49 for ; Mon, 3 Oct 2005 07:39:22 +0000 (GMT) (envelope-from robert@orcafat.com) Received: from localhost (localhost [127.0.0.1]) by slay.orcafat.com (Postfix) with ESMTP id 06DC6E9 for ; Mon, 3 Oct 2005 09:39:21 +0200 (CEST) Received: from slay.orcafat.com ([127.0.0.1]) by localhost (slay [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 22025-03 for ; Mon, 3 Oct 2005 09:39:19 +0200 (CEST) Received: from big.koppla.com (gw.koppla.com [213.115.165.161]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by slay.orcafat.com (Postfix) with ESMTP id 9CC00DC for ; Mon, 3 Oct 2005 09:39:19 +0200 (CEST) From: Robert Lindgren Date: Mon, 3 Oct 2005 09:39:18 +0200 User-Agent: KMail/1.8.2 Cc: freebsd-smp@freebsd.org References: <200510022209.38594.robert@orcafat.com> <4340636D.3090408@getdts.com> In-Reply-To: <4340636D.3090408@getdts.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline To: Undisclosed.Recipients: ; Message-Id: <200510030939.18882.robert@orcafat.com> X-Virus-Scanned: by amavisd-maia-1.0.0-rc5 (Debian) at orcafat.com Subject: Re: FreeBSD 5.4 network performance with SMP X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2005 07:39:24 -0000 On Monday 03 October 2005 00:47, Matt Hartzell wrote: > In my experience I had to disable the USB in the bios. > > Do a quick vmstat -i and note how many interrupts are on the uhci > device - with no devices even connected to the usb bus. uhci isn't loaded. It's not comipiled in my kernels at all. > > Also, what sysctl settings are you using? I use the standard settings. > > If your network supports it, how about using jumbo frames? We would I do that? It works OK with a UP kernel, so why should I need to use jumbo frames on a SMP kernel? I've never had to do that on 4.11 for example and I'm confident that jumbo frames won't help me at all. > > Robert Lindgren wrote: > >Hi All, > > > >I have a strange little problem on Dell 1800's. If HT,SMP is enabled in > > the kernel (5.4p7), performance on the integrated em0 is not especially > > good, half the speed of the current 100Mbit connection. This is true from > > the server and a little better to the server. On a pci rl0 card in the > > same machine performance is even worse. About 15Mbit from the server! > > > >Then I read a lot of people having poor performance from the rl0 cards and > > a lot of FreeBSD people bashing it for being crap, even in the source > > code for the module it's stated that these cards aren't so good. I guess > > that is right to some extent but not 15Mbit/sec. > > > >So when I rebuilt the kernel without HT and SMP I get full wirespeed from > > the em0 and the rl0 (100Mbit/sec). Is there anything I've been doing > > wrong to get hit by this poor performance with the SMP kernel. I read the > > disabling USB could improve performance but tried that with the HT kernel > > without any luck. > > > >The server has 1 3Ghz cpu, and I would prefer too run a HT enabled kernel, > > but not with that kind of performance, so please advice. The same > > machines (Dell 1800) run linux with HT enabled without any network > > performance degradation. > > > >Best regards > >Robert Lindgren > > > >_______________________________________________ > >freebsd-smp@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-smp > >To unsubscribe, send any mail to "freebsd-smp-unsubscribe@freebsd.org" From owner-freebsd-smp@FreeBSD.ORG Fri Oct 7 14:23:37 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD67916A420 for ; Fri, 7 Oct 2005 14:23:37 +0000 (GMT) (envelope-from tip@astudillo.net) Received: from dky118.neoplus.adsl.tpnet.pl (dky118.neoplus.adsl.tpnet.pl [83.24.28.118]) by mx1.FreeBSD.org (Postfix) with SMTP id F36D243D49 for ; Fri, 7 Oct 2005 14:23:27 +0000 (GMT) (envelope-from tip@astudillo.net) Received: from unknown (HELO Tom) (192.168.92.146) by dky118.neoplus.adsl.tpnet.pl with SMTP; Fri, 7 Oct 2005 02:42:04 +0200 Content-Transfer-Encoding: 7bit Message-Id: <3210613038.9799336835@dky118.neoplus.adsl.tpnet.pl> Content-Type: text/plain; charset=us-ascii To: freebsd-smp@freebsd.org From: Vivien Mayfield Date: Fri, 7 Oct 2005 14:23:27 +0000 (GMT) Subject: Your trusted source for prescription medication. X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2005 14:23:38 -0000 Get the most out of life! Viagra has helped millions of men! http://hdjhazm.dm0f1z1fja0ldvdoivdo0vvv.spinousdl.com/?mdoygtj UN-AMERICAN, adj. Wicked, intolerable, heathenish. Some things have to be believed to be seen. Crude, immoral, vulgar and senseless. A man thinks that by mouthing hard words he understands hard things. All exact science is dominated by the idea of approximation. Nothing can bring you peace but yourself. Verbosity leads to unclear, inarticulate things. From owner-freebsd-smp@FreeBSD.ORG Fri Oct 7 23:11:50 2005 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D40DD16A41F for ; Fri, 7 Oct 2005 23:11:50 +0000 (GMT) (envelope-from rogers@netrunner.nu) Received: from netrunner.nu (netrunner.nu [80.232.32.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D97843D45 for ; Fri, 7 Oct 2005 23:11:49 +0000 (GMT) (envelope-from rogers@netrunner.nu) Received: from netrunner.nu (localhost [127.0.0.1]) by netrunner.nu (8.13.3/8.13.3) with ESMTP id j97NBBoI058685 for ; Sat, 8 Oct 2005 01:11:11 +0200 (CEST) (envelope-from rogers@netrunner.nu) Received: from localhost (rogers@localhost) by netrunner.nu (8.13.3/8.13.3/Submit) with ESMTP id j97NBBhv058682 for ; Sat, 8 Oct 2005 01:11:11 +0200 (CEST) (envelope-from rogers@netrunner.nu) Date: Sat, 8 Oct 2005 01:11:11 +0200 (CEST) From: Roger Skjetlein To: freebsd-smp@freebsd.org Message-ID: <20051008011047.T58661@netrunner.nu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Poor network performance in 5.4 AMD64 Snap 8 with SMP support X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2005 23:11:51 -0000 Hi, I have a server with a Supermikro mainboard and two Xeon 3.0Ghz/2MB. When running kernel with SMP support the network performance is very poor, this is very visible on NFS traffic. When running UP kernel the performance is as espected and very good. I have tried some different GB nics and the problems is static and independent of modell/brand. I have also disabled as much as hardware possible, no usb, sio and etc. Typical indicator is also that the load on the box is skyrocketing when it starts to serve nfs client, but the cpu load is very load. Is there any way around this problem? Regards, RS