From owner-freebsd-current@FreeBSD.ORG Mon Jan 26 07:09:29 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2213E16A4CE for ; Mon, 26 Jan 2004 07:09:29 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33BD643D45 for ; Mon, 26 Jan 2004 07:09:27 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i0QF7DUd011725; Mon, 26 Jan 2004 10:07:13 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i0QF7DXs011722; Mon, 26 Jan 2004 10:07:13 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 26 Jan 2004 10:07:13 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Harald Schmalzbauer In-Reply-To: <200401261310.15763@harrymail> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org Subject: Re: 5.2-rel NFS lockup and networking performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 15:09:29 -0000 On Mon, 26 Jan 2004, Harald Schmalzbauer wrote: > I just upgraded one of my servers from 5.1 to 5.2. Now when I dd from a > linux box on a nfs-mounted 5.2 testfile writing 100kKb takes about 5 > Minutes because the machine repeatedly "locks up". It seems every disk > access is blocked for a while. (on the linux(debian woody) box: "dd > if=/dev/zero of=/mnt/nfs/testfile bs=1k count=100000") But not only > that NFS is broken with linux (I can do the same from umy 5.2-current > without locking up the 5.2-rel machine), also the performence is even > more awful than with 5.1. Max transfers are about 4.5Mbyte/s and with > samba I can get 3.5Mbyte/s. On the same machine with 5.1 it was about > 8Mbyte/ s for NFS and about 5.5Mbyte/s for samba. I think I'll have to > switch back to 4.9 since I can remember having had more than 9Mbyte/s > even for samba with much older hardware arround FreeBSD 4.4. Can anyone > confirm that 4.9 is still performing like this? > > Does anybody know about this NFS bug and this really weak networking > performance? Regarding performance: Is this an SMP box? If not, if you haven't already, could you recompile your kernel without 'device apic' and 'options SMP'? Also, there have been recent reports that a number of older SMP systems are having interrupt storm problems with recent ACPI drops and the new interrupt code. If your system meets this description, you might also try disabling ACPI. Then rerun your tests and see what, if any, has changed. Regarding the lockup: could you try compiling your kernel with 'options BREAK_TO_DEBUGGER' -- when the system appears to freeze, hit Ctrl-Alt-Esc on the console to drop to DDB. If this works, the output of 'ps' would be very helpful. If this doesn't work, or you don't feel like typing in pages of ps output, try hooking up a second box as a serial console -- using a serial break gets into the debugger more reliably, and you can copy and paste. :-) I'd also be interested in seeing stack traces for the nfsd processes on the NFS server. You can use "trace pid" in DDB to generate those traces. Also of interest is the output of "show lockedvnods". Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research