From owner-freebsd-stable@FreeBSD.ORG Sat Mar 11 02:10:46 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8830916AF01 for ; Sat, 11 Mar 2006 02:10:46 +0000 (GMT) (envelope-from grafan@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.197]) by mx1.FreeBSD.org (Postfix) with ESMTP id 06BFD48DEC for ; Fri, 10 Mar 2006 19:02:24 +0000 (GMT) (envelope-from grafan@gmail.com) Received: by xproxy.gmail.com with SMTP id s11so580539wxc for ; Fri, 10 Mar 2006 11:02:18 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=CI2teEB21ta//xuI4vRebzKaJa79Buk0/nsBjsiqdf34GAAuYG2czGICO52TCS2bRzsA2gVXc9gAzp9DmtOKpCY42efUGOy6sWwGx9F3fQ8mcbJbszpt1MOjxS4DaPvrluZ9dEmqYrqkSsArp/spU53WhvKljUi9qURvielVlLk= Received: by 10.70.60.17 with SMTP id i17mr167724wxa; Fri, 10 Mar 2006 11:02:18 -0800 (PST) Received: by 10.70.125.17 with HTTP; Fri, 10 Mar 2006 11:02:18 -0800 (PST) Message-ID: <6eb82e0603101102ke9cec8lde806f466a87e7bf@mail.gmail.com> Date: Fri, 10 Mar 2006 14:02:18 -0500 From: "Rong-En Fan" To: stable@freebsd.org In-Reply-To: <6eb82e0603100940s7d505e26w5351d708b714aefe@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <6eb82e0603100940s7d505e26w5351d708b714aefe@mail.gmail.com> Cc: Subject: Re: nfsclient process stucks in nfsaio X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Mar 2006 02:10:46 -0000 Hi, forget to mention all the clients/servers here are SMP kernel. After some Googling, a post on current@ 2005/01/12 "NFS problems, locking up" is hightly related to my situation. An workaround is to set debug.mpsafenet=3D0, just verified this indeed works. Now I'm turning on INVARIANTS, WITNESS to see if there are some output. However, I'm afriad that I can not get a serial console access to these machines (and thus no ddb output :( ). Thanks, Rong-En Fan On 3/10/06, Rong-En Fan wrote: > Hi, > > After upgrading several our nfs clients from 5.4-RELEASE to 6.0-RELEASE > and some are now 6.1-PRERELEASE (a weeks ago). From time to time, > we saw some processes stuck in nfsaio, and unkillable. These processes > generate lots of traffic to nfs server (write to nfs, but nfs server's di= sk does > not really in write. from netstat, client sends ~100Mbps, on nfs server, = iostat > does not show me ~12.5MB/s). The nfsd on the server side is either in RUN > or in ufs state. Server is running 5.5-PRELEASE as of yesterday. > > Client mount options: rw,nosuid,bg,intr,nodev. Both client and server > are running > rpc.lockd, rpc.statd. I'm sure it's not related to any locking problems. > > I have another set of nfs server/client both running 6.0-RELEASE. And I c= an > easily reproduce this situation on these two boxesnes, just by running > > dd if=3D/dev/zero of=3D/nfs/ooo bs=3D1m > > If I do not add bs=3D1m, it works fine. Of all the boxes I mentioned abov= e, > I did not do anything special to kernel config, i.e., they are GENERIC w/= o > unnecessary devices and w/ firewal. Basically, I can do anything on thes= e > two boxes (they are not in production mode). Any suggestion are welcome. > > Thanks, > Rong-En Fan >