From owner-freebsd-current@FreeBSD.ORG Wed Jun 14 09:54:56 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6DDD716A561; Wed, 14 Jun 2006 09:54:56 +0000 (UTC) (envelope-from jb@what-creek.com) Received: from what-creek.com (what-creek.com [66.111.37.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7559243D6D; Wed, 14 Jun 2006 09:54:52 +0000 (GMT) (envelope-from jb@what-creek.com) Received: by what-creek.com (Postfix, from userid 102) id CDB9478C1D; Wed, 14 Jun 2006 09:54:49 +0000 (GMT) Date: Wed, 14 Jun 2006 09:54:49 +0000 From: John Birrell To: Robert Watson Message-ID: <20060614095449.GA82424@what-creek.com> References: <20060613021543.GA71283@what-creek.com> <20060613213617.GA78337@what-creek.com> <20060614103614.Y34121@fledge.watson.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060614103614.Y34121@fledge.watson.org> User-Agent: Mutt/1.4.2.1i Cc: current@freebsd.org Subject: Re: DTrace for FreeBSD - fbt output X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2006 09:54:56 -0000 On Wed, Jun 14, 2006 at 10:39:14AM +0100, Robert Watson wrote: > I don't suppose it's possible to get stack traces or fractional stack > traces (last 2-3 frames)? There are a number of comon consumers of > critical sections, but the number does seem inordinately high, especially > with respect to the number of calls of the common consumers. Critical > sections are acquired during access to per-cpu caches in UMA, and for > per-cpu stats in malloc(9), as well as in context switches and in > interrupts. DTrace has stack access actions. I still need to work on porting them, but walking up the stack is very much something that DTrace needs to do. > BTW, it looks like the below is running with invariants turned on? It may > be more interesting to run the below without invariants, since invariants > significantly changes the behavior of the memory allocation paths due to > memory trashing and additional bookkeeping. Yes, I have both invariants and witness turned on because they present the difficult case with respect to function recursion. If I innocently call functions from the DTrace probe context which turn out to call other non-dtrace-legal functions which can be, themselves, instrumented by FBT, a probe can crash the system. Witness often ends in tears at the moment. 8-( > > critical_enter 4364385478 > > critical_exit 4364385478 The points I wanted to make about these calls are: 1. critical_enter is called so often that the debug printf that /could/ be compiled into it should cause an increase in global warming if you tried. 8-) One of the big wins with DTrace is that you don't need (and shouldn't) compile printfs into the kernel. DTrace provides mechanisms to aggregate info (like the count example I gave) which condense data to a point where you can make use of it and the system is capable of logging it. 2. If you accept the fact the it is never feasible to enable the looging in critical_enter, then the function reduces to just curthread->td_critnest++ which would be better implemented as a macro, thereby saving many billions of processor cycles over a buildworld. I need to look further at what critical_exit is doing the most of the time. I had hoped to be able to get the info to post here , but my ISP has just decided to change backbones which is causing extreme grief and I'm changing to another ISP -- my 4th change in the last 12 months. ADSL ISPs in Australia suck. Grumble. -- John Birrell