From owner-freebsd-performance@FreeBSD.ORG Tue Apr 7 01:48:56 2015 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 123B9C4F for ; Tue, 7 Apr 2015 01:48:56 +0000 (UTC) Received: from omnigroup.com (omnigroup.com [198.151.161.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "omnigroup.com", Issuer "The Omni Group CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EC535768 for ; Tue, 7 Apr 2015 01:48:55 +0000 (UTC) Received: from machamp.omnigroup.com (machamp.omnigroup.com [198.151.161.135]) by omnigroup.com (Postfix) with ESMTP id 189F31FFF0D9 for ; Mon, 6 Apr 2015 18:40:06 -0700 (PDT) Received: from [10.4.3.73] (pfsense.omnigroup.com [198.151.161.131]) by machamp.omnigroup.com (Postfix) with ESMTPSA id 13B49DF86EF for ; Mon, 6 Apr 2015 18:37:58 -0700 (PDT) From: Wim Lewis Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Capturing a snapshot of user stacks Message-Id: <2B4F43A6-CB6A-4183-8847-968D6EA4CA9A@omnigroup.com> Date: Mon, 6 Apr 2015 18:40:05 -0700 To: freebsd-performance@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 01:48:56 -0000 I=92m looking for a tool which can capture all (or a selected subset) of = the user stacks on the system, even for tasks that are blocked in a = syscall at the time. Dtrace can capture a user stack but only when an event happens in that = task; and procstat -k can capture all the stacks but only the kernel = stacks. Is there a way to, in effect, have a dtrace probe fire in every = blocked thread? Or another way to get this information? A little explanation: I=92m trying to discover why a system occasionally = becomes slow. When this happens, I want to capture some profiling = information for later perusal. I can get a lot of information from = dtrace (a la = http://www.brendangregg.com/FlameGraphs/offcpuflamegraphs.html ) and = pmc, but that won=92t tell me anything about a process that=92s blocked = the whole time, or is blocked at the start of the interval but unblocks = partway through (as is likely to be the case if I only start collecting = data once the problem occurs). Any ideas? Wim Lewis / wiml@omnigroup.com