From owner-freebsd-questions@FreeBSD.ORG Tue Feb 28 07:11:00 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 662F61065670 for ; Tue, 28 Feb 2012 07:11:00 +0000 (UTC) (envelope-from msabramo@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 23B1B8FC0A for ; Tue, 28 Feb 2012 07:10:59 +0000 (UTC) Received: by obbwc7 with SMTP id wc7so1650049obb.13 for ; Mon, 27 Feb 2012 23:10:59 -0800 (PST) Received-SPF: pass (google.com: domain of msabramo@gmail.com designates 10.182.155.68 as permitted sender) client-ip=10.182.155.68; Authentication-Results: mr.google.com; spf=pass (google.com: domain of msabramo@gmail.com designates 10.182.155.68 as permitted sender) smtp.mail=msabramo@gmail.com; dkim=pass header.i=msabramo@gmail.com Received: from mr.google.com ([10.182.155.68]) by 10.182.155.68 with SMTP id vu4mr6239900obb.61.1330413059605 (num_hops = 1); Mon, 27 Feb 2012 23:10:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=j9jLgPqBZPmFxYPIQrRk4qJqdhjhTa2UNFSB7lylohM=; b=E83g1dEV83+QEJDZ9tmJquR4HCpvL/L1mCJqBoBb1WUyc/94XM+/dVwzIL9RH/8o2z OG+B0HEpH5Sv4ntLBhxUqZthu83wco+ALQcIxgUt4MqNlubPVZS1Hxsu3xfgEnEgoZ4y qxKY/NBgLriE41Zw7lZXiCmil7Df1TiRcvIsA= MIME-Version: 1.0 Received: by 10.182.155.68 with SMTP id vu4mr5538870obb.61.1330413059517; Mon, 27 Feb 2012 23:10:59 -0800 (PST) Received: by 10.60.24.7 with HTTP; Mon, 27 Feb 2012 23:10:59 -0800 (PST) In-Reply-To: References: Date: Mon, 27 Feb 2012 23:10:59 -0800 Message-ID: From: Marc Abramowitz To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: DTrace userland X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Feb 2012 07:11:00 -0000 Another strange behavior: [Tab 1] $ /bin/sleep 300 & [1] 1806 [Tab 2] $ sudo dtrace -n 'pid1806:sleep::entry' $ echo $? 158 [Tab 1] [1]+ Killed: 9 /bin/sleep 300 Something seems very wrong that DTrace is killing processes and causing kernel panics. Marc On Mon, Feb 27, 2012 at 10:22 PM, Marc Abramowitz wrote: > I'm using FreeBSD 9.0 on amd64 in VMware Fusion and trying to DTrace > userland programs. I think I must be doing something wrong. > > I recompiled my kernel and world, following the instructions at > http://wiki.freebsd.org/DTrace and I've read > http://wiki.freebsd.org/DTrace/userland: > > The test.c pid provider example worked fine for me: > > $ sudo dtrace -s pid.d -c ./test > dtrace: script 'pid.d' matched 2 probes > dtrace: buffer size lowered to 1m > CPU ID FUNCTION:NAME > 0 43030 main:entry > 0 43031 sleep:entry > 0 43031 sleep:entry > 0 43031 sleep:entry > > As does a simple probe of test.c specified with the -n option: > > [marca@freebsd9-0 ~]$ sudo dtrace -n 'pid$target:test:main:entry' -c > ./test > dtrace: description 'pid$target:test:main:entry' matched 1 probe > dtrace: buffer size lowered to 1m > CPU ID FUNCTION:NAME > 0 43030 main:entry > > When I start trying to dtrace other programs, things don't go so well... > > $ sudo dtrace -n ":::entry" -c /usr/local/bin/python > Python 2.4.5 (#2, Dec 5 2011, 15:19:09) > [GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9 > Type "help", "copyright", "credits" or "license" for more information. > >>> import os > >>> os.getpid() > 1603 > >>> > dtrace: failed to control pid 1603: process exited with status 0 > > $ sudo dtrace -n 'pid$target:::entry' -c '/bin/cat hello_world.txt' > dtrace: description 'pid$target:::entry' matched 3315 probes > dtrace: buffer size lowered to 1m > CPU ID FUNCTION:NAME > 0 43448 _rtld_bind:entry > 0 43903 rlock_acquire:entry > 0 43125 def_thread_set_flag:entry > (Had to hit Ctrl-C to exit; it never displayed hello_world.txt to stdout) > > [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo make install > ... > [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n > 'pid$target:::entry' -c '/usr/local/bin/gcat config.log' > dtrace: description 'pid$target:::entry' matched 3823 probes > dtrace: buffer size lowered to 1m > CPU ID FUNCTION:NAME > 0 43524 _rtld_bind:entry > 0 43979 rlock_acquire:entry > 0 43201 def_thread_set_flag:entry > ^C > > $ sudo dtrace -n 'pid$target:cat:main:entry' -c '/bin/cat hello_world.txt' > causes a kernel panic. > According to the core.txt file, it was a "Fatal trap 10: trace trap while > in kernel mode" and here's the KDB backtrace: > > KDB: stack backtrace: > #0 0xffffffff8089025e at kdb_backtrace+0x5e > #1 0xffffffff80858ce7 at panic+0x187 > #2 0xffffffff80b4bf20 at trap_fatal+0x290 > #3 0xffffffff80b4c540 at trap+0x180 > #4 0xffffffff80b36963 at calltrap+0x8 > #5 0xffffffff8162583d at dtrace_assfail+0x2d > #6 0xffffffff8188aa2e at fasttrap_provider_free+0x1de > #7 0xffffffff8188ad13 at fasttrap_pid_cleanup_cb+0x1c3 > #8 0xffffffff8086dfa1 at softclock+0x3a1 > #9 0xffffffff8082d724 at intr_event_execute_handlers+0x104 > #10 0xffffffff8082eee4 at ithread_loop+0xa4 > #11 0xffffffff8082a34f at fork_exit+0x11f > #12 0xffffffff80b36e8e at fork_trampoline+0xe > > [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n > 'pid$target:gcat::entry' -c '/usr/local/bin/gcat config.log' > (Another kernel panic) > > I can provide full crash dumps if necessary. > > Any idea what's going on here? > > Cheers, > Marc > > > >