Date: Tue, 28 Feb 2012 12:24:41 -0800 From: Marc Abramowitz <msabramo@gmail.com> To: freebsd-questions@freebsd.org Subject: Re: DTrace userland Message-ID: <CAAgNOZ0U52Va0gNWiXx3C_pTssqr-P%2BoEqSr0mwkzCvrQLEuMw@mail.gmail.com> In-Reply-To: <CAAgNOZ2Vx9JsNnBmQaojFjfL7CTCbRTQm_bzT1-DPk3889QgGg@mail.gmail.com> References: <CAAgNOZ2niHqw6SLWUkmTch-bKh1HPYrUX20QvLYPF-SKZ7wg8A@mail.gmail.com> <CAAgNOZ2Vx9JsNnBmQaojFjfL7CTCbRTQm_bzT1-DPk3889QgGg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Here's another way to cause a kernel panic: [marca@freebsd9-0 ~]$ sudo kldload dtraceall [marca@freebsd9-0 ~]$ cat -n test.c 1 #include <stdio.h> 2 3 int main() 4 { 5 sleep(15); 6 7 FILE *fp = fopen("hello.txt", "w"); 8 fprintf(fp, "Here I am at %s:%d.\n", __FILE__, __LINE__); 9 fclose(fp); 10 } [marca@freebsd9-0 ~]$ gcc test.c -o test [marca@freebsd9-0 ~]$ sudo dtrace -n 'pid$target:test:main:entry' -c ./test dtrace: description 'pid$target:test:main:entry' matched 1 probe dtrace: buffer size lowered to 1m CPU ID FUNCTION:NAME 0 43030 main:entry (Kernel panic! After reboot....) [marca@freebsd9-0 ~]$ cat hello.txt Here I am at test.c:8. Interestingly, the crash doesn't occur until after the sleep and the fprintf call, so it looks the kernel panic happens as a result of the traced process _exiting_... Marc On Mon, Feb 27, 2012 at 11:10 PM, Marc Abramowitz <msabramo@gmail.com>wrote: > Another strange behavior: > > [Tab 1] > $ /bin/sleep 300 & > [1] 1806 > > [Tab 2] > $ sudo dtrace -n 'pid1806:sleep::entry' > $ echo $? > 158 > > [Tab 1] > [1]+ Killed: 9 /bin/sleep 300 > > Something seems very wrong that DTrace is killing processes and causing > kernel panics. > > Marc > > On Mon, Feb 27, 2012 at 10:22 PM, Marc Abramowitz <msabramo@gmail.com>wrote: > >> I'm using FreeBSD 9.0 on amd64 in VMware Fusion and trying to DTrace >> userland programs. I think I must be doing something wrong. >> >> I recompiled my kernel and world, following the instructions at >> http://wiki.freebsd.org/DTrace and I've read >> http://wiki.freebsd.org/DTrace/userland: >> >> The test.c pid provider example worked fine for me: >> >> $ sudo dtrace -s pid.d -c ./test >> dtrace: script 'pid.d' matched 2 probes >> dtrace: buffer size lowered to 1m >> CPU ID FUNCTION:NAME >> 0 43030 main:entry >> 0 43031 sleep:entry >> 0 43031 sleep:entry >> 0 43031 sleep:entry >> >> As does a simple probe of test.c specified with the -n option: >> >> [marca@freebsd9-0 ~]$ sudo dtrace -n 'pid$target:test:main:entry' -c >> ./test >> dtrace: description 'pid$target:test:main:entry' matched 1 probe >> dtrace: buffer size lowered to 1m >> CPU ID FUNCTION:NAME >> 0 43030 main:entry >> >> When I start trying to dtrace other programs, things don't go so well... >> >> $ sudo dtrace -n ":::entry" -c /usr/local/bin/python >> Python 2.4.5 (#2, Dec 5 2011, 15:19:09) >> [GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import os >> >>> os.getpid() >> 1603 >> >>> >> dtrace: failed to control pid 1603: process exited with status 0 >> >> $ sudo dtrace -n 'pid$target:::entry' -c '/bin/cat hello_world.txt' >> dtrace: description 'pid$target:::entry' matched 3315 probes >> dtrace: buffer size lowered to 1m >> CPU ID FUNCTION:NAME >> 0 43448 _rtld_bind:entry >> 0 43903 rlock_acquire:entry >> 0 43125 def_thread_set_flag:entry >> (Had to hit Ctrl-C to exit; it never displayed hello_world.txt to stdout) >> >> [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo make install >> ... >> [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n >> 'pid$target:::entry' -c '/usr/local/bin/gcat config.log' >> dtrace: description 'pid$target:::entry' matched 3823 probes >> dtrace: buffer size lowered to 1m >> CPU ID FUNCTION:NAME >> 0 43524 _rtld_bind:entry >> 0 43979 rlock_acquire:entry >> 0 43201 def_thread_set_flag:entry >> ^C >> >> $ sudo dtrace -n 'pid$target:cat:main:entry' -c '/bin/cat hello_world.txt' >> causes a kernel panic. >> According to the core.txt file, it was a "Fatal trap 10: trace trap while >> in kernel mode" and here's the KDB backtrace: >> >> KDB: stack backtrace: >> #0 0xffffffff8089025e at kdb_backtrace+0x5e >> #1 0xffffffff80858ce7 at panic+0x187 >> #2 0xffffffff80b4bf20 at trap_fatal+0x290 >> #3 0xffffffff80b4c540 at trap+0x180 >> #4 0xffffffff80b36963 at calltrap+0x8 >> #5 0xffffffff8162583d at dtrace_assfail+0x2d >> #6 0xffffffff8188aa2e at fasttrap_provider_free+0x1de >> #7 0xffffffff8188ad13 at fasttrap_pid_cleanup_cb+0x1c3 >> #8 0xffffffff8086dfa1 at softclock+0x3a1 >> #9 0xffffffff8082d724 at intr_event_execute_handlers+0x104 >> #10 0xffffffff8082eee4 at ithread_loop+0xa4 >> #11 0xffffffff8082a34f at fork_exit+0x11f >> #12 0xffffffff80b36e8e at fork_trampoline+0xe >> >> [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n >> 'pid$target:gcat::entry' -c '/usr/local/bin/gcat config.log' >> (Another kernel panic) >> >> I can provide full crash dumps if necessary. >> >> Any idea what's going on here? >> >> Cheers, >> Marc >> >> >> >> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAgNOZ0U52Va0gNWiXx3C_pTssqr-P%2BoEqSr0mwkzCvrQLEuMw>