Date: Thu, 4 Dec 2014 18:34:39 -0800 From: 'Mark Johnston' <markj@freebsd.org> To: dteske@FreeBSD.org Cc: 'Julian Elischer' <julian@freebsd.org>, freebsd-dtrace@freebsd.org Subject: Re: DTrace script to trace processes entering vfs::vop_remove Message-ID: <20141205023439.GA8710@charmander.picturesperfect.net> In-Reply-To: <04f001d01004$abaefd30$030cf790$@FreeBSD.org> References: <032e01d00f4f$98a04e20$c9e0ea60$@FreeBSD.org> <20141204004501.GB29167@charmander.picturesperfect.net> <03ed01d00f66$89db0ee0$9d912ca0$@FreeBSD.org> <20141204182232.GB81713@charmander.picturesperfect.net> <04f001d01004$abaefd30$030cf790$@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Dec 04, 2014 at 12:55:40PM -0800, dteske@FreeBSD.org wrote: > > > > -----Original Message----- > > From: Mark Johnston [mailto:markjdb@gmail.com] On Behalf Of 'Mark > > Johnston' > > Sent: Thursday, December 4, 2014 10:23 AM > > To: dteske@FreeBSD.org > > Cc: freebsd-dtrace@freebsd.org; 'Julian Elischer' > > Subject: Re: DTrace script to trace processes entering vfs::vop_remove > > > > On Wed, Dec 03, 2014 at 06:03:45PM -0800, dteske@FreeBSD.org wrote: > > > > > > > > > > -----Original Message----- > > > > From: Mark Johnston [mailto:markjdb@gmail.com] On Behalf Of Mark > > > > Johnston > > > > Sent: Wednesday, December 3, 2014 4:45 PM > > > > To: dteske@FreeBSD.org > > > > Cc: freebsd-dtrace@freebsd.org; 'Julian Elischer' > > > > Subject: Re: DTrace script to trace processes entering vfs::vop_remove > > > > > > > > On Wed, Dec 03, 2014 at 03:19:31PM -0800, dteske@FreeBSD.org wrote: > > > > > Hi markj, list, > > > > > > > > > > I wrote a script for $work to help me find out "who on Earth > > > > > keeps deleting files XYZ?" from a particular storage server. > > > > > > > > > > Please find attached a copy of watch_vop_remove.d which > > > > > has the following sample output: > > > > > > > > > > 2014 Dec 3 11:58:52 rm[75596]: /tmp/foo > > > > > -+= 72846 0.0 -bash > > > > > \-+= 75589 0.0 /bin/bash /usr/home/support/bash_script > > > > > \-+= 75596 0.0 rm -f /tmp/foo > > > > > > > > > > The above sample output was displayed when executing the following > > shell > > > > > script: > > > > > > > > > > #!/bin/bash > > > > > touch /tmp/foo > > > > > rm -f /tmp/foo > > > > > > > > > > The output format displayed for each vop_remove() call is as > follows: > > > > > > > > > > DATE process[PID]: PATH_TO_DELETE > > > > > -+= GPID UID.GID grandparent_process [arguments (up to 3)] > > > > > \-+= PPID UID.GID parent_process [arguments (up to 3)] > > > > > \-+= PID UID.GID process [arguments (up to 3)] > > > > > > > > This is neat. I just had a few comments: > > > > - You can use walltimestamp when printing the date and time, instead > of > > > > timestamp + blah. > > > > > > I read that online as well, however: > > > walltimestamp appears to _always_ be zero. > > > > Right, it wasn't working properly on 8.0. :( > > > > gnn committed a fix for that as r238537. > > > > > > > > > > > > - It's possible to get the full argv of the current process with > > > > curpsinfo->pr_psargs. It can be done for other processes too; see > > > > /usr/lib/dtrace/psinfo.d. (This might not be true depending on the > > > > FreeBSD version you're on.) > > > > > > Thanks! I'll have a look. > > > > > > > - Running this script with a make -j4 buildkernel causes dtrace to run > > > > out of dynamic variable space. > > > > > > > > > > Any recommendation on how to fix that? > > > > > > #pragma D option dynvarsize=what_exactly? > > > (16m causes a warning that it's lowering the dynamic variable memory) > > > > It looks like a leak - once I start seeing the errors, no file removals > > are logged at all. Dynamic variables need to be set to 0 once they're > > finished with in order to release the consumed memory. > > > > Thanks! Should be fixed in the latest (attached) version > (watch_vop_remove2.d). > > However, I read here: > http://wikis.oracle.com/display/DTrace/Variables > > Quote: Always assign zero to associative array elements that are no longer > in use. > > And I read some more about the different variable types in DTrace: > http://dtrace.org/blogs/brendan/2011/11/25/dtrace-variable-types/ > > It would appear that I've solved the issue by getting rid of associative > arrays. > > Can you give the latest (attached) a try? Yup, it appears to fix the issue for me. Thanks! -Mark
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141205023439.GA8710>