From owner-freebsd-dtrace@FreeBSD.ORG  Wed Aug 21 05:00:17 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 4C9A49D;
 Wed, 21 Aug 2013 05:00:17 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com
 [IPv6:2607:f8b0:400d:c00::230])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id F174D246F;
 Wed, 21 Aug 2013 05:00:16 +0000 (UTC)
Received: by mail-qa0-f48.google.com with SMTP id o19so187295qap.0
 for <multiple recipients>; Tue, 20 Aug 2013 22:00:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:date:from:to:subject:message-id:mime-version:content-type
 :content-disposition:user-agent;
 bh=J1HPrLgQ8fQjbrHFM0ua/vcLKtp92r1grSvk1DL8vPs=;
 b=dGYKNu2tml+M04HzChzYSYMQUls5jZcrGik8IPb0CX31LD/PzTHCEN5jzcGAm8J+gk
 Gh5HY6Zl0lVUjgmTY9E25Upc3vHbwJpLSaI+vqCchPoDZqysUCnznuGOE2NM2I3yQBoN
 BiD/BpGNE272P9TbRBx0nj0CN5h7w6lpALwYSxIQRIbx4tpqDG0+5fp8nORwRRZxT16L
 zRy1MMhRKtF6/qttbjPXYHZUbbZAy9D7YptnVWYjGUlyse/JIe59/h3HrSQoQ4cnFbAL
 HtPqjMtQFfCUxNwnFOi+/fTaIHfg1TFcsoRka6et4soeJ+foVBsdGvTYpoAiT5m/ziqf
 mk7A==
X-Received: by 10.224.80.9 with SMTP id r9mr6255880qak.89.1377061215755;
 Tue, 20 Aug 2013 22:00:15 -0700 (PDT)
Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13])
 by mx.google.com with ESMTPSA id m8sm8861411qay.0.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Tue, 20 Aug 2013 22:00:15 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Wed, 21 Aug 2013 01:00:10 -0400
From: Mark Johnston <markj@freebsd.org>
To: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
Subject: DTrace network providers
Message-ID: <20130821045926.GA17196@raichu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Aug 2013 05:00:17 -0000

Hello!

I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
the Solaris documentation here:

https://wikis.oracle.com/display/DTrace/ip+Provider
https://wikis.oracle.com/display/DTrace/tcp+Provider
https://wikis.oracle.com/display/DTrace/udp+Provider

My implementation of these providers makes use of dynamic translators,
for which FreeBSD support was added in r254468; this patch won't compile
with earlier revisions. The use of dynamic translators means that
existing DTrace scripts which use these providers will just work when run
on FreeBSD - no modifications needed. In particular, all of the examples
in the links above will work properly on FreeBSD with my diff.

I've collected a bunch of example scripts for these providers and placed
them here:

http://people.freebsd.org/~markj/dtrace/network-providers/

To run one you just need to execute "dtrace -s <script>".

In general these providers make it trivial to monitor or count packets
and bytes on a per-host/port/protocol/interface basis. One can also do
neat things like watch TCP connection state transitions in real time
(tcpstate.d) and measure connection latency. All of the probes
correspond to logical events in their respective protocol
implementations; all of the providers have send and receive probes, and
the tcp provider has a few more.

I didn't have to make any major changes to add support for these
providers, but I've made a few small tweaks:

1. Add a little helper function for TCP state changes instead of
   sprinkling a bunch of SDT_PROBE calls throughout the code.
2. Set the IPv6 payload size in TCP and UDP before calling ip6_output().
   This is done for the send probes so that the ipinfo_t argument
   can be used to get the payload size. It's not quite correct since it
   doesn't include the length of IPv6 option headers (they aren't known
   yet), but I think that's ok for the purposes of these providers.
3. Set the IPv4 header version in udp_output(). This is already done for
   IPv6.

I'm hoping that none of this (or the rest of the diff) is controversial,
but I wanted to make sure, so any review would be really appreciated.

The patch is here:

http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff

It depends on r254468. To use it, just recompile the kernel (assuming
that KDTRACE_HOOKS is enabled) and copy
$SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
/usr/lib/dtrace. Then the example scripts I linked above should just
work.

Thanks!
-Mark

From owner-freebsd-dtrace@FreeBSD.ORG  Wed Aug 21 06:45:02 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 9853F589
 for <freebsd-dtrace@freebsd.org>; Wed, 21 Aug 2013 06:45:02 +0000 (UTC)
 (envelope-from brendan.gregg@joyent.com)
Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com
 [209.85.220.53])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 720392B23
 for <freebsd-dtrace@freebsd.org>; Wed, 21 Aug 2013 06:45:02 +0000 (UTC)
Received: by mail-pa0-f53.google.com with SMTP id lb1so473180pab.26
 for <freebsd-dtrace@freebsd.org>; Tue, 20 Aug 2013 23:44:56 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=google.com; s=20120113;
 h=x-gm-message-state:mime-version:in-reply-to:references:date
 :message-id:subject:from:to:cc:content-type;
 bh=TJ2/FyNp0IQDh+qOE7SYETPkg2/rAGZ0z69g7yZy4ts=;
 b=pR52z0a03bmZvDVD57Rrgbax36P6Gu/tPFkS/Mi+BmHsZZKx2X8PKCeu3v3mjJp/Um
 lP+Xze8AmIyFROYUaUZgc2WJn0SejWncCNNrBQb35frpQlDAH1DnmNshZA+YvNcHBg2z
 SRjsp6/GQYnF+GCr/t0hvbW9xTlIlwx4ZYr1J+IXFBoDufqodEJp5BO5mWOJ/V00PEDU
 XOMG8CeKQNHfB0JKctY6La2QpEuTRWLX4Fsh2RSLRhyz1xMbFTQ1prLAWrV4hIX9tTkZ
 5l3HhaAP8UKQkqpnPqE4QVA9S95O18gHesSQwGWEw4/tQTkTO1+uIaCGNJ0IptkLHaLs
 fxvg==
X-Gm-Message-State: ALoCoQkh9XSsVLlXvPRF0HoStWhWou7xSIPjR19RL3dJ28TEmgKi+2cgrB8mRNafgMW/A3Vrk2KD
MIME-Version: 1.0
X-Received: by 10.67.24.7 with SMTP id ie7mr7810330pad.112.1377067496223; Tue,
 20 Aug 2013 23:44:56 -0700 (PDT)
Received: by 10.70.55.34 with HTTP; Tue, 20 Aug 2013 23:44:56 -0700 (PDT)
In-Reply-To: <20130821045926.GA17196@raichu>
References: <20130821045926.GA17196@raichu>
Date: Tue, 20 Aug 2013 23:44:56 -0700
Message-ID: <CA+XzFFhw9beGi8=7heCY07dExvno=oMBYgzUrpht2LYXSbutqg@mail.gmail.com>
Subject: Re: DTrace network providers
From: Brendan Gregg <brendan.gregg@joyent.com>
To: Mark Johnston <markj@freebsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.14
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Aug 2013 06:45:02 -0000

G'Day,

On Tue, Aug 20, 2013 at 10:00 PM, Mark Johnston <markj@freebsd.org> wrote:

> Hello!
>
> I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> the Solaris documentation here:
>
> https://wikis.oracle.com/display/DTrace/ip+Provider
> https://wikis.oracle.com/display/DTrace/tcp+Provider
> https://wikis.oracle.com/display/DTrace/udp+Provider
>
>
Fantastic!


> My implementation of these providers makes use of dynamic translators,
> for which FreeBSD support was added in r254468; this patch won't compile
> with earlier revisions. The use of dynamic translators means that
> existing DTrace scripts which use these providers will just work when run
> on FreeBSD - no modifications needed. In particular, all of the examples
> in the links above will work properly on FreeBSD with my diff.
>
> I've collected a bunch of example scripts for these providers and placed
> them here:
>
> http://people.freebsd.org/~markj/dtrace/network-providers/
>
>
They look familiar. :-) Which is a good sign for this port, as that's
what's supposed to happen: the same scripts run on any OS.

To run one you just need to execute "dtrace -s <script>".
>

Or chmod 755 the ones with interpreter lines.


> In general these providers make it trivial to monitor or count packets
> and bytes on a per-host/port/protocol/interface basis. One can also do
> neat things like watch TCP connection state transitions in real time
> (tcpstate.d) and measure connection latency. All of the probes
> correspond to logical events in their respective protocol
> implementations; all of the providers have send and receive probes, and
> the tcp provider has a few more.
>
> I didn't have to make any major changes to add support for these
> providers, but I've made a few small tweaks:
>
> 1. Add a little helper function for TCP state changes instead of
>    sprinkling a bunch of SDT_PROBE calls throughout the code.
>

Makes sense.


> 2. Set the IPv6 payload size in TCP and UDP before calling ip6_output().
>    This is done for the send probes so that the ipinfo_t argument
>    can be used to get the payload size. It's not quite correct since it
>    doesn't include the length of IPv6 option headers (they aren't known
>    yet), but I think that's ok for the purposes of these providers.
>

Excluding the IP header length is correct for ipinfo_t->ip_plength, which
is just supposed to be the payload length.

I see the patch makes an assumption about IPv4 header size being 20 bytes,
instead of picking it from the header (IHL), like:

        ip_plength = ntohs(I->ipha_length) -
            ((I->ipha_version_and_hdr_length & 0xf) << 2);

3. Set the IPv4 header version in udp_output(). This is already done for
>    IPv6.
>
> I'm hoping that none of this (or the rest of the diff) is controversial,
> but I wanted to make sure, so any review would be really appreciated.
>
> The patch is here:
>
>
> http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
>
> It depends on r254468. To use it, just recompile the kernel (assuming
> that KDTRACE_HOOKS is enabled) and copy
> $SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
> /usr/lib/dtrace. Then the example scripts I linked above should just
> work.
>

Great work Mark!

Here's some (from many) one-liners that this should make possible (more in
the DTrace book):

Who is connecting to what:
dtrace -n 'tcp:::accept-established { @[args[3]->tcps_raddr,
args[3]->tcps_lport] = count(); }'

Who isn't connecting to what:
dtrace -n 'tcp:::accept-refused { @[args[2]->ip_daddr, args[4]->tcp_sport]
= count(); }'

What am I connecting to?
dtrace -n 'tcp:::connect-established { @[args[3]->tcps_raddr ,
args[3]->tcps_rport] = count(); }'

IP payload bytes for TCP send, size distribution by destination address:
dtrace -n 'tcp:::send { @[args[2]->ip_daddr] =
quantize(args[2]->ip_plength); }'

Many of these are very low overhead ways to understand network behavior -
lower than sniffing every packet. Plus you can pull out kernel state that
is never seen on the wire.

Brendan

-- 
Brendan Gregg, Joyent                      http://dtrace.org/blogs/brendan

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 00:40:53 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id CB8401AB;
 Thu, 22 Aug 2013 00:40:53 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com
 [IPv6:2607:f8b0:400d:c01::232])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 7A15A2A13;
 Thu, 22 Aug 2013 00:40:53 +0000 (UTC)
Received: by mail-qc0-f178.google.com with SMTP id s1so649959qcw.37
 for <multiple recipients>; Wed, 21 Aug 2013 17:40:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-type:content-disposition:in-reply-to:user-agent;
 bh=kTIvIZ51MixjWaQ/QoOqOAnakrsvo5xoAsbMOBi7r6k=;
 b=k5V47JUBbsoFmdonpf2vib70rMXbmN0YaUvbHi3vPwsdUtvYlWJFbQ8VrvGH+LYkQ9
 cYb+gGxk7F+UUpbYxwufYKzqMBgcr/vDyqc2l4b00SUXEUtTS15AK3/cXxDDzAbgYAjm
 4QFSlTNeczqJa0hf9dto/0XCh+Leu6FwMfGPyV1UmXwQMSykse3my2VCdv9OXsLIEzBj
 ovj+d36Uvv681C3EiiVLIT4ok5Ez+np9QxW1n8QJrxR7eEMNIo4lPU8RcOa6yy7G02pc
 A+uqw4jkK+UCeh56m0r7SRW9kbGnmduczMgjk8mSKtNc+djDAyoIVkI1wyNG3JtwHavG
 tPAQ==
X-Received: by 10.224.3.197 with SMTP id 5mr14289860qao.25.1377132052667;
 Wed, 21 Aug 2013 17:40:52 -0700 (PDT)
Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13])
 by mx.google.com with ESMTPSA id m10sm14648160qae.12.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Wed, 21 Aug 2013 17:40:51 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Wed, 21 Aug 2013 20:40:29 -0400
From: Mark Johnston <markj@freebsd.org>
To: Brendan Gregg <brendan.gregg@joyent.com>
Subject: Re: DTrace network providers
Message-ID: <20130822004029.GA49869@raichu>
References: <20130821045926.GA17196@raichu>
 <CA+XzFFhw9beGi8=7heCY07dExvno=oMBYgzUrpht2LYXSbutqg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CA+XzFFhw9beGi8=7heCY07dExvno=oMBYgzUrpht2LYXSbutqg@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 00:40:54 -0000

On Tue, Aug 20, 2013 at 11:44:56PM -0700, Brendan Gregg wrote:
> G'Day,
> 
> On Tue, Aug 20, 2013 at 10:00 PM, Mark Johnston <markj@freebsd.org> wrote:
> 
> > Hello!
> >
> > I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> > the Solaris documentation here:
> >
> > https://wikis.oracle.com/display/DTrace/ip+Provider
> > https://wikis.oracle.com/display/DTrace/tcp+Provider
> > https://wikis.oracle.com/display/DTrace/udp+Provider
> >
> >
> Fantastic!
> 
> 
> > My implementation of these providers makes use of dynamic translators,
> > for which FreeBSD support was added in r254468; this patch won't compile
> > with earlier revisions. The use of dynamic translators means that
> > existing DTrace scripts which use these providers will just work when run
> > on FreeBSD - no modifications needed. In particular, all of the examples
> > in the links above will work properly on FreeBSD with my diff.
> >
> > I've collected a bunch of example scripts for these providers and placed
> > them here:
> >
> > http://people.freebsd.org/~markj/dtrace/network-providers/
> >
> >
> They look familiar. :-) Which is a good sign for this port, as that's
> what's supposed to happen: the same scripts run on any OS.

Yep - by "collect" I mostly meant "copied from the DTrace book." :)
They were very useful for testing. At first all the examples had to be
changed to use the xlate operator on arguments, but now everything just
works.

> 
> To run one you just need to execute "dtrace -s <script>".
> >
> 
> Or chmod 755 the ones with interpreter lines.
> 
> 
> > In general these providers make it trivial to monitor or count packets
> > and bytes on a per-host/port/protocol/interface basis. One can also do
> > neat things like watch TCP connection state transitions in real time
> > (tcpstate.d) and measure connection latency. All of the probes
> > correspond to logical events in their respective protocol
> > implementations; all of the providers have send and receive probes, and
> > the tcp provider has a few more.
> >
> > I didn't have to make any major changes to add support for these
> > providers, but I've made a few small tweaks:
> >
> > 1. Add a little helper function for TCP state changes instead of
> >    sprinkling a bunch of SDT_PROBE calls throughout the code.
> >
> 
> Makes sense.
> 
> 
> > 2. Set the IPv6 payload size in TCP and UDP before calling ip6_output().
> >    This is done for the send probes so that the ipinfo_t argument
> >    can be used to get the payload size. It's not quite correct since it
> >    doesn't include the length of IPv6 option headers (they aren't known
> >    yet), but I think that's ok for the purposes of these providers.
> >
> 
> Excluding the IP header length is correct for ipinfo_t->ip_plength, which
> is just supposed to be the payload length.
> 
> I see the patch makes an assumption about IPv4 header size being 20 bytes,
> instead of picking it from the header (IHL), like:
> 
>         ip_plength = ntohs(I->ipha_length) -
>             ((I->ipha_version_and_hdr_length & 0xf) << 2);

Ah, thanks for pointing this out! I've fixed this in the revision here:

http://people.freebsd.org/~markj/patches/network-providers/network-providers-2.diff

Now that field is obtained with:

ip_plength = p == NULL ? 0 :
    ((struct ip *)p)->ip_v == 4 ?
    ntohs(((struct ip *)p)->ip_len) - (((struct ip *)p)->ip_hl << 2) :
    ntohs(((struct ip6_hdr *)p)->ip6_ctlun.ip6_un1.ip6_un1_plen);

> 
> 3. Set the IPv4 header version in udp_output(). This is already done for
> >    IPv6.
> >
> > I'm hoping that none of this (or the rest of the diff) is controversial,
> > but I wanted to make sure, so any review would be really appreciated.
> >
> > The patch is here:
> >
> >
> > http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
> >
> > It depends on r254468. To use it, just recompile the kernel (assuming
> > that KDTRACE_HOOKS is enabled) and copy
> > $SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
> > /usr/lib/dtrace. Then the example scripts I linked above should just
> > work.
> >
> 
> Great work Mark!
> 
> Here's some (from many) one-liners that this should make possible (more in
> the DTrace book):
> 
> Who is connecting to what:
> dtrace -n 'tcp:::accept-established { @[args[3]->tcps_raddr,
> args[3]->tcps_lport] = count(); }'
> 
> Who isn't connecting to what:
> dtrace -n 'tcp:::accept-refused { @[args[2]->ip_daddr, args[4]->tcp_sport]
> = count(); }'
> 
> What am I connecting to?
> dtrace -n 'tcp:::connect-established { @[args[3]->tcps_raddr ,
> args[3]->tcps_rport] = count(); }'
> 
> IP payload bytes for TCP send, size distribution by destination address:
> dtrace -n 'tcp:::send { @[args[2]->ip_daddr] =
> quantize(args[2]->ip_plength); }'
> 
> Many of these are very low overhead ways to understand network behavior -
> lower than sniffing every packet. Plus you can pull out kernel state that
> is never seen on the wire.
> 
> Brendan
> 
> -- 
> Brendan Gregg, Joyent                      http://dtrace.org/blogs/brendan

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 10:01:49 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 8AACBB9B;
 Thu, 22 Aug 2013 10:01:49 +0000 (UTC)
 (envelope-from koobs.freebsd@gmail.com)
Received: from mail-pa0-x230.google.com (mail-pa0-x230.google.com
 [IPv6:2607:f8b0:400e:c03::230])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 590C62E7A;
 Thu, 22 Aug 2013 10:01:49 +0000 (UTC)
Received: by mail-pa0-f48.google.com with SMTP id kp13so1980655pab.7
 for <multiple recipients>; Thu, 22 Aug 2013 03:01:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=message-id:date:from:reply-to:user-agent:mime-version:to:subject
 :references:in-reply-to:content-type:content-transfer-encoding;
 bh=116XxaQxVIhq6Qfo6A7jtVDFc5yNrFaib8ceBqCMndI=;
 b=WNJzFSLgb5fEdIlAjSvCLn9zSppOF0UdmvL+/Ikg18SlJCKmTzErh2r2fLcHwrwzmp
 9W43iz3FfmTlXAspkM2nYGjOGDDg832awydfK+x1AUyNESrE1jP8+9vH+qcU2w8nMJyv
 1wGsSuHn9MnvJaCqgWpU/dPKSkPC48ceueEmw8ZuYs/yKKBrLDfvx5gwfgU5qfAfXmgm
 Y3gbvGySKYK6vW+mbSyzvfoJ3tKvtDaWmMCacyayA9fzOj6HT42qt3O2TC4J7/NDMoBA
 xJdRTwT++XBhsAoXWse7Jp1cHUMqspfi/a9z72EJyj6e2y5XNSpXT9u7cNAR1/GGBtVi
 IOJw==
X-Received: by 10.67.30.70 with SMTP id kc6mr14192646pad.32.1377165708898;
 Thu, 22 Aug 2013 03:01:48 -0700 (PDT)
Received: from [192.168.1.7] (ppp59-167-128-11.static.internode.on.net.
 [59.167.128.11])
 by mx.google.com with ESMTPSA id tz3sm9702056pbc.20.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Thu, 22 Aug 2013 03:01:48 -0700 (PDT)
Message-ID: <5215E184.2050402@FreeBSD.org>
Date: Thu, 22 Aug 2013 20:01:40 +1000
From: Kubilay Kocak <koobs.freebsd@gmail.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:24.0) Gecko/20100101 Thunderbird/24.0
MIME-Version: 1.0
To: Mark Johnston <markj@freebsd.org>, freebsd-net@freebsd.org, 
 freebsd-dtrace@freebsd.org
Subject: Re: DTrace network providers
References: <20130821045926.GA17196@raichu>
In-Reply-To: <20130821045926.GA17196@raichu>
X-Enigmail-Version: 1.5.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: koobs@FreeBSD.org
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 10:01:49 -0000

On 21/08/2013 3:00 PM, Mark Johnston wrote:
> Hello!
> 
> I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> the Solaris documentation here:

Wow, nice work Mark!

> I've collected a bunch of example scripts for these providers and placed
> them here:
> 
> http://people.freebsd.org/~markj/dtrace/network-providers/
> 
> To run one you just need to execute "dtrace -s <script>".

Do you think it might be worth getting this up and polished in the
FreeBSD Wiki, hopefully with a view to including it in future documentation?

The wiki might also be a good place to give the community a high-level
view and checklist for what we have and what needs to be done, improved
upon or tested.

A great outcome would be helping you to scale ongoing development for
DTrace on FreeBSD, and getting the eyes on it that we need to make it
more useful over time.

Thanks again for all your effort :)

koobs


From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 15:36:14 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 832B83D1;
 Thu, 22 Aug 2013 15:36:14 +0000 (UTC) (envelope-from gnn@freebsd.org)
Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 57B9D25B1;
 Thu, 22 Aug 2013 15:36:14 +0000 (UTC)
Received: from [38.105.238.108] (port=53895 helo=gnnmac.hudson-trading.com)
 by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128)
 (Exim 4.80.1) (envelope-from <gnn@freebsd.org>)
 id 1VCWvk-000619-BV; Thu, 22 Aug 2013 11:36:12 -0400
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\))
Subject: Re: DTrace network providers
From: George Neville-Neil <gnn@freebsd.org>
In-Reply-To: <20130821045926.GA17196@raichu>
Date: Thu, 22 Aug 2013 11:36:13 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <5CA94104-0489-4DD2-8EC2-D42FF7339E39@freebsd.org>
References: <20130821045926.GA17196@raichu>
To: Mark Johnston <markj@freebsd.org>
X-Mailer: Apple Mail (2.1508)
X-AntiAbuse: This header was added to track abuse,
 please include it with any abuse report
X-AntiAbuse: Primary Hostname - vps.hungerhost.com
X-AntiAbuse: Original Domain - freebsd.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - freebsd.org
X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id:
 gnn@neville-neil.com
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 15:36:14 -0000


On Aug 21, 2013, at 1:00 , Mark Johnston <markj@freebsd.org> wrote:

> Hello!
>=20
> I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> the Solaris documentation here:
>=20
> https://wikis.oracle.com/display/DTrace/ip+Provider
> https://wikis.oracle.com/display/DTrace/tcp+Provider
> https://wikis.oracle.com/display/DTrace/udp+Provider
>=20
> My implementation of these providers makes use of dynamic translators,
> for which FreeBSD support was added in r254468; this patch won't =
compile
> with earlier revisions. The use of dynamic translators means that
> existing DTrace scripts which use these providers will just work when =
run
> on FreeBSD - no modifications needed. In particular, all of the =
examples
> in the links above will work properly on FreeBSD with my diff.
>=20
> I've collected a bunch of example scripts for these providers and =
placed
> them here:
>=20
> http://people.freebsd.org/~markj/dtrace/network-providers/
>=20
> To run one you just need to execute "dtrace -s <script>".
>=20
> In general these providers make it trivial to monitor or count packets
> and bytes on a per-host/port/protocol/interface basis. One can also do
> neat things like watch TCP connection state transitions in real time
> (tcpstate.d) and measure connection latency. All of the probes
> correspond to logical events in their respective protocol
> implementations; all of the providers have send and receive probes, =
and
> the tcp provider has a few more.
>=20
> I didn't have to make any major changes to add support for these
> providers, but I've made a few small tweaks:
>=20
> 1. Add a little helper function for TCP state changes instead of
>   sprinkling a bunch of SDT_PROBE calls throughout the code.
> 2. Set the IPv6 payload size in TCP and UDP before calling =
ip6_output().
>   This is done for the send probes so that the ipinfo_t argument
>   can be used to get the payload size. It's not quite correct since it
>   doesn't include the length of IPv6 option headers (they aren't known
>   yet), but I think that's ok for the purposes of these providers.
> 3. Set the IPv4 header version in udp_output(). This is already done =
for
>   IPv6.
>=20
> I'm hoping that none of this (or the rest of the diff) is =
controversial,
> but I wanted to make sure, so any review would be really appreciated.
>=20
> The patch is here:
>=20
> =
http://people.freebsd.org/~markj/patches/network-providers/network-provide=
rs-1.diff
>=20
> It depends on r254468. To use it, just recompile the kernel (assuming
> that KDTRACE_HOOKS is enabled) and copy
> $SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
> /usr/lib/dtrace. Then the example scripts I linked above should just
> work.
>=20

Excellent work.  I'll test these today.

Best,
George



From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 17:25:38 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 68DBF7B6;
 Thu, 22 Aug 2013 17:25:38 +0000 (UTC)
 (envelope-from hiren.panchasara@gmail.com)
Received: from mail-ee0-x22b.google.com (mail-ee0-x22b.google.com
 [IPv6:2a00:1450:4013:c00::22b])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 642A72D8D;
 Thu, 22 Aug 2013 17:25:37 +0000 (UTC)
Received: by mail-ee0-f43.google.com with SMTP id e52so1070089eek.16
 for <multiple recipients>; Thu, 22 Aug 2013 10:25:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=h5b35NcSips+BUQ4X7nq1RGHr6YUtYV0fPxyT9Lmock=;
 b=G6qayvAHkSoAmGFgsP042lVdJtAbIT8B31GFz+h71/Ua4GY7A8FncJGtnpopWrTguE
 CjBvckHZzYt73OK0SOtD+4BltRhgjLkFapcKk2w85OfRO985xAbvuSiyOSYD12XI3HAl
 ecTnqd0WqAdcW6PE2YFjMhAQirigGjO3LwV9Jo8Jy7TVXilps3NMiIYVy8guP0n1hcLr
 ODMSsiWURalGsp5J/QV/3hLvsIOzJtyUacjbdjV0Ladi0SFFAiGmyVPUPlQSnsKTwqfb
 GJ4L0MjmJPICtmqqKNwaUW+RS6cegdGkcjVIFiiGiPIWkYQoj5gIPYXo3Fovx/kat+Lc
 hm3A==
MIME-Version: 1.0
X-Received: by 10.15.77.199 with SMTP id p47mr5332859eey.50.1377192335701;
 Thu, 22 Aug 2013 10:25:35 -0700 (PDT)
Received: by 10.14.105.137 with HTTP; Thu, 22 Aug 2013 10:25:35 -0700 (PDT)
In-Reply-To: <5CA94104-0489-4DD2-8EC2-D42FF7339E39@freebsd.org>
References: <20130821045926.GA17196@raichu>
 <5CA94104-0489-4DD2-8EC2-D42FF7339E39@freebsd.org>
Date: Thu, 22 Aug 2013 10:25:35 -0700
Message-ID: <CALCpEUEZYLs0D3ufW4mKRybqgGKcX98+VGbnWWRHnRVRo0XDmQ@mail.gmail.com>
Subject: Re: DTrace network providers
From: hiren panchasara <hiren.panchasara@gmail.com>
To: George Neville-Neil <gnn@freebsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.14
Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>,
 freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 17:25:38 -0000

On Thu, Aug 22, 2013 at 8:36 AM, George Neville-Neil <gnn@freebsd.org>wrote:

>
> On Aug 21, 2013, at 1:00 , Mark Johnston <markj@freebsd.org> wrote:
>
> > Hello!
> >
> > I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> > the Solaris documentation here:
> >
> > https://wikis.oracle.com/display/DTrace/ip+Provider
> > https://wikis.oracle.com/display/DTrace/tcp+Provider
> > https://wikis.oracle.com/display/DTrace/udp+Provider
> >
> > My implementation of these providers makes use of dynamic translators,
> > for which FreeBSD support was added in r254468; this patch won't compile
> > with earlier revisions. The use of dynamic translators means that
> > existing DTrace scripts which use these providers will just work when run
> > on FreeBSD - no modifications needed. In particular, all of the examples
> > in the links above will work properly on FreeBSD with my diff.
> >
> > I've collected a bunch of example scripts for these providers and placed
> > them here:
> >
> > http://people.freebsd.org/~markj/dtrace/network-providers/
> >
> > To run one you just need to execute "dtrace -s <script>".
> >
> > In general these providers make it trivial to monitor or count packets
> > and bytes on a per-host/port/protocol/interface basis. One can also do
> > neat things like watch TCP connection state transitions in real time
> > (tcpstate.d) and measure connection latency. All of the probes
> > correspond to logical events in their respective protocol
> > implementations; all of the providers have send and receive probes, and
> > the tcp provider has a few more.
> >
> > I didn't have to make any major changes to add support for these
> > providers, but I've made a few small tweaks:
> >
> > 1. Add a little helper function for TCP state changes instead of
> >   sprinkling a bunch of SDT_PROBE calls throughout the code.
> > 2. Set the IPv6 payload size in TCP and UDP before calling ip6_output().
> >   This is done for the send probes so that the ipinfo_t argument
> >   can be used to get the payload size. It's not quite correct since it
> >   doesn't include the length of IPv6 option headers (they aren't known
> >   yet), but I think that's ok for the purposes of these providers.
> > 3. Set the IPv4 header version in udp_output(). This is already done for
> >   IPv6.
> >
> > I'm hoping that none of this (or the rest of the diff) is controversial,
> > but I wanted to make sure, so any review would be really appreciated.
> >
> > The patch is here:
> >
> >
> http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
> >
> > It depends on r254468. To use it, just recompile the kernel (assuming
> > that KDTRACE_HOOKS is enabled) and copy
> > $SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
> > /usr/lib/dtrace. Then the example scripts I linked above should just
> > work.
> >
>
> Excellent work.  I'll test these today.
>

+1. Great work.

Updating my laptop right now to test.

cheers,
Hiren

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 17:41:49 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id C7F16EFB
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 17:41:49 +0000 (UTC)
 (envelope-from venture37@gmail.com)
Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com
 [IPv6:2a00:1450:4010:c03::22a])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 4FE112ED4
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 17:41:49 +0000 (UTC)
Received: by mail-la0-f42.google.com with SMTP id ep20so1695642lab.15
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 10:41:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=subject:references:from:content-type:in-reply-to:message-id:date:to
 :content-transfer-encoding:mime-version;
 bh=TLF9yAny5+n+ZhXkFpVDW0FjKaJ3dJLMR3682Aepj0k=;
 b=r1gOa7DTmm6F+XuentSvww5Sk+jggq9HM/BlAdEZTh5T/5FtRxTByjWPQ8fVgxUavx
 BX4206TrKDCStYbVl7Cy03XRakLeyg3kbJQsci30Ydql8AtEmaPBtfsMTL+H/zUsHo7I
 n9Rqd5Emweh4KMRYIC2Q75r3UD17wX3m7RYQYZ/qfwy8NB+0RR8G9mC4fWy3ujKtf4uS
 pFVWKombDv04QqxwhSaEAL9V0Q2CAcQlP7z2qtlm2tbr9dbt9/B8e1HWLLluhFJ6z7BL
 3UHUKiknioFmbNKXAhcrB6RF2pLhW5P2iBbFuc5eKqXJzx2mP5HGt4f8ZqIp+LJoAJlA
 iiyg==
X-Received: by 10.152.44.164 with SMTP id f4mr2301984lam.38.1377193307103;
 Thu, 22 Aug 2013 10:41:47 -0700 (PDT)
Received: from [46.250.130.34] (fm.zvq.me. [46.250.130.34])
 by mx.google.com with ESMTPSA id n15sm5227521laa.2.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Thu, 22 Aug 2013 10:41:46 -0700 (PDT)
Subject: Re: DTrace network providers
References: <20130821045926.GA17196@raichu>
From: Sevan / Venture37 <venture37@gmail.com>
Content-Type: text/plain;
	charset=us-ascii
X-Mailer: iPhone Mail (9B208)
In-Reply-To: <20130821045926.GA17196@raichu>
Message-Id: <D684F555-F35D-451B-A662-92E47D045194@gmail.com>
Date: Thu, 22 Aug 2013 21:41:41 +0400
To: "freebsd-dtrace@freebsd.org" <freebsd-dtrace@freebsd.org>
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (1.0)
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 17:41:49 -0000


On 21 Aug 2013, at 09:00, Mark Johnston <markj@freebsd.org> wrote:

> 
> http://people.freebsd.org/~markj/dtrace/network-providers/

Great stuff, ip/sent_size_dist.d is missing the hash bang line btw


Sevan

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 21:00:40 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 12968277
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 21:00:40 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-qe0-x233.google.com (mail-qe0-x233.google.com
 [IPv6:2607:f8b0:400d:c02::233])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id C85AD221F
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 21:00:39 +0000 (UTC)
Received: by mail-qe0-f51.google.com with SMTP id cy11so1451685qeb.10
 for <freebsd-dtrace@freebsd.org>; Thu, 22 Aug 2013 14:00:39 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-type:content-disposition:in-reply-to:user-agent;
 bh=v4hfba5euk1ltbykKLsDar9MkvUXRaM7pRn4buFPFN0=;
 b=cVJop6ZqleVX4ct8TM84xnlX7Qb5TbLBvvDmB3BG06bH3PC5XgxUNLGfpgMlApL/1g
 SV1Gqn561TNm6L7p1At8nUAufW/RaYtZSR5YUIkauq/mZRynPX6rSEzHogTp9/kV1z3C
 UizQGPFMgJpBX0NOz3Qdd7i9go2E3yB4PBDDgKFbLhb2LdmTkxn8cyiZ+cQzBzUfX0+9
 kxC/EHFK+hEaI9SSXtAgGw6360lDiBP3FQi8EYrrYeUiSr97RUqZVzGhX/7hqXA0vYr8
 bWzrKNyxPdcYrd/lahZg1w+7sHLwQLEERHIgJHcFIOLMIc7b1F6u2WOtQ82AFvUkoR75
 wrjg==
X-Received: by 10.224.164.200 with SMTP id f8mr20289995qay.8.1377205238998;
 Thu, 22 Aug 2013 14:00:38 -0700 (PDT)
Received: from charmander.sandvine.com ([64.7.137.182])
 by mx.google.com with ESMTPSA id a7sm18695146qeg.2.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Thu, 22 Aug 2013 14:00:38 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Thu, 22 Aug 2013 17:01:23 -0400
From: Mark Johnston <markj@freebsd.org>
To: Sevan / Venture37 <venture37@gmail.com>
Subject: Re: DTrace network providers
Message-ID: <20130822210123.GB59491@charmander.sandvine.com>
References: <20130821045926.GA17196@raichu>
 <D684F555-F35D-451B-A662-92E47D045194@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <D684F555-F35D-451B-A662-92E47D045194@gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: "freebsd-dtrace@freebsd.org" <freebsd-dtrace@freebsd.org>
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 21:00:40 -0000

On Thu, Aug 22, 2013 at 09:41:41PM +0400, Sevan / Venture37 wrote:
> 
> On 21 Aug 2013, at 09:00, Mark Johnston <markj@freebsd.org> wrote:
> 
> > 
> > http://people.freebsd.org/~markj/dtrace/network-providers/
> 
> Great stuff, ip/sent_size_dist.d is missing the hash bang line btw

Fixed, thanks!

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 21:27:02 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id ECFF5AB2;
 Thu, 22 Aug 2013 21:27:02 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com
 [IPv6:2607:f8b0:400d:c01::233])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 9D592242E;
 Thu, 22 Aug 2013 21:27:02 +0000 (UTC)
Received: by mail-qc0-f179.google.com with SMTP id n10so1304634qcx.24
 for <multiple recipients>; Thu, 22 Aug 2013 14:27:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-type:content-disposition:in-reply-to:user-agent;
 bh=Hmc0xD+EPyYO24B4jyscOWqxeUpfEhKI+rfMuN1iJHM=;
 b=tVIt84k0dl6mNe8bnceDMR9G11PWdZgV9l4j+fvC4SBX+U+J/wKeCnn5qph2P53n9y
 SVt5pPmNseyKtxpsWqwShvdcll7grgXgL03k42MiVO4jsuwuYE0px9mHLeQO9txKyd2/
 VFiq0+1gy8UfH5tzaPde99CoXjI4oIc7tKvCFisgo+viNMudAjNXltLWbRuG3+CvmiDa
 xdYW5upTBBre8TnWmiJHRFO/Dsf46PZkIZJhAWB4o06eVqvrSCEbj/2ezsHH7BaoQMBI
 9wIiQkrM3GWF4vTJR0fPgW1CPK7rmny55AQyjt33n1pQ5l3VDyzd/psH00lrSNJ8qm1k
 sLsg==
X-Received: by 10.49.71.147 with SMTP id v19mr17938226qeu.54.1377206821705;
 Thu, 22 Aug 2013 14:27:01 -0700 (PDT)
Received: from charmander.sandvine.com ([64.7.137.182])
 by mx.google.com with ESMTPSA id t4sm17564727qas.1.1969.12.31.16.00.00
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Thu, 22 Aug 2013 14:27:01 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Thu, 22 Aug 2013 17:27:46 -0400
From: Mark Johnston <markj@freebsd.org>
To: koobs@FreeBSD.org
Subject: Re: DTrace network providers
Message-ID: <20130822212746.GC59491@charmander.sandvine.com>
References: <20130821045926.GA17196@raichu>
 <5215E184.2050402@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5215E184.2050402@FreeBSD.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 21:27:03 -0000

On Thu, Aug 22, 2013 at 08:01:40PM +1000, Kubilay Kocak wrote:
> On 21/08/2013 3:00 PM, Mark Johnston wrote:
> > Hello!
> > 
> > I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
> > the Solaris documentation here:
> 
> Wow, nice work Mark!
> 
> > I've collected a bunch of example scripts for these providers and placed
> > them here:
> > 
> > http://people.freebsd.org/~markj/dtrace/network-providers/
> > 
> > To run one you just need to execute "dtrace -s <script>".
> 
> Do you think it might be worth getting this up and polished in the
> FreeBSD Wiki, hopefully with a view to including it in future documentation?

I'm still thinking about how to document this stuff. I'd like to write a
man page which defines and describes our SDT probes and provides
examples. A wiki would be a good place for examples, but I'd rather not
document stuff there as it'll eventually fall out of sync with what the
code does.

> 
> The wiki might also be a good place to give the community a high-level
> view and checklist for what we have and what needs to be done, improved
> upon or tested.

Yep, it'd be good to have somewhere to document outstanding DTrace work,
missing features, bugs and so on. I'll document all of the problems and
shortcomings that I'm aware of, and add a wiki page for them (or append
to an existing page).

> 
> A great outcome would be helping you to scale ongoing development for
> DTrace on FreeBSD, and getting the eyes on it that we need to make it
> more useful over time.
> 
> Thanks again for all your effort :)
> 
> koobs
> 

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 22:05:04 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 0769AF1A;
 Thu, 22 Aug 2013 22:05:04 +0000 (UTC) (envelope-from yuri@rawbw.com)
Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45])
 by mx1.freebsd.org (Postfix) with ESMTP id E42D527C2;
 Thu, 22 Aug 2013 22:05:03 +0000 (UTC)
Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1])
 (authenticated bits=0)
 by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id r7MM52VC057380;
 Thu, 22 Aug 2013 15:05:03 -0700 (PDT) (envelope-from yuri@rawbw.com)
Message-ID: <52168B0E.1050308@rawbw.com>
Date: Thu, 22 Aug 2013 15:05:02 -0700
From: Yuri <yuri@rawbw.com>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:17.0) Gecko/20130822 Thunderbird/17.0.8
MIME-Version: 1.0
To: Mark Johnston <markj@freebsd.org>
Subject: Re: DTrace network providers
References: <20130821045926.GA17196@raichu>
In-Reply-To: <20130821045926.GA17196@raichu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 22:05:04 -0000

On 08/20/2013 22:00, Mark Johnston wrote:
> The patch is here:
>
> http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
>
> It depends on r254468. To use it, just recompile the kernel (assuming

Your patch fails to apply, see below.

I use clean r254468 as you suggested.

Yuri

--------------------------
|diff --git a/sys/netinet/ip_output.c b/sys/netinet/ip_output.c
|index 0a87e7a..15196e0 100644
|--- a/sys/netinet/ip_output.c
|+++ b/sys/netinet/ip_output.c
--------------------------
Patching file a/sys/netinet/ip_output.c using Plan A...
Hunk #1 succeeded at 34.
Hunk #2 succeeded at 48.
Hunk #3 succeeded at 66.
Hunk #4 failed at 625.
Hunk #5 succeeded at 660 with fuzz 2.
1 out of 5 hunks failed--saving rejects to a/sys/netinet/ip_output.c.rej

--------------------------
|diff --git a/sys/netinet6/nd6.c b/sys/netinet6/nd6.c
|index 7755da1..9eaa0aa 100644
|--- a/sys/netinet6/nd6.c
|+++ b/sys/netinet6/nd6.c
--------------------------
Patching file a/sys/netinet6/nd6.c using Plan A...
Hunk #1 succeeded at 34.
Hunk #2 succeeded at 51.
Hunk #3 succeeded at 64.
Hunk #4 failed at 2086.
1 out of 4 hunks failed--saving rejects to a/sys/netinet6/nd6.c.rej
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:


From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 22:30:51 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id BF6E1DA7;
 Thu, 22 Aug 2013 22:30:51 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com
 [IPv6:2a00:1450:400c:c00::22d])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 05C88297E;
 Thu, 22 Aug 2013 22:30:50 +0000 (UTC)
Received: by mail-wg0-f45.google.com with SMTP id n12so2166293wgh.0
 for <multiple recipients>; Thu, 22 Aug 2013 15:30:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:in-reply-to:references:date:message-id:subject
 :from:to:cc:content-type;
 bh=ksNU0nMbfSv3FXiWYW8m9I5OV3c2lnXHlGLbVVT0Bok=;
 b=w97aiukrJQDnpfcwqefco4hq/j+S/ouDAE3ZMYtCqFFPZ7Xtb2i8eAkLakvHuOxqzp
 6zpUNS+7ROLSH5tSNpEvOxXo0iOH/5SFV9DjElMFw+9NzARzxQ19WcbCVu/Emog1vy7Q
 Kma/TQ5+G87oG3U+beIje+DbizmvFz6SjeFJmZIhQrzedNGypHYfFkGfxIFxKO4pag/w
 95BZ5Z93gJ17drRZ5hGBeRxWd659tC/0+jbqnX++mpKqg5ai54xc+VCt+Pnb9bG6pxUs
 Nqwa/pEldaTYeDC7xu2M7d/LCn3kPm2Xr7JhHk9zMLEgJcd5fa2CpMJ/wAZ8IE2cDvLG
 GO1g==
MIME-Version: 1.0
X-Received: by 10.180.102.234 with SMTP id fr10mr1146350wib.6.1377210649369;
 Thu, 22 Aug 2013 15:30:49 -0700 (PDT)
Sender: markjdb@gmail.com
Received: by 10.194.36.200 with HTTP; Thu, 22 Aug 2013 15:30:49 -0700 (PDT)
In-Reply-To: <52168B0E.1050308@rawbw.com>
References: <20130821045926.GA17196@raichu>
	<52168B0E.1050308@rawbw.com>
Date: Thu, 22 Aug 2013 18:30:49 -0400
X-Google-Sender-Auth: -L7QkDw5DvR_yhvbLz9bnfh6CRk
Message-ID: <CAMw1wOwsrdOOMuLJPgzaRfiCimrdxti6W+1JkWV2uqbW5bb+Pw@mail.gmail.com>
Subject: Re: DTrace network providers
From: Mark Johnston <markj@freebsd.org>
To: Yuri <yuri@rawbw.com>
Content-Type: text/plain; charset=ISO-8859-1
X-Content-Filtered-By: Mailman/MimeDel 2.1.14
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 22:30:51 -0000

On Thu, Aug 22, 2013 at 6:05 PM, Yuri <yuri@rawbw.com> wrote:

> On 08/20/2013 22:00, Mark Johnston wrote:
>
>> The patch is here:
>>
>> http://people.freebsd.org/~**markj/patches/network-**
>> providers/network-providers-1.**diff<http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff>
>>
>> It depends on r254468. To use it, just recompile the kernel (assuming
>>
>
> Your patch fails to apply, see below.
>
> I use clean r254468 as you suggested.
>

My apologies! It looks like r254523 introduced a conflict. r254468 is a
minimum dependency.

I'd suggest trying again with r254523 or later.

-Mark


> Yuri
>
> --------------------------
> |diff --git a/sys/netinet/ip_output.c b/sys/netinet/ip_output.c
> |index 0a87e7a..15196e0 100644
> |--- a/sys/netinet/ip_output.c
> |+++ b/sys/netinet/ip_output.c
> --------------------------
> Patching file a/sys/netinet/ip_output.c using Plan A...
> Hunk #1 succeeded at 34.
> Hunk #2 succeeded at 48.
> Hunk #3 succeeded at 66.
> Hunk #4 failed at 625.
> Hunk #5 succeeded at 660 with fuzz 2.
> 1 out of 5 hunks failed--saving rejects to a/sys/netinet/ip_output.c.rej
>
> --------------------------
> |diff --git a/sys/netinet6/nd6.c b/sys/netinet6/nd6.c
> |index 7755da1..9eaa0aa 100644
> |--- a/sys/netinet6/nd6.c
> |+++ b/sys/netinet6/nd6.c
> --------------------------
> Patching file a/sys/netinet6/nd6.c using Plan A...
> Hunk #1 succeeded at 34.
> Hunk #2 succeeded at 51.
> Hunk #3 succeeded at 64.
> Hunk #4 failed at 2086.
> 1 out of 4 hunks failed--saving rejects to a/sys/netinet6/nd6.c.rej
> Hmm...  The next patch looks like a unified diff to me...
> The text leading up to this was:
>
>

From owner-freebsd-dtrace@FreeBSD.ORG  Thu Aug 22 22:52:48 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 6D610A46;
 Thu, 22 Aug 2013 22:52:48 +0000 (UTC)
 (envelope-from hiren.panchasara@gmail.com)
Received: from mail-ea0-x22a.google.com (mail-ea0-x22a.google.com
 [IPv6:2a00:1450:4013:c01::22a])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id A713B2BFB;
 Thu, 22 Aug 2013 22:52:47 +0000 (UTC)
Received: by mail-ea0-f170.google.com with SMTP id h14so1157721eak.15
 for <multiple recipients>; Thu, 22 Aug 2013 15:52:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=mf/R/ThotMoHd/yKNVliW0c4m/crXf/6h/tvmfZEP40=;
 b=gIExZxpPljiQ/B+O1KJeZVnJqA2x3gV7AL3zhsxZ7qORWNTmEASBM55mBAu/Fn7lQ5
 i0pVxxXlkbFwq/Ef2/xmHB+UrRn0yZMbdphrWwy67avrtZNtvqCUHC7XkAkrQ+dAppKT
 fVS0SRSfMIi2MNIRQZ/q2awjhwepmEOKw173WN1/MrqrBhmxnt9XSNKSUiqux4pbk6i1
 nQcQ5w4/kfblNYSClYwVxSz+YvqjNgDqWkzCEH+eO+ppHxpnBdLW/q2jExDktdPvdw0N
 45IaD8eFKj4oQsQzh17blzqdfAOvskCzCqVY0aHL8abGdUV12tH4bLg9giBtXiI4K5Rc
 7qOA==
MIME-Version: 1.0
X-Received: by 10.14.220.200 with SMTP id o48mr449286eep.62.1377211965898;
 Thu, 22 Aug 2013 15:52:45 -0700 (PDT)
Received: by 10.14.105.137 with HTTP; Thu, 22 Aug 2013 15:52:45 -0700 (PDT)
In-Reply-To: <52168B0E.1050308@rawbw.com>
References: <20130821045926.GA17196@raichu>
	<52168B0E.1050308@rawbw.com>
Date: Thu, 22 Aug 2013 15:52:45 -0700
Message-ID: <CALCpEUENZ10pz6kJTPv3v5j+LQeTPQ-kQ-3BoBPVUso8tBX9nQ@mail.gmail.com>
Subject: Re: DTrace network providers
From: hiren panchasara <hiren.panchasara@gmail.com>
To: Yuri <yuri@rawbw.com>
Content-Type: text/plain; charset=UTF-8
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2013 22:52:48 -0000

On Thu, Aug 22, 2013 at 3:05 PM, Yuri <yuri@rawbw.com> wrote:
> On 08/20/2013 22:00, Mark Johnston wrote:
>>
>> The patch is here:
>>
>>
>> http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
>>
>> It depends on r254468. To use it, just recompile the kernel (assuming
>
>
> Your patch fails to apply, see below.
fwiw, I update the laptop to an hour back CURRENT r254665  and patch
applied cleanly.

cheers,
hiren
>
> I use clean r254468 as you suggested.
>
> Yuri
>
> --------------------------
> |diff --git a/sys/netinet/ip_output.c b/sys/netinet/ip_output.c
> |index 0a87e7a..15196e0 100644
> |--- a/sys/netinet/ip_output.c
> |+++ b/sys/netinet/ip_output.c
> --------------------------
> Patching file a/sys/netinet/ip_output.c using Plan A...
> Hunk #1 succeeded at 34.
> Hunk #2 succeeded at 48.
> Hunk #3 succeeded at 66.
> Hunk #4 failed at 625.
> Hunk #5 succeeded at 660 with fuzz 2.
> 1 out of 5 hunks failed--saving rejects to a/sys/netinet/ip_output.c.rej
>
> --------------------------
> |diff --git a/sys/netinet6/nd6.c b/sys/netinet6/nd6.c
> |index 7755da1..9eaa0aa 100644
> |--- a/sys/netinet6/nd6.c
> |+++ b/sys/netinet6/nd6.c
> --------------------------
> Patching file a/sys/netinet6/nd6.c using Plan A...
> Hunk #1 succeeded at 34.
> Hunk #2 succeeded at 51.
> Hunk #3 succeeded at 64.
> Hunk #4 failed at 2086.
> 1 out of 4 hunks failed--saving rejects to a/sys/netinet6/nd6.c.rej
> Hmm...  The next patch looks like a unified diff to me...
> The text leading up to this was:
>
>
> _______________________________________________
> freebsd-dtrace@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace
> To unsubscribe, send any mail to "freebsd-dtrace-unsubscribe@freebsd.org"

From owner-freebsd-dtrace@FreeBSD.ORG  Fri Aug 23 06:07:15 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 6DC03D97;
 Fri, 23 Aug 2013 06:07:15 +0000 (UTC) (envelope-from yuri@rawbw.com)
Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45])
 by mx1.freebsd.org (Postfix) with ESMTP id 563662EE3;
 Fri, 23 Aug 2013 06:07:15 +0000 (UTC)
Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1])
 (authenticated bits=0)
 by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id r7N677bc049132;
 Thu, 22 Aug 2013 23:07:08 -0700 (PDT) (envelope-from yuri@rawbw.com)
Message-ID: <5216FC0B.1060304@rawbw.com>
Date: Thu, 22 Aug 2013 23:07:07 -0700
From: Yuri <yuri@rawbw.com>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:17.0) Gecko/20130822 Thunderbird/17.0.8
MIME-Version: 1.0
To: Mark Johnston <markj@freebsd.org>
Subject: Re: DTrace network providers
References: <20130821045926.GA17196@raichu> <52168B0E.1050308@rawbw.com>
 <CAMw1wOwsrdOOMuLJPgzaRfiCimrdxti6W+1JkWV2uqbW5bb+Pw@mail.gmail.com>
In-Reply-To: <CAMw1wOwsrdOOMuLJPgzaRfiCimrdxti6W+1JkWV2uqbW5bb+Pw@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: freebsd-net@freebsd.org, freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 23 Aug 2013 06:07:15 -0000

On 08/22/2013 15:30, Mark Johnston wrote:
> My apologies! It looks like r254523 introduced a conflict. r254468 is a
> minimum dependency.
>
> I'd suggest trying again with r254523 or later.

I don't know what the problem is, but I tried on r254523 and on the 
later r254677, and am getting build error during buildkernel, see below.
And I did successful buildworld and installworld before this.

Yuri


--- buildkernel errors ---
/usr/src/sys/netinet/in_kdtrace.c:165:1: error: redefinition of 
'sdt_provider_ip'
SDT_PROVIDER_DEFINE(ip);
^
/usr/src/sys/sys/sdt.h:136:22: note: expanded from macro 
'SDT_PROVIDER_DEFINE'
         struct sdt_provider sdt_provider_##prov[1] = 
{                          \
                             ^
<scratch space>:127:1: note: expanded from here
sdt_provider_ip
^
/usr/src/sys/netinet/in_kdtrace.c:38:1: note: previous definition is here
SDT_PROVIDER_DEFINE(ip);
^
/usr/src/sys/sys/sdt.h:136:22: note: expanded from macro 
'SDT_PROVIDER_DEFINE'
         struct sdt_provider sdt_provider_##prov[1] = 
{                          \
                             ^
<scratch space>:203:1: note: expanded from here
sdt_provider_ip
^
/usr/src/sys/netinet/in_kdtrace.c:166:1: error: redefinition of 
'sdt_provider_tcp'
SDT_PROVIDER_DEFINE(tcp);
^
/usr/src/sys/sys/sdt.h:136:22: note: expanded from macro 
'SDT_PROVIDER_DEFINE'
         struct sdt_provider sdt_provider_##prov[1] = 
{                          \
                             ^


From owner-freebsd-dtrace@FreeBSD.ORG  Fri Aug 23 18:50:32 2013
Return-Path: <owner-freebsd-dtrace@FreeBSD.ORG>
Delivered-To: freebsd-dtrace@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id C85C4F05;
 Fri, 23 Aug 2013 18:50:32 +0000 (UTC)
 (envelope-from hiren.panchasara@gmail.com)
Received: from mail-ea0-x22e.google.com (mail-ea0-x22e.google.com
 [IPv6:2a00:1450:4013:c01::22e])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id C382E2B76;
 Fri, 23 Aug 2013 18:50:31 +0000 (UTC)
Received: by mail-ea0-f174.google.com with SMTP id z15so448469ead.5
 for <multiple recipients>; Fri, 23 Aug 2013 11:50:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=OrgPzddQtJihucr5G0Ej/FFUVGFgZhq9zV66P4utY0I=;
 b=PRzs/OrPdudPcloTsCQ3ZKAzGV9THhFq2s+6pekAEXXP6o63w43xd/BBMwMJQWBlyY
 RigP2/BrWw/l/39CMm5r9AEuGVxuLDlUgpIZAMiA9b5sSqvI/b4d+HTI1w0mcXxkkI18
 W2CRkHeR1I6tsjw3/2P8X2QkiOYf2Lf1lUR7YuCec1147ayPU3KWz73sPHgRjDxAAiKb
 l19O1HJ6GmfJeKxbL8YE+eh9s4VfU+pxwe1Y7l953oNfpTwtcT3c2x9SR4fWFiKxeMCf
 8YO0SGEfF3m22PV2UPqbI3l3hZEEY03arrrMqdiFkOyqJYYQ0dJiRiBJ+ijcn22OewFk
 zhRQ==
MIME-Version: 1.0
X-Received: by 10.14.173.4 with SMTP id u4mr714895eel.81.1377283830014; Fri,
 23 Aug 2013 11:50:30 -0700 (PDT)
Received: by 10.14.105.137 with HTTP; Fri, 23 Aug 2013 11:50:29 -0700 (PDT)
In-Reply-To: <CALCpEUEZYLs0D3ufW4mKRybqgGKcX98+VGbnWWRHnRVRo0XDmQ@mail.gmail.com>
References: <20130821045926.GA17196@raichu>
 <5CA94104-0489-4DD2-8EC2-D42FF7339E39@freebsd.org>
 <CALCpEUEZYLs0D3ufW4mKRybqgGKcX98+VGbnWWRHnRVRo0XDmQ@mail.gmail.com>
Date: Fri, 23 Aug 2013 11:50:29 -0700
Message-ID: <CALCpEUFiAVe6Yj7Tqpzm-09G2Zn0POukF6_P7NHZbL8uGcvOQw@mail.gmail.com>
Subject: Re: DTrace network providers
From: hiren panchasara <hiren.panchasara@gmail.com>
To: George Neville-Neil <gnn@freebsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.14
Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>,
 freebsd-dtrace@freebsd.org
X-BeenThere: freebsd-dtrace@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "A discussion list for developers working on DTrace in FreeBSD."
 <freebsd-dtrace.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-dtrace>, 
 <mailto:freebsd-dtrace-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-dtrace>
List-Post: <mailto:freebsd-dtrace@freebsd.org>
List-Help: <mailto:freebsd-dtrace-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace>,
 <mailto:freebsd-dtrace-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 23 Aug 2013 18:50:33 -0000

On Thu, Aug 22, 2013 at 10:25 AM, hiren panchasara <
hiren.panchasara@gmail.com> wrote:

>
>
>
> On Thu, Aug 22, 2013 at 8:36 AM, George Neville-Neil <gnn@freebsd.org>wrote:
>
>>
>> On Aug 21, 2013, at 1:00 , Mark Johnston <markj@freebsd.org> wrote:
>>
>> > Hello!
>> >
>> > I've ported the ip, tcp and udp DTrace providers to FreeBSD, following
>> > the Solaris documentation here:
>> >
>> > https://wikis.oracle.com/display/DTrace/ip+Provider
>> > https://wikis.oracle.com/display/DTrace/tcp+Provider
>> > https://wikis.oracle.com/display/DTrace/udp+Provider
>> >
>> > My implementation of these providers makes use of dynamic translators,
>> > for which FreeBSD support was added in r254468; this patch won't compile
>> > with earlier revisions. The use of dynamic translators means that
>> > existing DTrace scripts which use these providers will just work when
>> run
>> > on FreeBSD - no modifications needed. In particular, all of the examples
>> > in the links above will work properly on FreeBSD with my diff.
>> >
>> > I've collected a bunch of example scripts for these providers and placed
>> > them here:
>> >
>> > http://people.freebsd.org/~markj/dtrace/network-providers/
>> >
>> > To run one you just need to execute "dtrace -s <script>".
>> >
>> > In general these providers make it trivial to monitor or count packets
>> > and bytes on a per-host/port/protocol/interface basis. One can also do
>> > neat things like watch TCP connection state transitions in real time
>> > (tcpstate.d) and measure connection latency. All of the probes
>> > correspond to logical events in their respective protocol
>> > implementations; all of the providers have send and receive probes, and
>> > the tcp provider has a few more.
>> >
>> > I didn't have to make any major changes to add support for these
>> > providers, but I've made a few small tweaks:
>> >
>> > 1. Add a little helper function for TCP state changes instead of
>> >   sprinkling a bunch of SDT_PROBE calls throughout the code.
>> > 2. Set the IPv6 payload size in TCP and UDP before calling ip6_output().
>> >   This is done for the send probes so that the ipinfo_t argument
>> >   can be used to get the payload size. It's not quite correct since it
>> >   doesn't include the length of IPv6 option headers (they aren't known
>> >   yet), but I think that's ok for the purposes of these providers.
>> > 3. Set the IPv4 header version in udp_output(). This is already done for
>> >   IPv6.
>> >
>> > I'm hoping that none of this (or the rest of the diff) is controversial,
>> > but I wanted to make sure, so any review would be really appreciated.
>> >
>> > The patch is here:
>> >
>> >
>> http://people.freebsd.org/~markj/patches/network-providers/network-providers-1.diff
>> >
>> > It depends on r254468. To use it, just recompile the kernel (assuming
>> > that KDTRACE_HOOKS is enabled) and copy
>> > $SRC/cddl/lib/libdtrace/{ip,tcp,udp}.d (added by the patch) to
>> > /usr/lib/dtrace. Then the example scripts I linked above should just
>> > work.
>> >
>>
>> Excellent work.  I'll test these today.
>>
>
> +1. Great work.
>
> Updating my laptop right now to test.
>

Tried a bunch of basic things and it works great!

Thanks again.
Hiren

>
> cheers,
> Hiren
>