Date: Tue, 30 Dec 2014 01:28:56 +0100 From: Nikolay Denev <ndenev@gmail.com> To: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: setfib and RSTs Message-ID: <CA%2BP_MZEgR8f8yANSwTz--t11LfmkmjfWsOdNPfGaKKuaPUDhfg@mail.gmail.com> In-Reply-To: <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net> References: <CA%2BP_MZHi1hMGq8qbOvpYSMLk4Usr4CAXjR1PdKRt7dE_vMJ7UQ@mail.gmail.com> <54A0FDD9.4090009@freebsd.org> <CA%2BP_MZGsX3_UB2wigrD-WWB0aN=bSf9Xxw147y2vWLAJ6_syng@mail.gmail.com> <CAJ-Vmo=B3c9p6nMpHA6z75oKBErTrX0Ru8MDhHw1MgN=Ts_%2B_w@mail.gmail.com> <CA%2BP_MZGa8s9PzhoA=kY8YRcG5kRsVb=oP8pjtOXV%2BDH_meG2nQ@mail.gmail.com> <81C17F4A-B0AA-48C9-ABFB-6B14B7223643@lists.zabbadoz.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 30, 2014 at 12:51 AM, Bjoern A. Zeeb <
bzeeb-lists@lists.zabbadoz.net> wrote:
>
> > On 29 Dec 2014, at 21:03 , Nikolay Denev <ndenev@gmail.com> wrote:
> >
> > No, no PR yet, but I will file one. I wanted to collect some more data
> > first.
> >
> > So, I've did some dtrace digging :
> >
> > [20:54][root@nas:~]#cat reset.d
> > #!/usr/sbin/dtrace -s
> >
> > fbt:kernel:tcp_dropwithreset:entry
> > {
> > printf("reason %d fib %d src_port %d dst_port %d", args[4], args[2] =
?
> > args[2]->t_inpcb->inp_inc.inc_fibnum : -1, ntohs(args[1]->th_sport),
> > ntohs(args[1]->th_dport));
> > /* stack(); */
> > }
> > =E2=80=A6
>
> > The port numbers here match RST packets that I'm seeing with tcpdump in
> > another window.
> > reason 3 is BANDLIM_RST_CLOSEDPORT (from icmp_var.h)
> > Looking at tcp_input.c I see that there are cases where the INPCB does
> not
> > exists, and from what I see this is how the FIB gets determined.
> > Also here I see that tcp_dropwithreset() is called with tcpcb set to
> NULL,
> > so probably this is why the FIB is not found.
> >
> > Why this is happening, I have no idea yet.
>
> Could you also check for the mbuf *m and the fibnum from the pkthdr there=
?
>
> It might be even more interesting to see this for tcp_respond and the
> following ip_output as well, in case you want to keep state in the d
> script; otherwise just tcp_dropwithreset and/or tcp_respond should be fi=
ne.
>
> Usually I would expect for the tcp_dropwithreset case that inp will be
> NULL in tcp_respond, the mbuf *m and th will be valid and thus the FIB
> number from the incoming mbuf would be re-used as the mbuf will be re-use=
d,
> but for that the mbuf needs to be properly tagged on receive.
>
> /bz
>
> =E2=80=94
> Bjoern A. Zeeb Charles Haddon Spurgeon:
> "Friendship is one of the sweetest joys of life. Many might have failed
> beneath the bitterness of their trial had they not found a friend."
>
>
If I got this right i see FIB 0 here:
fbt:kernel:tcp_dropwithreset:entry
{
this->mbuf =3D (struct mbuf *)args[0];
printf("reason %d fib %d src_port %d dst_port %d", args[4],
this->mbuf->M_dat.MH.MH_pkthdr.fibnum, ntohs(args[1]->th_sport),
ntohs(args[1]->th_dport));
}
I've did some tracing of the the transmission-daemon, and I saw that after
it calls connect() on the sockets (which are non-blocking), the connect
returns with EINPROGRESS (as expected),
but in most cases after about 20-60us close() is called. (not sure why it's
doing this, but this is what I saw)
So, by that time the SYN-ACK from the remote end is received the socket is
already gone and RST is sent using default FIB, which does seem to be the
correct behaviour.
I've now set the fib for the tun interface with "ifconfig tun2 fib 1", and
I see the RSTs sent back over the tunnel.
--Nikolay
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BP_MZEgR8f8yANSwTz--t11LfmkmjfWsOdNPfGaKKuaPUDhfg>