From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 18:47:10 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99FD5513; Mon, 29 Dec 2014 18:47:10 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22F271918; Mon, 29 Dec 2014 18:47:10 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x13so19571465wgg.19; Mon, 29 Dec 2014 10:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Lpc32/eYs60nhbCUH77/H7mlNpPSyA7es1cmPrKShnc=; b=EAnWfnvXlCBW1GW9X9UgcQFb08ZcneonYaHIpLh+UHY3S8zu5vij019rUkdwPBSrJr 7c0NXXMgOOMhZ3gTobrZJ9AIUolyOg9sOYClE4Bp6CSJunxuFIRaNaZzP+f3JM4c9Lph o11oOlZVx3zqsEG3YCkWyEpimlYSAIUOCK/i2bcy8fIQIvsGlVZsIbCpzJlg7UOs20QI jOOHYpTSCbXuPVCrngJHsawVi2Xza92TWAOjNjGBBp6Wg3Wmo8DGKmDQ5CrJHoP29P2B +kJAxH81KDq2C6N/E7bafjlMDHrhXslmukSwoXESQTwQLAQW5E/si82k1oh7ts8jHXEn LdHA== MIME-Version: 1.0 X-Received: by 10.180.87.36 with SMTP id u4mr97077712wiz.20.1419878828626; Mon, 29 Dec 2014 10:47:08 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.106.195 with HTTP; Mon, 29 Dec 2014 10:47:08 -0800 (PST) In-Reply-To: References: <54A0FDD9.4090009@freebsd.org> Date: Mon, 29 Dec 2014 10:47:08 -0800 X-Google-Sender-Auth: JJC7FnuXucDhlUgrf6ejAF_ocJI Message-ID: Subject: Re: setfib and RSTs From: Adrian Chadd To: Nikolay Denev Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2014 18:47:10 -0000 Have you filed a PR about this? This seems like it's pretty important to fix. (For other non-torrent reasons too; like people wishing to use this for things like tor browsing.) -adrian On 29 December 2014 at 05:11, Nikolay Denev wrote: > On Mon, Dec 29, 2014 at 8:08 AM, Julian Elischer wrote: > >> On 12/26/14 10:41 PM, Nikolay Denev wrote: >> >>> Hi, >>> >>> I have a process (bittorrent client) running in a non-default fib and >>> using >>> a VPN for default gateway: >>> >>> from /etc/rc.local : >>> >>> /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 >>> /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config >>> /usr/local/etc/openvpn/provider.ovpn >>> /usr/sbin/setfib 1 /usr/sbin/service transmission onestart >>> >>> Then openvpn installs default gateway in fib 1 to point to the tun(4) >>> interface. >>> >>> Stil, I'm seeing RST packets from the bittorrent client process to be sent >>> not via the tunnel, but to the default gateway of the lan which seems >>> wrong. As if when the kernel generates the RST it does not take into >>> account the FIB of the socket? >>> >> it's possible that you are correct. >> I checked that RST and other generated packets used the FIB for the >> session if it existed when they are generated, >> but I don't know what they do when a single unexpected packet enters..You >> may need toset the fib in received packets using either the ipfw setfib >> command or the ifconfig fib command. The RST should then use the same FIB >> to respond. >> >> let me know.. >> >> >> Any ideas? >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> >> > > The weird thing was that the RST was a response to a SYN-ACK. > What I saw was the transmission bittorent daemon to try to connect to some > peer over tun2 (the openvpn connection), > so it sends the SYN, the peer responds with SYN-ACK, then I see RST back to > the peer, but not over the tun2, but over lagg0 (as originally mentioned). > > Here 10.8.0.31 is the tun2 address, and XX.XX.XX.XX is one of the peers > that I was able to capture this happening. (both SYN-ACK and RST's are > retransmitted, as the RSTs get blocked by the pf running on the default gw > as it does not have state for them, as the VPN is a running on a host > inside the network) > > # Captured on tun2: > > [14:03][ndenev@macbook:~]%tcpdump -vnr 1.pcap host XX.XX.XX.XX > reading from file 1.pcap, link-type NULL (BSD loopback) > 20:43:39.623920 IP (tos 0x0, ttl 64, id 12450, offset 0, flags [DF], proto > TCP (6), length 60) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [S], cksum 0xace8 (correct), > seq 2985341180, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > 515982338 ecr 0], length 0 > 20:43:39.912558 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5a3d > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207923992 ecr 515982338,nop,wscale 7], length 0 > 20:43:41.132751 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x590a > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207924299 ecr 515982338,nop,wscale 7], length 0 > 20:43:43.141474 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5716 > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207924799 ecr 515982338,nop,wscale 7], length 0 > 20:43:47.129657 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x532e > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207925799 ecr 515982338,nop,wscale 7], length 0 > 20:43:55.133735 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x4b5e > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207927799 ecr 515982338,nop,wscale 7], length 0 > 20:44:11.134635 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x3bbe > (correct), seq 809114142, ack 2985341181, win 28960, options [mss > 1368,sackOK,TS val 207931799 ecr 515982338,nop,wscale 7], length 0 > 20:45:11.290740 IP (tos 0x0, ttl 64, id 61675, offset 0, flags [DF], proto > TCP (6), length 60) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [S], cksum 0xdbfb (correct), > seq 2733189104, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val > 516074004 ecr 0], length 0 > 20:45:11.595504 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x81f1 > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207946912 ecr 516074004,nop,wscale 7], length 0 > 20:45:12.748574 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x80cf > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207947202 ecr 516074004,nop,wscale 7], length 0 > 20:45:14.746692 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 60) > XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x7edb > (correct), seq 2072870049, ack 2733189105, win 28960, options [mss > 1368,sackOK,TS val 207947702 ecr 516074004,nop,wscale 7], length 0 > > # Captured on lagg0: > > [14:04][ndenev@macbook:~]%tcpdump -vnr 2.pcap host XX.XX.XX.XX > reading from file 2.pcap, link-type EN10MB (Ethernet) > 20:43:39.912570 IP (tos 0x0, ttl 64, id 12524, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->a2c)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:41.132761 IP (tos 0x0, ttl 64, id 13085, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->7fb)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:43.141485 IP (tos 0x0, ttl 64, id 14702, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->1aa)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:47.129669 IP (tos 0x0, ttl 64, id 16915, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->f904)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:43:55.133748 IP (tos 0x0, ttl 64, id 23534, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->df29)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:44:11.134645 IP (tos 0x0, ttl 64, id 30066, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->c5a5)!) > 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x978a), seq 2985341181, win 0, length 0 > 20:45:11.595514 IP (tos 0x0, ttl 64, id 62112, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4877)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:12.748583 IP (tos 0x0, ttl 64, id 62741, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4602)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:14.746703 IP (tos 0x0, ttl 64, id 63874, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->4195)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:18.754657 IP (tos 0x0, ttl 64, id 65318, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->3bf1)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:26.759522 IP (tos 0x0, ttl 64, id 3473, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->2d87)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > 20:45:42.758524 IP (tos 0x0, ttl 64, id 10814, offset 0, flags [DF], proto > TCP (6), length 40, bad cksum 0 (->10da)!) > 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect > -> 0x2cb1), seq 2733189105, win 0, length 0 > [14:04][ndenev@macbook:~]% > > > I'll try the IPFW workaround, I'm not seeing this at the moment (looks like > it was related to how many active torrents there were, as now there are > none). > > --Nikolay > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"