From owner-freebsd-net@FreeBSD.ORG Wed Nov 27 10:58:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FB8B7CE for ; Wed, 27 Nov 2013 10:58:13 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 09470281A for ; Wed, 27 Nov 2013 10:58:13 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id g10so9711463pdj.15 for ; Wed, 27 Nov 2013 02:58:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=mKBTiK5+aSUldwQtgMkekKqyz+ohAThM1nhErNV1j2Y=; b=okXLZd10t2m3c/KmJm7yQ/89XqIVIqGaMuBAhOqq3AuUBa5ISlPwykRgLyyZSUBx1e wSfzXddusOzHUJJhE007tLnxgcQzwLUJMina6kmAX00Yj8g6lLMK/GM9UxxjPiwH6SIh 7Ad502eur/WQWm/WBqoAPO2I8S1a+uGX8ner2Ki2ilGIJHwmjhY7LlGs8gB49FtjMZFe bhXhkRHxrcLBXgDLQjydpKP2sc1eHgZLrQDwIQ6a4Rthoh8jM28iX+GNYaPRfcXrp/xw 8dbxTRlfiuI81Z9UEZ+tZT8B0TgNuJKf4Ou/98NvcpnLJceIAsKzoxuBp2zOz7oUze8w F5dw== MIME-Version: 1.0 X-Received: by 10.67.30.70 with SMTP id kc6mr40220461pad.32.1385549892602; Wed, 27 Nov 2013 02:58:12 -0800 (PST) Sender: ermal.luci@gmail.com Received: by 10.70.4.163 with HTTP; Wed, 27 Nov 2013 02:58:12 -0800 (PST) In-Reply-To: <87zjoqu3wr.fsf@marcos.anarc.at> References: <87zjoqu3wr.fsf@marcos.anarc.at> Date: Wed, 27 Nov 2013 11:58:12 +0100 X-Google-Sender-Auth: WL56F0B83arVrLyxu3VmwDmOSRw Message-ID: Subject: Re: OpenBGPd + TCP-MD5 sig fails after a few weeks From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: =?ISO-8859-1?Q?Antoine_Beaupr=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.16 Cc: freebsd-net , "Bjoern A. Zeeb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 10:58:13 -0000 On Wed, Nov 27, 2013 at 12:41 AM, Antoine Beaupr=E9 wr= ote: > [please CC me I am not on the list. also cc'ing bzeeb since he was > working on a patch for this two years ago] > > Hi, > > I have configured an OpenBGPd daemon to connect to another provider with = a > TCP-MD5 password. > > It used to work when I set it up 30 days ago, but somehow the session is > down now. > > # bgpctl show > Neighbor AS MsgRcvd MsgSent OutQ Up/Down > State/PrfRcvd > Cogent 174 0 0 0 Never Active > > I suspect this was working only because the remote server was > initializing the connexion and not us. > > What is ackward is that OpenBGPd doesn't seem to properly initialise the > socket with an MD5 signature: > > 17:54:59.479900 IP (tos 0xc0, ttl 1, id 14983, offset 0, flags [DF], prot= o > TCP (6), length 60, bad cksum 0 (->c0d9)!) > 38.104.152.102.16295 > 38.104.152.101.179: Flags [S], cksum 0x096d > (correct), seq 1556933933, win 65535, options [mss 1460,nop,wscale > 6,sackOK,TS val 1324688 ecr 0], length 0 > 17:54:59.480593 IP (tos 0x0, ttl 255, id 30414, offset 0, flags > [none],proto TCP (6), length 40) > 38.104.152.101.179 > 38.104.152.102.16295: Flags [R.], cksum 0xa7df > (correct), seq 0, ack 1556933934, win 0, length 0 > > Usually, this should mention "md5valid" in the options field if it is > enabled. > > This actually works with netcat: > > # nc -v -S 38.104.152.101 179 > nc: connect to 38.104.152.101 port 179 (tcp) failed: Connection refused > > # tcpdump -M [...] -i bge0 -n -vvv port 179 > tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size > 65535 bytes > 18:15:43.534592 IP (tos 0x0, ttl 64, id 27666, offset 0, flags [DF], prot= o > TCP (6), length 80, bad cksum 0 (->50fa)!) > 38.104.152.102.26043 > 38.104.152.101.179: Flags [S], cksum 0xe73a > (correct), seq 3803575904, win 65535, options [mss 1460,nop,wscale > 6,sackOK,TS val 2568742 ecr 0,nop,nop,md5valid], length 0 > 18:15:43.536592 IP (tos 0x0, ttl 255, id 30526, offset 0, flags [none], > proto TCP (6), length 60) > 38.104.152.101.179 > 38.104.152.102.26043: Flags [R.], cksum 0x1566 > (correct), seq 0, ack 3803575905, win 0, options [md5valid,eol], length 0 > > Notice, however, how the other side is still reseting our connexion, so > there's something wrong there - but it could be that they simply blocked > us because of too many failed attempts. > > The SAD association is set properly: > > # setkey -D > 38.104.152.102 38.104.152.101 > tcp mode=3Dany spi=3D4096(0x00001000) reqid=3D0(0x00000000) > A: tcp-md5 [...] > seq=3D0x00000000 replay=3D0 flags=3D0x00000040 state=3Dmature > created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013 > diff: 1181(s) hard: 0(s) soft: 0(s) > last: hard: 0(s) soft: 0(s) > current: 0(bytes) hard: 0(bytes) soft: 0(bytes) > allocated: 0 hard: 0 soft: 0 > sadb_seq=3D1 pid=3D31485 refcnt=3D1 > 38.104.152.101 38.104.152.102 > tcp mode=3Dany spi=3D4096(0x00001000) reqid=3D0(0x00000000) > A: tcp-md5 [...] > seq=3D0x00000000 replay=3D0 flags=3D0x00000040 state=3Dmature > created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013 > diff: 1181(s) hard: 0(s) soft: 0(s) > last: hard: 0(s) soft: 0(s) > current: 0(bytes) hard: 0(bytes) soft: 0(bytes) > allocated: 0 hard: 0 soft: 0 > sadb_seq=3D0 pid=3D31485 refcnt=3D1 > > here's our ipsec.conf: > > add -n 38.104.152.101 38.104.152.102 tcp 0x1000 -A tcp-md5 "[...]"; > add -n 38.104.152.102 38.104.152.101 tcp 0x1000 -A tcp-md5 "[...]"; > > I have tried both openbgpd-5.2.20121209 and openbgpd-5.2.20121014. I > have also tried to disable TCP verification through sysctl, no luck: > > net.inet.tcp.signature_verify_input: 0 > > We have the following kernel configuration: > > include GENERIC > ident KOUMBIT1 > device pf > device pflog > device pfsync > options ALTQ > options ALTQ_CBQ > options ALTQ_RED > options ALTQ_RIO > options ALTQ_HFSC > options ALTQ_CDNR > options ALTQ_PRIQ > options IPSEC #IP security > options TCP_SIGNATURE > device crypto > options DEVICE_POLLING > device carp > > Our bgpd.conf doesn't specify a tcp password, otherwise we end up with > the following error: > > root@rtr0:/usr/local/etc # bgpd -d > startup > rereading config > no kernel support for PF_KEY > route decision engine ready > session engine ready > RDE reconfigured > listening on 0.0.0.0 > listening on :: > SE reconfigured > neighbor 38.104.152.101 (Cogent): state change None -> Idle, reason: None > neighbor 38.104.152.101 (Cogent): pfkey setup failed > > Why is it that outgoing TCP connexions do not respect setkey settings? > > I read a little bit of the source code, and it seems that openbgpd is > stuck in a catch-22: it can't setup the SAD associations (because they > are handled by setkey) so it doesn't set the MD5SIG option on the > socket... One horrible solution I found was to change session_connect() > as such: > > - if (peer->conf.auth.method !=3D AUTH_NONE && sysdep.no_pfkey) { > + if (peer->conf.auth.method !=3D AUTH_NONE && sysdep.no_pfkey || 1= ) { > log_peer_warnx(&peer->conf, > - "ipsec or md5sig configured but not available"); > - bgp_fsm(peer, EVNT_CON_OPENFAIL); > - return (-1); > + "ipsec or md5sig configured but not available, > assuming set externally"); > + /* bgp_fsm(peer, EVNT_CON_OPENFAIL); */ > + /* return (-1); */ > + if (setsockopt(peer->fd, IPPROTO_TCP, TCP_MD5SIG, > + &opt, sizeof(opt)) =3D=3D -1) { > + log_peer_warn(&peer->conf, "setsockopt md5sig"); > + bgp_fsm(peer, EVNT_CON_OPENFAIL); > + return (-1); > + } > + > } > > It's pretty nasty, but with this at least the connexion gets initialized > going out with the right socket options. > > You can use the port here https://github.com/pfsense/pfsense-tools/tree/master/pfPorts/openbgpd It has integration with pfkey sockets of FreeBSD in the daemon itself and you have to specify only th espd policy through setkey. It works for pfSense. > A. > > PS: this is a problem similar to what was reported here: > > http://lists.freebsd.org/pipermail/freebsd-net/2012-January/030921.html > > -- > Il faut tout un village pour =E9lever un enfant. > - Proverbe africain > --=20 Ermal