From owner-freebsd-security@FreeBSD.ORG Thu Nov 20 21:35:33 2014 Return-Path: Delivered-To: freebsd-security@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1F74433; Thu, 20 Nov 2014 21:35:33 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 923ACF9D; Thu, 20 Nov 2014 21:35:33 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sAKLZQGr098565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 20 Nov 2014 13:35:27 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sAKLZQk2098564; Thu, 20 Nov 2014 13:35:26 -0800 (PST) (envelope-from jmg) Date: Thu, 20 Nov 2014 13:35:26 -0800 From: John-Mark Gurney To: freebsd-net@FreeBSD.org, freebsd-security@FreeBSD.org Subject: IPsec is very broken... Message-ID: <20141120213526.GH24601@funkthat.com> Mail-Followup-To: freebsd-net@FreeBSD.org, freebsd-security@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 20 Nov 2014 13:35:27 -0800 (PST) X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Nov 2014 21:35:33 -0000 As I'm about to commit my AES-GCM work, I've been trying to do some testing to make sure I didn't break IPsec. The first major issue I ran across was transport mode... ae@ has been nice enough to get ICMP working in transport mode for IPv4 and IPv6, but it looks like TCP is still broken. I haven't tested UDP yet... So, IPsec even w/o crypto is fundamentally broken here... It's clear that not many people run transport mode... If someone could create a good test suite that ensures makes sure basic IPsec traffic passes, that would be a huge win for us. The tests should test a complete cross product of: { tunnel, transport } x { TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this list. There is a lock order reversal which someone should look at: # setkey -DP lock order reversal: 1st 0xffffffff817fc658 sptree (fast ipsec security policy database) @ netipsec/key.c:2453 2nd 0xffffffff818bd920 rawcb (rawcb) @ netipsec/keysock.c:303 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe001de9d330 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe001de9d3e0 witness_checkorder() at witness_checkorder+0xdad/frame 0xfffffe001de9d470 __mtx_lock_flags() at __mtx_lock_flags+0xa8/frame 0xfffffe001de9d4c0 key_sendup_mbuf() at key_sendup_mbuf+0xb0/frame 0xfffffe001de9d500 key_spddump() at key_spddump+0x188/frame 0xfffffe001de9d540 key_parse() at key_parse+0x8fa/frame 0xfffffe001de9d740 key_output() at key_output+0xeb/frame 0xfffffe001de9d7a0 sosend_generic() at sosend_generic+0x405/frame 0xfffffe001de9d850 kern_sendit() at kern_sendit+0x20b/frame 0xfffffe001de9d900 sendit() at sendit+0x129/frame 0xfffffe001de9d950 sys_sendto() at sys_sendto+0x4d/frame 0xfffffe001de9d9a0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe001de9dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe001de9dab0 --- syscall (133, FreeBSD ELF64, sys_sendto), rip = 0x800b5c5aa, rsp = 0x7fffffff6ae8, rbp = 0x7fffffffeb50 --- The following is more OpenCrypto or aesni related, but as IPsec is a consumer, they are related. It also looks like OpenCrypto session management wasn't on anyone's list, partly because the docs for it aren't very good, and until AES-NI came along, if you provided an explicit IV, the software stack would "magicly" work. Now w/ AES-NI, that is no longer the case... Currently, the aesni driver only has one FPU context allocated per session, but IPsec will reuse the same session for different requests from different threads. This means that you can get panics like: panic: System call sendto returing with kernel FPU ctx leaked or: panic: dummy ctx As the same session fpu context will be assigned to different kernel threads stomping on each other's state. Both transport and tunnel mode suffer (panics) from this issue. I'm working w/ kib on a possible solution to this. There are other issues, such as all the session management code is looked up via linked lists... This isn't good for performance... One of the reasons why this was done was to allow migrating sessions between capable devices, such that if one was overloaded/busy, you could use another one that isn't as busy... There are better solutions/patterns now that weren't used back then that should get us around this issue.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."