From owner-freebsd-bugs@FreeBSD.ORG Wed Jan 25 19:40:01 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D279106566B for ; Wed, 25 Jan 2012 19:40:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 73C5E8FC1E for ; Wed, 25 Jan 2012 19:40:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0PJe1Jg030195 for ; Wed, 25 Jan 2012 19:40:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0PJe18j030194; Wed, 25 Jan 2012 19:40:01 GMT (envelope-from gnats) Resent-Date: Wed, 25 Jan 2012 19:40:01 GMT Resent-Message-Id: <201201251940.q0PJe18j030194@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Maxim Ignatenko Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7766106572E for ; Wed, 25 Jan 2012 19:32:43 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id 94E018FC0A for ; Wed, 25 Jan 2012 19:32:43 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q0PJWhd5085491 for ; Wed, 25 Jan 2012 19:32:43 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id q0PJWhEd085487; Wed, 25 Jan 2012 19:32:43 GMT (envelope-from nobody) Message-Id: <201201251932.q0PJWhEd085487@red.freebsd.org> Date: Wed, 25 Jan 2012 19:32:43 GMT From: Maxim Ignatenko To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/164490: Incorrect IP checksum on pfil pass from ip_output() X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 19:40:01 -0000 >Number: 164490 >Category: kern >Synopsis: Incorrect IP checksum on pfil pass from ip_output() >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jan 25 19:40:01 UTC 2012 >Closed-Date: >Last-Modified: >Originator: Maxim Ignatenko >Release: 9-STABLE >Organization: >Environment: FreeBSD imax 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #8 r228733: Thu Jan 12 08:15:33 EET 2012 root@imax:/usr/obj/usr/src/sys/IMAX i386 >Description: IP checksum in ipfw on "out" appears to be incorrect: % sudo ipfw show 00100 3899334 2047281422 allow ip from any to any via lo0 00200 0 0 deny ip from 127.0.0.0/8 to any 00300 0 0 deny ip from any to 127.0.0.0/8 00550 8 420 ngtee 10 ip from any to 192.168.56.101 out 00600 1822684 1114344681 allow ip from any to any 65535 0 0 deny ip from any to any % route -n get 192.168.10.10 route to: 192.168.10.10 destination: 192.168.10.0 mask: 255.255.255.0 interface: lagg0 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 % route -n get 192.168.56.101 route to: 192.168.56.101 destination: 192.168.56.0 mask: 255.255.255.0 interface: vboxnet0 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 Next we run "ping -c1 192.168.10.10" on 192.168.56.101 and get these tcpdumps: On gateway interface facing to 192.168.10.10: % sudo tcpdump -i lagg0 -nXvvv -s 0 host 192.168.10.10 and icmp tcpdump: listening on lagg0, link-type EN10MB (Ethernet), capture size 65535 bytes 21:10:15.171175 IP (tos 0x0, ttl 63, id 157, offset 0, flags [none], proto ICMP (1), length 84) 192.168.56.101 > 192.168.10.10: ICMP echo request, id 19972, seq 0, length 64 0x0000: 4500 0054 009d 0000 3f01 b74c c0a8 3865 E..T....?..L..8e 0x0010: c0a8 0a0a 0800 4450 4e04 0000 4f20 5397 ......DPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 21:10:15.173669 IP (tos 0x0, ttl 64, id 13333, offset 0, flags [none], proto ICMP (1), length 84) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 4001 82d4 c0a8 0a0a E..T4...@....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 Response checksum is 0x82d4 On 192.168.101.56: % sudo tcpdump -i em0 -nXvvv -s 0 host 192.168.10.10 and icmp tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes 21:10:15.128315 IP (tos 0x0, ttl 64, id 157, offset 0, flags [none], proto ICMP (1), length 84) 192.168.56.101 > 192.168.10.10: ICMP echo request, id 19972, seq 0, length 64 0x0000: 4500 0054 009d 0000 4001 b64c c0a8 3865 E..T....@..L..8e 0x0010: c0a8 0a0a 0800 4450 4e04 0000 4f20 5397 ......DPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 21:10:15.155980 IP (tos 0x0, ttl 63, id 13333, offset 0, flags [none], proto ICMP (1), length 84) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 3f01 83d4 c0a8 0a0a E..T4...?....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 Here TTL decremented and checksum changed to 0x83d4 On gateway's ng_iface attached to ng_ipfw:10: % sudo tcpdump -i ng0 -n -Xs0 -vvv host 192.168.10.10 tcpdump: WARNING: ng0: no IPv4 address assigned tcpdump: listening on ng0, link-type NULL (BSD loopback), capture size 65535 bytes 21:10:15.173749 IP (tos 0x0, ttl 63, id 13333, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 82d4 (->83d4)!) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 3f01 82d4 c0a8 0a0a E..T4...?....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 And here we get packet with decreased TTL but with old checksum 0x82d4 File with this description is attached to preserve formatting >How-To-Repeat: >Fix: Probably checksum should be recalculated in ip_forward() or in ip_output() before passing packet to pfil. Patch attached with submission follows: IP checksum in ipfw on "out" appears to be incorrect: % sudo ipfw show 00100 3899334 2047281422 allow ip from any to any via lo0 00200 0 0 deny ip from 127.0.0.0/8 to any 00300 0 0 deny ip from any to 127.0.0.0/8 00550 8 420 ngtee 10 ip from any to 192.168.56.101 out 00600 1822684 1114344681 allow ip from any to any 65535 0 0 deny ip from any to any % route -n get 192.168.10.10 route to: 192.168.10.10 destination: 192.168.10.0 mask: 255.255.255.0 interface: lagg0 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 % route -n get 192.168.56.101 route to: 192.168.56.101 destination: 192.168.56.0 mask: 255.255.255.0 interface: vboxnet0 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 Next we run "ping -c1 192.168.10.10" on 192.168.56.101 and get these tcpdumps: On gateway interface facing to 192.168.10.10: % sudo tcpdump -i lagg0 -nXvvv -s 0 host 192.168.10.10 and icmp tcpdump: listening on lagg0, link-type EN10MB (Ethernet), capture size 65535 bytes 21:10:15.171175 IP (tos 0x0, ttl 63, id 157, offset 0, flags [none], proto ICMP (1), length 84) 192.168.56.101 > 192.168.10.10: ICMP echo request, id 19972, seq 0, length 64 0x0000: 4500 0054 009d 0000 3f01 b74c c0a8 3865 E..T....?..L..8e 0x0010: c0a8 0a0a 0800 4450 4e04 0000 4f20 5397 ......DPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 21:10:15.173669 IP (tos 0x0, ttl 64, id 13333, offset 0, flags [none], proto ICMP (1), length 84) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 4001 82d4 c0a8 0a0a E..T4...@....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 Response checksum is 0x82d4 On 192.168.101.56: % sudo tcpdump -i em0 -nXvvv -s 0 host 192.168.10.10 and icmp tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes 21:10:15.128315 IP (tos 0x0, ttl 64, id 157, offset 0, flags [none], proto ICMP (1), length 84) 192.168.56.101 > 192.168.10.10: ICMP echo request, id 19972, seq 0, length 64 0x0000: 4500 0054 009d 0000 4001 b64c c0a8 3865 E..T....@..L..8e 0x0010: c0a8 0a0a 0800 4450 4e04 0000 4f20 5397 ......DPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 21:10:15.155980 IP (tos 0x0, ttl 63, id 13333, offset 0, flags [none], proto ICMP (1), length 84) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 3f01 83d4 c0a8 0a0a E..T4...?....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 Here TTL decremented and checksum changed to 0x83d4 On gateway's ng_iface attached to ng_ipfw:10: % sudo tcpdump -i ng0 -n -Xs0 -vvv host 192.168.10.10 tcpdump: WARNING: ng0: no IPv4 address assigned tcpdump: listening on ng0, link-type NULL (BSD loopback), capture size 65535 bytes 21:10:15.173749 IP (tos 0x0, ttl 63, id 13333, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 82d4 (->83d4)!) 192.168.10.10 > 192.168.56.101: ICMP echo reply, id 19972, seq 0, length 64 0x0000: 4500 0054 3415 0000 3f01 82d4 c0a8 0a0a E..T4...?....... 0x0010: c0a8 3865 0000 4c50 4e04 0000 4f20 5397 ..8e..LPN...O.S. 0x0020: 0001 d7ef 0809 0a0b 0c0d 0e0f 1011 1213 ................ 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 3637 4567 And here we get packet with decreased TTL but with old checksum 0x82d4 >Release-Note: >Audit-Trail: >Unformatted: