From owner-freebsd-net@FreeBSD.ORG Sun Jul 31 18:39:58 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DA0A1065670 for ; Sun, 31 Jul 2011 18:39:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id BF2D38FC16 for ; Sun, 31 Jul 2011 18:39:57 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAG2gNU6DaFvO/2dsb2JhbABAFoQxpA6BQAEBAQECAQEBASArIAsbDgoCAg0ZAikBCSYGCAcEARwEh0sErjmQD4ErhAeBEASQb4IMkQA X-IronPort-AV: E=Sophos;i="4.67,296,1309752000"; d="scan'208";a="132941337" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 31 Jul 2011 14:39:56 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C9F43B3F33; Sun, 31 Jul 2011 14:39:56 -0400 (EDT) Date: Sun, 31 Jul 2011 14:39:56 -0400 (EDT) From: Rick Macklem To: Jeremiah Lott Message-ID: <1211623207.1226575.1312137596762.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org Subject: Re: LOR with nfsclient "sillyrename" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 18:39:58 -0000 Jeremiah Lott wrote: > On Jul 22, 2011, at 5:11 PM, Rick Macklem wrote: > > > Please try the attached patch (which is also at): > > http://people.freebsd.org/~rmacklem/oldsilly.patch > > http://people.freebsd.org/~rmacklem/newsilly.patch > > (for the old and new clients in -current, respectively) > > > > - I think oldsilly.patch should apply to the 7.n kernel > > sources, although you might have to do the edit by hand? > > It applied with minimal futzing. > Just to clarify.. Was there anything other than different line #s needed? (If I am going to MFC it back to stable/7, I'll need to know, since I don't currently have a stable/7 system installed to test with.) Thanks for letting me know how it's going, rick > > Please let me know how testing goes with it, rick > > Unfortunately we've never reproduced the original problem in the lab. > Only in the field under heavy stress. I did build a kernel with the > patch and run it under some of our tests, it seems to work correctly. > We'll continue to test it, but I wanted to give you an update. Thanks > a lot for your help. > > Jeremiah Lott > Avere Systems > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Aug 1 07:01:04 2011 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 300DB106566C; Mon, 1 Aug 2011 07:01:04 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0782B8FC23; Mon, 1 Aug 2011 07:01:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p71713UV055011; Mon, 1 Aug 2011 07:01:03 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p71713lJ055004; Mon, 1 Aug 2011 07:01:03 GMT (envelope-from linimon) Date: Mon, 1 Aug 2011 07:01:03 GMT Message-Id: <201108010701.p71713lJ055004@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/159345: [lagg] [panic] kernel with if_lagg compiled in panices at start if if_lagg.ko is loaded X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 07:01:04 -0000 Old Synopsis: [if_lagg] [panic] kernel with if_lagg compiled in panices at start if if_lagg.ko is loaded New Synopsis: [lagg] [panic] kernel with if_lagg compiled in panices at start if if_lagg.ko is loaded Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Mon Aug 1 07:00:37 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=159345 From owner-freebsd-net@FreeBSD.ORG Mon Aug 1 10:30:10 2011 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DDD31065676; Mon, 1 Aug 2011 10:30:10 +0000 (UTC) (envelope-from pluknet@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 15FBD8FC1A; Mon, 1 Aug 2011 10:30:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p71AU9au073478; Mon, 1 Aug 2011 10:30:09 GMT (envelope-from pluknet@freefall.freebsd.org) Received: (from pluknet@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p71AU9IE073472; Mon, 1 Aug 2011 10:30:09 GMT (envelope-from pluknet) Date: Mon, 1 Aug 2011 10:30:09 GMT Message-Id: <201108011030.p71AU9IE073472@freefall.freebsd.org> To: pluknet@FreeBSD.org, freebsd-net@FreeBSD.org, pluknet@FreeBSD.org From: pluknet@FreeBSD.org Cc: Subject: Re: kern/159345: [lagg] [panic] kernel with if_lagg compiled in panices at start if if_lagg.ko is loaded X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 10:30:10 -0000 Synopsis: [lagg] [panic] kernel with if_lagg compiled in panices at start if if_lagg.ko is loaded Responsible-Changed-From-To: freebsd-net->pluknet Responsible-Changed-By: pluknet Responsible-Changed-When: Mon Aug 1 10:29:53 UTC 2011 Responsible-Changed-Why: Take. http://www.freebsd.org/cgi/query-pr.cgi?pr=159345 From owner-freebsd-net@FreeBSD.ORG Mon Aug 1 11:07:13 2011 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07912106567C for ; Mon, 1 Aug 2011 11:07:13 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E99A98FC2A for ; Mon, 1 Aug 2011 11:07:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p71B7C23014640 for ; Mon, 1 Aug 2011 11:07:12 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p71B7CxB014638 for freebsd-net@FreeBSD.org; Mon, 1 Aug 2011 11:07:12 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Aug 2011 11:07:12 GMT Message-Id: <201108011107.p71B7CxB014638@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 11:07:13 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/158426 net [e1000] [panic] _mtx_lock_sleep: recursed on non-recur o kern/158156 net [bce] bce driver shows "no carrier" on IBM blade (HS22 f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157429 net [re] Realtek RTL8169 doesn't work with re(4) o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155772 net ifconfig(8): ioctl (SIOCAIFADDR): File exists on direc o kern/155680 net [multicast] problems with multicast s kern/155642 net [request] Add driver for Realtek RTL8191SE/RTL8192SE W o kern/155604 net [flowtable] Flowtable excessively caches dest MAC addr o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155585 net [tcp] [panic] tcp_output tcp_mtudisc loop until kernel o kern/155420 net [vlan] adding vlan break existent vlan o bin/155365 net [patch] routed(8): if.c in routed fails to compile if o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/155004 net [bce] [panic] kernel panic in bce0 driver o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [request]: Port brcm80211 driver from Linux to FreeBSD o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m o kern/146426 net [mwl] 802.11n rates not possible on mwl o kern/146425 net [mwl] mwl dropping all packets during and after high u f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o bin/145934 net [patch] add count option to netstat(1) o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 o kern/144572 net [carp] CARP preemption mode traffic partially goes to f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/141023 net [carp] CARP arp replays with wrong src mac o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph o kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138620 net [lagg] [patch] lagg port bpf-writes blocked o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o bin/136661 net [patch] ndp(8) ignores -f option o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/136426 net [panic] spawning several dhclients in parallel panics o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre o kern/133218 net [carp] [hang] use of carp(4) causes system to freeze f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132285 net [carp] alias gives incorrect hash in dmesg o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/132107 net [carp] carp(4) advskew setting ignored when carp IP us o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow o kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/127050 net [carp] ipv6 does not work on carp interfaces [regressi o kern/126945 net [carp] CARP interface destruction with ifconfig destro o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126714 net [carp] CARP interface renaming makes system no longer o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125816 net [carp] [if_bridge] carp stuck in init when using bridg f kern/125442 net [carp] [lagg] CARP combined with LAGG causes system pa o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 f kern/123045 net [ng_mppc] ng_mppc_decompress - disabling node o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o kern/120130 net [carp] [panic] carp causes kernel panics in any conste o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117448 net [carp] 6.2 kernel crash [regression] o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging s kern/90086 net [hang] 5.4p8 on supermicro P8SCT hangs during boot if o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ p kern/85320 net [gre] [patch] possible depletion of kernel stack in ip o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/79895 net [ipf] 5.4-RC2 breaks ipfilter NAT when using netgraph o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr s kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 377 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Aug 1 17:30:40 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0876106564A for ; Mon, 1 Aug 2011 17:30:40 +0000 (UTC) (envelope-from prvs=1194f7599d=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 978018FC16 for ; Mon, 1 Aug 2011 17:30:39 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Mon, 01 Aug 2011 18:19:55 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 01 Aug 2011 18:19:55 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014420253.msg for ; Mon, 01 Aug 2011 18:19:55 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1194f7599d=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-net@freebsd.org Message-ID: From: "Steven Hartland" To: Date: Mon, 1 Aug 2011 18:20:28 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Subject: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 17:30:40 -0000 We've been investiging an ongoing issue with some machines we have in Amsterdam, when receiving files via scp from our London DC. Initially we suspected a igb driver issue but after continued diagnosis including disabling hardware tso and cksums on both end we're currently investigating the possibility of a tcp stack issue. What I believe we're seeing from the tcpdump trace is the packet loss on the network results in an unrecoverable tcp session. >From my very limited knowledge of tcp, I would expect to see the session to recover after receiving the lost packet by either a fast retransmit or standard retransmit. My interpretation of the capture is the packet is indeed resent by the sender but that the receiver continues to respond with TCP Dup ACK for said missing packet instead of recovering. Trace is below, which starts from just before the point the receiver sees the dropped packet to the point where the sender gives up and resets the session. Is my analysis correct and does it look like this is a tcp stack bug? tshark -z "proto,colinfo,tcp.seq,tcp.seq" -z "proto,colinfo,tcp.nxtseq,tcp.nxtseq" -r tcp-stall.pcap 1 0.000000 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 1449 tcp.seq == 1 2 0.000028 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 2897 tcp.seq == 1449 3 0.000034 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=2897 Win=9102 Len=0 TSV=1902423651 TSER=261560659 tcp.seq == 49 4 0.000051 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 4345 tcp.seq == 2897 5 0.000078 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 5793 tcp.seq == 4345 6 0.000084 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=5793 Win=8921 Len=0 TSV=1902423651 TSER=261560659 tcp.seq == 49 7 0.000100 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 7241 tcp.seq == 5793 8 0.000135 10.10.1.10 -> 10.10.1.20 SSH [TCP Previous segment lost] Encrypted request packet len=1448 tcp.nxtseq == 10137 tcp.seq == 8689 9 0.000147 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=10137 tcp.seq == 49 10 0.000167 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 11585 tcp.seq == 10137 11 0.000173 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#1] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=11585 tcp.seq == 49 12 0.000180 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 13033 tcp.seq == 11585 13 0.000186 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#2] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=13033 tcp.seq == 49 14 0.000191 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 14481 tcp.seq == 13033 15 0.000197 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#3] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=14481 tcp.seq == 49 16 0.000201 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 15929 tcp.seq == 14481 17 0.000209 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#4] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=15929 tcp.seq == 49 18 0.000229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 17377 tcp.seq == 15929 19 0.000237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#5] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=17377 tcp.seq == 49 20 0.000460 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 18825 tcp.seq == 17377 21 0.000469 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#6] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=18825 tcp.seq == 49 22 0.000505 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 20273 tcp.seq == 18825 23 0.000511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#7] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=20273 tcp.seq == 49 24 0.000513 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 21721 tcp.seq == 20273 25 0.000519 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#8] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=21721 tcp.seq == 49 26 0.000528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 23169 tcp.seq == 21721 27 0.000535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#9] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 28 0.000539 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 24617 tcp.seq == 23169 29 0.000546 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#10] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 30 0.000556 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 26065 tcp.seq == 24617 31 0.000562 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#11] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 32 0.000565 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 27513 tcp.seq == 26065 33 0.000572 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9011 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 34 0.000579 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 35 0.000596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 28961 tcp.seq == 27513 36 0.000604 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#1] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 37 0.000621 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 30409 tcp.seq == 28961 38 0.000627 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#2] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 39 0.000650 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 31857 tcp.seq == 30409 40 0.000657 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#3] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 41 0.000672 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 33305 tcp.seq == 31857 42 0.000679 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#4] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 43 0.000799 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 34753 tcp.seq == 33305 44 0.000806 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#5] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 45 0.000857 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 36201 tcp.seq == 34753 46 0.000866 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#6] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 47 0.000939 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 37649 tcp.seq == 36201 48 0.000946 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#7] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 49 0.000955 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 39097 tcp.seq == 37649 50 0.000963 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#8] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 51 0.001256 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 40545 tcp.seq == 39097 52 0.001268 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#9] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 53 0.001300 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 41993 tcp.seq == 40545 54 0.001309 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#10] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 55 0.001352 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 43441 tcp.seq == 41993 56 0.001359 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#11] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 57 0.001431 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 44889 tcp.seq == 43441 58 0.001439 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#12] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 59 0.001468 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 46337 tcp.seq == 44889 60 0.001475 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#13] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 61 0.001492 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 47785 tcp.seq == 46337 62 0.001498 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#14] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 63 0.001569 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 49233 tcp.seq == 47785 64 0.001576 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#15] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 65 0.001879 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 50681 tcp.seq == 49233 66 0.001890 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#16] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 67 0.001925 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 52129 tcp.seq == 50681 68 0.001933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#17] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 69 0.002006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 53577 tcp.seq == 52129 70 0.002016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#18] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 71 0.002218 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 55025 tcp.seq == 53577 72 0.002225 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#19] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 73 0.002281 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 56473 tcp.seq == 55025 74 0.002288 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#20] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 75 0.002343 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 57921 tcp.seq == 56473 76 0.002350 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#21] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 77 0.002525 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 59369 tcp.seq == 57921 78 0.002535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#22] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 79 0.002580 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 60817 tcp.seq == 59369 80 0.002589 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#23] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 81 0.002772 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 62265 tcp.seq == 60817 82 0.002779 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#24] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 83 0.002826 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 63713 tcp.seq == 62265 84 0.002834 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#25] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 85 0.002889 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 65161 tcp.seq == 63713 86 0.002896 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#26] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 87 0.002967 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 66609 tcp.seq == 65161 88 0.002974 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#27] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 89 0.003411 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 68057 tcp.seq == 66609 90 0.003419 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#28] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 91 0.003457 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 69505 tcp.seq == 68057 92 0.003464 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#29] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 93 0.003557 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 70953 tcp.seq == 69505 94 0.003564 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#30] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 95 0.003644 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 72401 tcp.seq == 70953 96 0.003653 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#31] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 97 0.003675 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 73849 tcp.seq == 72401 98 0.003683 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#32] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 99 0.003728 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 75297 tcp.seq == 73849 100 0.003735 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#33] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 101 0.003780 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 76745 tcp.seq == 75297 102 0.003792 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#34] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 103 0.003829 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 78193 tcp.seq == 76745 104 0.003842 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#35] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 105 0.003874 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 79641 tcp.seq == 78193 106 0.003883 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#36] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 107 0.004165 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 81089 tcp.seq == 79641 108 0.004172 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#37] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 109 0.004279 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 82537 tcp.seq == 81089 110 0.004287 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#38] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 111 0.004340 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 83985 tcp.seq == 82537 112 0.004347 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#39] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 113 0.004400 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 85433 tcp.seq == 83985 114 0.004408 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#40] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 115 0.004887 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 86881 tcp.seq == 85433 116 0.004894 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#41] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 117 0.004940 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 88329 tcp.seq == 86881 118 0.004947 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#42] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 119 0.005006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 89777 tcp.seq == 88329 120 0.005015 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#43] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 121 0.005140 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 91225 tcp.seq == 89777 122 0.005150 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#44] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 123 0.005194 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 92673 tcp.seq == 91225 124 0.005203 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#45] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 125 0.005263 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 94121 tcp.seq == 92673 126 0.005270 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#46] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 127 0.005413 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 95569 tcp.seq == 94121 128 0.005420 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#47] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 129 0.005480 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 97017 tcp.seq == 95569 130 0.005487 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#48] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 131 0.005504 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 98465 tcp.seq == 97017 132 0.005511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#49] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 133 0.005528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 99913 tcp.seq == 98465 134 0.005536 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#50] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 135 0.005553 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 101361 tcp.seq == 99913 136 0.005561 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#51] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 137 0.005576 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 102809 tcp.seq == 101361 138 0.005583 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#52] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 139 0.005627 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 104257 tcp.seq == 102809 140 0.005634 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#53] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 141 0.005689 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 105705 tcp.seq == 104257 142 0.005696 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#54] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 143 0.005704 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 107153 tcp.seq == 105705 144 0.005710 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#55] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 145 0.005716 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 108601 tcp.seq == 107153 146 0.005724 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#56] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 147 0.005727 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 110049 tcp.seq == 108601 148 0.005734 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#57] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 149 0.005739 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 111497 tcp.seq == 110049 150 0.005746 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#58] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 151 0.005756 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 112945 tcp.seq == 111497 152 0.005765 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#59] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 153 0.005768 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 114393 tcp.seq == 112945 154 0.005777 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#60] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 155 0.005781 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 115841 tcp.seq == 114393 156 0.005789 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#61] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 157 0.006154 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 117289 tcp.seq == 115841 158 0.006162 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#62] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 159 0.006200 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 118737 tcp.seq == 117289 160 0.006207 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#63] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 161 0.006229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 120185 tcp.seq == 118737 162 0.006237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#64] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 163 0.006253 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 121633 tcp.seq == 120185 164 0.006259 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#65] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 165 0.006278 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 123081 tcp.seq == 121633 166 0.006285 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#66] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 167 0.006294 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 124529 tcp.seq == 123081 168 0.006300 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#67] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 169 0.006304 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 125977 tcp.seq == 124529 170 0.006310 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#68] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 171 0.006315 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 127425 tcp.seq == 125977 172 0.006323 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#69] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 173 0.006333 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 128873 tcp.seq == 127425 174 0.006340 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#70] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 175 0.006342 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 130321 tcp.seq == 128873 176 0.006349 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#71] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 177 0.006358 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 131769 tcp.seq == 130321 178 0.006365 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#72] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 179 0.006368 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 133217 tcp.seq == 131769 180 0.006375 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#73] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 181 0.006388 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 134665 tcp.seq == 133217 182 0.006395 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#74] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 183 0.006435 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 136113 tcp.seq == 134665 184 0.006446 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#75] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 185 0.006506 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 137561 tcp.seq == 136113 186 0.006516 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#76] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 187 0.006558 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 139009 tcp.seq == 137561 188 0.006565 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#77] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 189 0.006568 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 140457 tcp.seq == 139009 190 0.006574 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#78] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 191 0.006579 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 141905 tcp.seq == 140457 192 0.006585 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#79] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 193 0.006596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 143353 tcp.seq == 141905 194 0.006603 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#80] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 195 0.006606 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 144801 tcp.seq == 143353 196 0.006612 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#81] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 197 0.006654 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 146249 tcp.seq == 144801 198 0.006661 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#82] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 199 0.006696 10.10.1.10 -> 10.10.1.20 SSH [TCP Fast Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 200 0.006703 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#83] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 TSER=261560666 SLE=8689 SRE=23169 tcp.seq == 49 201 0.246652 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 202 0.246664 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#84] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423897 TSER=261560896 SLE=8689 SRE=23169 tcp.seq == 49 203 0.518331 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 204 0.518343 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#85] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424168 TSER=261561156 SLE=8689 SRE=23169 tcp.seq == 49 205 0.852522 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 206 0.852535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#86] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424501 TSER=261561476 SLE=8689 SRE=23169 tcp.seq == 49 207 1.311920 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 208 1.311933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#87] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424958 TSER=261561916 SLE=8689 SRE=23169 tcp.seq == 49 209 2.022170 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 210 2.022184 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#88] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902425666 TSER=261562596 SLE=8689 SRE=23169 tcp.seq == 49 211 3.233004 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 212 3.233016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#89] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902426872 TSER=261563756 SLE=8689 SRE=23169 tcp.seq == 49 213 5.444340 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 214 5.444353 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#90] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902429074 TSER=261565876 SLE=8689 SRE=23169 tcp.seq == 49 215 9.658815 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 216 9.658826 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#91] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902433272 TSER=261569916 SLE=8689 SRE=23169 tcp.seq == 49 217 17.882735 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 218 17.882748 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#92] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902441463 TSER=261577796 SLE=8689 SRE=23169 tcp.seq == 49 219 34.113339 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 220 34.113352 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#93] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902457629 TSER=261593356 SLE=8689 SRE=23169 tcp.seq == 49 221 50.343593 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 222 50.343607 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#94] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902473795 TSER=261608916 SLE=8689 SRE=23169 tcp.seq == 49 223 66.567449 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq == 7241 224 66.567461 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#95] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902489954 TSER=261624476 SLE=8689 SRE=23169 tcp.seq == 49 225 82.793601 10.10.1.10 -> 10.10.1.20 TCP 28296 > ssh [RST, ACK] Seq=146249 Ack=49 Win=4163 Len=0 TSV=261640036 TSER=1902489954 tcp.seq == 146249 Just to confirm both machines involved are running 8.2-RELEASE with both hardware checksums and offloading disabled using -tso -rxcsum -txcsum. The sender is using em and the receiver using igb. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 01:51:45 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A039C1065686; Tue, 2 Aug 2011 01:51:45 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 191008FC0A; Tue, 2 Aug 2011 01:51:43 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 02:40:20 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 02:40:20 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014427038.msg; Tue, 02 Aug 2011 02:40:20 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk> From: "Steven Hartland" To: , References: Date: Tue, 2 Aug 2011 02:40:51 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 01:51:45 -0000 ----- Original Message ----- > What I believe we're seeing from the tcpdump trace is the > packet loss on the network results in an unrecoverable tcp > session. > > From my very limited knowledge of tcp, I would expect to see > the session to recover after receiving the lost packet by > either a fast retransmit or standard retransmit. > > My interpretation of the capture is the packet is indeed resent > by the sender but that the receiver continues to respond > with TCP Dup ACK for said missing packet instead of recovering. > > Trace is below, which starts from just before the point > the receiver sees the dropped packet to the point where the sender > gives up and resets the session. > > Is my analysis correct and does it look like this is a tcp > stack bug? Something I've just noticed that might be of note for this is that the receiving machine seeing the stall seems to be increasing its "discarded due to memory problems" count at a decent rate:- 127187 discarded due to memory problems the 1 second later during a stall 127192 discarded due to memory problems I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the following comment:- * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat * should work but causes packets to be dropped when they shouldn't. I notice this code is relatively new, so I'm wondering if this may be something to do with what we're seeing, possibly still dropping packets it shouldn't? @Lawrence apologies' for the direct mail, but I believe you where the original author this particular change so wondered if you may be able to shed any light on this? > > tshark -z "proto,colinfo,tcp.seq,tcp.seq" -z "proto,colinfo,tcp.nxtseq,tcp.nxtseq" -r tcp-stall.pcap > 1 0.000000 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 1449 tcp.seq == 1 > 2 0.000028 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 2897 tcp.seq == 1449 > 3 0.000034 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=2897 Win=9102 Len=0 TSV=1902423651 TSER=261560659 > tcp.seq == 49 > 4 0.000051 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 4345 tcp.seq == 2897 > 5 0.000078 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 5793 tcp.seq == 4345 > 6 0.000084 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=5793 Win=8921 Len=0 TSV=1902423651 TSER=261560659 > tcp.seq == 49 > 7 0.000100 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 7241 tcp.seq == 5793 > 8 0.000135 10.10.1.10 -> 10.10.1.20 SSH [TCP Previous segment lost] Encrypted request packet len=1448 tcp.nxtseq == 10137 > tcp.seq == 8689 > 9 0.000147 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 TSER=261560660 > SLE=8689 SRE=10137 tcp.seq == 49 > 10 0.000167 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 11585 tcp.seq == 10137 > 11 0.000173 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#1] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=11585 tcp.seq == 49 > 12 0.000180 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 13033 tcp.seq == 11585 > 13 0.000186 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#2] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=13033 tcp.seq == 49 > 14 0.000191 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 14481 tcp.seq == 13033 > 15 0.000197 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#3] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=14481 tcp.seq == 49 > 16 0.000201 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 15929 tcp.seq == 14481 > 17 0.000209 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#4] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=15929 tcp.seq == 49 > 18 0.000229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 17377 tcp.seq == 15929 > 19 0.000237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#5] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=17377 tcp.seq == 49 > 20 0.000460 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 18825 tcp.seq == 17377 > 21 0.000469 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#6] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=18825 tcp.seq == 49 > 22 0.000505 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 20273 tcp.seq == 18825 > 23 0.000511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#7] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=20273 tcp.seq == 49 > 24 0.000513 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 21721 tcp.seq == 20273 > 25 0.000519 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#8] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=21721 tcp.seq == 49 > 26 0.000528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 23169 tcp.seq == 21721 > 27 0.000535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#9] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 28 0.000539 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 24617 tcp.seq == 23169 > 29 0.000546 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#10] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 30 0.000556 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 26065 tcp.seq == 24617 > 31 0.000562 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#11] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 32 0.000565 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 27513 tcp.seq == 26065 > 33 0.000572 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9011 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 34 0.000579 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 35 0.000596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 28961 tcp.seq == 27513 > 36 0.000604 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#1] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 37 0.000621 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 30409 tcp.seq == 28961 > 38 0.000627 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#2] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 39 0.000650 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 31857 tcp.seq == 30409 > 40 0.000657 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#3] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 41 0.000672 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 33305 tcp.seq == 31857 > 42 0.000679 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#4] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 43 0.000799 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 34753 tcp.seq == 33305 > 44 0.000806 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#5] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 45 0.000857 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 36201 tcp.seq == 34753 > 46 0.000866 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#6] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 47 0.000939 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 37649 tcp.seq == 36201 > 48 0.000946 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#7] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 49 0.000955 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 39097 tcp.seq == 37649 > 50 0.000963 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#8] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423652 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 51 0.001256 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 40545 tcp.seq == 39097 > 52 0.001268 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#9] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 53 0.001300 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 41993 tcp.seq == 40545 > 54 0.001309 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#10] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 55 0.001352 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 43441 tcp.seq == 41993 > 56 0.001359 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#11] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 57 0.001431 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 44889 tcp.seq == 43441 > 58 0.001439 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#12] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 59 0.001468 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 46337 tcp.seq == 44889 > 60 0.001475 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#13] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 61 0.001492 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 47785 tcp.seq == 46337 > 62 0.001498 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#14] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 63 0.001569 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 49233 tcp.seq == 47785 > 64 0.001576 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#15] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 65 0.001879 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 50681 tcp.seq == 49233 > 66 0.001890 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#16] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 67 0.001925 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 52129 tcp.seq == 50681 > 68 0.001933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#17] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 69 0.002006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 53577 tcp.seq == 52129 > 70 0.002016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#18] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423653 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 71 0.002218 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 55025 tcp.seq == 53577 > 72 0.002225 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#19] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 73 0.002281 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 56473 tcp.seq == 55025 > 74 0.002288 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#20] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 75 0.002343 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 57921 tcp.seq == 56473 > 76 0.002350 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#21] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 77 0.002525 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 59369 tcp.seq == 57921 > 78 0.002535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#22] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 79 0.002580 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 60817 tcp.seq == 59369 > 80 0.002589 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#23] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 81 0.002772 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 62265 tcp.seq == 60817 > 82 0.002779 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#24] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 83 0.002826 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 63713 tcp.seq == 62265 > 84 0.002834 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#25] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 85 0.002889 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 65161 tcp.seq == 63713 > 86 0.002896 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#26] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 87 0.002967 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 66609 tcp.seq == 65161 > 88 0.002974 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#27] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423654 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 89 0.003411 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 68057 tcp.seq == 66609 > 90 0.003419 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#28] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 91 0.003457 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 69505 tcp.seq == 68057 > 92 0.003464 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#29] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 93 0.003557 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 70953 tcp.seq == 69505 > 94 0.003564 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#30] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 95 0.003644 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 72401 tcp.seq == 70953 > 96 0.003653 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#31] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 97 0.003675 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 73849 tcp.seq == 72401 > 98 0.003683 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#32] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 99 0.003728 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 75297 tcp.seq == 73849 > 100 0.003735 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#33] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 101 0.003780 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 76745 tcp.seq == 75297 > 102 0.003792 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#34] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 103 0.003829 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 78193 tcp.seq == 76745 > 104 0.003842 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#35] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 105 0.003874 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 79641 tcp.seq == 78193 > 106 0.003883 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#36] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423655 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 107 0.004165 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 81089 tcp.seq == 79641 > 108 0.004172 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#37] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 109 0.004279 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 82537 tcp.seq == 81089 > 110 0.004287 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#38] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 111 0.004340 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 83985 tcp.seq == 82537 > 112 0.004347 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#39] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 113 0.004400 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 85433 tcp.seq == 83985 > 114 0.004408 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#40] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 115 0.004887 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 86881 tcp.seq == 85433 > 116 0.004894 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#41] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 117 0.004940 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 88329 tcp.seq == 86881 > 118 0.004947 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#42] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 119 0.005006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 89777 tcp.seq == 88329 > 120 0.005015 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#43] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423656 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 121 0.005140 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 91225 tcp.seq == 89777 > 122 0.005150 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#44] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 123 0.005194 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 92673 tcp.seq == 91225 > 124 0.005203 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#45] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 125 0.005263 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 94121 tcp.seq == 92673 > 126 0.005270 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#46] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 127 0.005413 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 95569 tcp.seq == 94121 > 128 0.005420 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#47] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 129 0.005480 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 97017 tcp.seq == 95569 > 130 0.005487 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#48] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 131 0.005504 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 98465 tcp.seq == 97017 > 132 0.005511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#49] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 133 0.005528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 99913 tcp.seq == 98465 > 134 0.005536 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#50] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 135 0.005553 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 101361 tcp.seq == 99913 > 136 0.005561 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#51] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 137 0.005576 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 102809 tcp.seq == 101361 > 138 0.005583 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#52] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 139 0.005627 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 104257 tcp.seq == 102809 > 140 0.005634 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#53] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 141 0.005689 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 105705 tcp.seq == 104257 > 142 0.005696 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#54] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 143 0.005704 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 107153 tcp.seq == 105705 > 144 0.005710 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#55] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 145 0.005716 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 108601 tcp.seq == 107153 > 146 0.005724 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#56] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 147 0.005727 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 110049 tcp.seq == 108601 > 148 0.005734 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#57] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 149 0.005739 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 111497 tcp.seq == 110049 > 150 0.005746 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#58] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 151 0.005756 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 112945 tcp.seq == 111497 > 152 0.005765 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#59] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 153 0.005768 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 114393 tcp.seq == 112945 > 154 0.005777 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#60] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 155 0.005781 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 115841 tcp.seq == 114393 > 156 0.005789 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#61] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423657 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 157 0.006154 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 117289 tcp.seq == 115841 > 158 0.006162 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#62] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 159 0.006200 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 118737 tcp.seq == 117289 > 160 0.006207 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#63] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 161 0.006229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 120185 tcp.seq == 118737 > 162 0.006237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#64] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 163 0.006253 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 121633 tcp.seq == 120185 > 164 0.006259 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#65] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 165 0.006278 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 123081 tcp.seq == 121633 > 166 0.006285 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#66] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 167 0.006294 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 124529 tcp.seq == 123081 > 168 0.006300 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#67] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 169 0.006304 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 125977 tcp.seq == 124529 > 170 0.006310 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#68] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 171 0.006315 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 127425 tcp.seq == 125977 > 172 0.006323 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#69] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 173 0.006333 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 128873 tcp.seq == 127425 > 174 0.006340 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#70] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 175 0.006342 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 130321 tcp.seq == 128873 > 176 0.006349 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#71] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 177 0.006358 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 131769 tcp.seq == 130321 > 178 0.006365 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#72] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 179 0.006368 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 133217 tcp.seq == 131769 > 180 0.006375 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#73] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 181 0.006388 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 134665 tcp.seq == 133217 > 182 0.006395 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#74] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 183 0.006435 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 136113 tcp.seq == 134665 > 184 0.006446 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#75] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 185 0.006506 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 137561 tcp.seq == 136113 > 186 0.006516 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#76] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 187 0.006558 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 139009 tcp.seq == 137561 > 188 0.006565 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#77] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 189 0.006568 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 140457 tcp.seq == 139009 > 190 0.006574 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#78] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 191 0.006579 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 141905 tcp.seq == 140457 > 192 0.006585 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#79] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 193 0.006596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 143353 tcp.seq == 141905 > 194 0.006603 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#80] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 195 0.006606 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 144801 tcp.seq == 143353 > 196 0.006612 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#81] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 197 0.006654 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 146249 tcp.seq == 144801 > 198 0.006661 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#82] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 > 199 0.006696 10.10.1.10 -> 10.10.1.20 SSH [TCP Fast Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 > tcp.seq == 7241 > 200 0.006703 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#83] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423658 > TSER=261560666 SLE=8689 SRE=23169 tcp.seq == 49 > 201 0.246652 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 202 0.246664 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#84] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902423897 > TSER=261560896 SLE=8689 SRE=23169 tcp.seq == 49 > 203 0.518331 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 204 0.518343 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#85] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424168 > TSER=261561156 SLE=8689 SRE=23169 tcp.seq == 49 > 205 0.852522 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 206 0.852535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#86] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424501 > TSER=261561476 SLE=8689 SRE=23169 tcp.seq == 49 > 207 1.311920 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 208 1.311933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#87] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902424958 > TSER=261561916 SLE=8689 SRE=23169 tcp.seq == 49 > 209 2.022170 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 210 2.022184 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#88] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902425666 > TSER=261562596 SLE=8689 SRE=23169 tcp.seq == 49 > 211 3.233004 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 212 3.233016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#89] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902426872 > TSER=261563756 SLE=8689 SRE=23169 tcp.seq == 49 > 213 5.444340 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 214 5.444353 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#90] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902429074 > TSER=261565876 SLE=8689 SRE=23169 tcp.seq == 49 > 215 9.658815 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 216 9.658826 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#91] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902433272 > TSER=261569916 SLE=8689 SRE=23169 tcp.seq == 49 > 217 17.882735 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 218 17.882748 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#92] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902441463 > TSER=261577796 SLE=8689 SRE=23169 tcp.seq == 49 > 219 34.113339 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 220 34.113352 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#93] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902457629 > TSER=261593356 SLE=8689 SRE=23169 tcp.seq == 49 > 221 50.343593 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 222 50.343607 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#94] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902473795 > TSER=261608916 SLE=8689 SRE=23169 tcp.seq == 49 > 223 66.567449 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 tcp.nxtseq == 8689 tcp.seq > == > 7241 > 224 66.567461 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#95] ssh > 28296 [ACK] Seq=49 Ack=7241 Win=9283 Len=0 TSV=1902489954 > TSER=261624476 SLE=8689 SRE=23169 tcp.seq == 49 > 225 82.793601 10.10.1.10 -> 10.10.1.20 TCP 28296 > ssh [RST, ACK] Seq=146249 Ack=49 Win=4163 Len=0 TSV=261640036 > TSER=1902489954 > tcp.seq == 146249 > > Just to confirm both machines involved are running 8.2-RELEASE > with both hardware checksums and offloading disabled using > -tso -rxcsum -txcsum. > > The sender is using em and the receiver using igb. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 09:18:42 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48453106566C for ; Tue, 2 Aug 2011 09:18:42 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id BD9628FC15 for ; Tue, 2 Aug 2011 09:18:39 +0000 (UTC) Received: (qmail 99912 invoked from network); 2 Aug 2011 08:13:16 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Aug 2011 08:13:16 -0000 Message-ID: <4E37C0F2.4080004@freebsd.org> Date: Tue, 02 Aug 2011 11:18:42 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Steven Hartland References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk> In-Reply-To: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 09:18:42 -0000 On 02.08.2011 03:40, Steven Hartland wrote: > ----- Original Message ----- >> What I believe we're seeing from the tcpdump trace is the >> packet loss on the network results in an unrecoverable tcp >> session. >> >> From my very limited knowledge of tcp, I would expect to see >> the session to recover after receiving the lost packet by >> either a fast retransmit or standard retransmit. >> >> My interpretation of the capture is the packet is indeed resent >> by the sender but that the receiver continues to respond >> with TCP Dup ACK for said missing packet instead of recovering. >> >> Trace is below, which starts from just before the point >> the receiver sees the dropped packet to the point where the sender >> gives up and resets the session. >> >> Is my analysis correct and does it look like this is a tcp >> stack bug? > > Something I've just noticed that might be of note for this is that > the receiving machine seeing the stall seems to be increasing its > "discarded due to memory problems" count at a decent rate:- > > 127187 discarded due to memory problems > the 1 second later during a stall > 127192 discarded due to memory problems > > I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the > following comment:- > > * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat > * should work but causes packets to be dropped when they shouldn't. > > I notice this code is relatively new, so I'm wondering if this may be > something to do with what we're seeing, possibly still dropping packets > it shouldn't? > > @Lawrence apologies' for the direct mail, but I believe you where the > original author this particular change so wondered if you may be > able to shed any light on this? You could be onto something here. Please try this patch: http://people.freebsd.org/~andre/tcp_reass.c-logdebug-20110802.diff You can enable the log output with sysctl net.inet.tcp.log_debug=1 and report the log output (comes at LOG_DEBUG level). -- Andre >> >> tshark -z "proto,colinfo,tcp.seq,tcp.seq" -z "proto,colinfo,tcp.nxtseq,tcp.nxtseq" -r tcp-stall.pcap >> 1 0.000000 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 1449 >> tcp.seq == 1 >> 2 0.000028 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 2897 >> tcp.seq == 1449 >> 3 0.000034 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=2897 Win=9102 Len=0 >> TSV=1902423651 TSER=261560659 >> tcp.seq == 49 >> 4 0.000051 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 4345 >> tcp.seq == 2897 >> 5 0.000078 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 5793 >> tcp.seq == 4345 >> 6 0.000084 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=5793 Win=8921 Len=0 >> TSV=1902423651 TSER=261560659 >> tcp.seq == 49 >> 7 0.000100 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 7241 >> tcp.seq == 5793 >> 8 0.000135 10.10.1.10 -> 10.10.1.20 SSH [TCP Previous segment lost] Encrypted request packet >> len=1448 tcp.nxtseq == 10137 >> tcp.seq == 8689 >> 9 0.000147 10.10.1.20 -> 10.10.1.10 TCP ssh > 28296 [ACK] Seq=49 Ack=7241 Win=8830 Len=0 >> TSV=1902423652 TSER=261560660 >> SLE=8689 SRE=10137 tcp.seq == 49 >> 10 0.000167 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 11585 >> tcp.seq == 10137 >> 11 0.000173 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#1] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=11585 tcp.seq == 49 >> 12 0.000180 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 13033 >> tcp.seq == 11585 >> 13 0.000186 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#2] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=13033 tcp.seq == 49 >> 14 0.000191 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 14481 >> tcp.seq == 13033 >> 15 0.000197 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#3] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=14481 tcp.seq == 49 >> 16 0.000201 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 15929 >> tcp.seq == 14481 >> 17 0.000209 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#4] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=15929 tcp.seq == 49 >> 18 0.000229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 17377 >> tcp.seq == 15929 >> 19 0.000237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#5] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=17377 tcp.seq == 49 >> 20 0.000460 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 18825 >> tcp.seq == 17377 >> 21 0.000469 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#6] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=18825 tcp.seq == 49 >> 22 0.000505 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 20273 >> tcp.seq == 18825 >> 23 0.000511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#7] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=20273 tcp.seq == 49 >> 24 0.000513 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 21721 >> tcp.seq == 20273 >> 25 0.000519 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#8] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=21721 tcp.seq == 49 >> 26 0.000528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 23169 >> tcp.seq == 21721 >> 27 0.000535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#9] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 28 0.000539 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 24617 >> tcp.seq == 23169 >> 29 0.000546 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#10] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 30 0.000556 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 26065 >> tcp.seq == 24617 >> 31 0.000562 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 9#11] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=8830 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 32 0.000565 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 27513 >> tcp.seq == 26065 >> 33 0.000572 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9011 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 34 0.000579 10.10.1.20 -> 10.10.1.10 TCP [TCP Window Update] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 35 0.000596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 28961 >> tcp.seq == 27513 >> 36 0.000604 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#1] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 37 0.000621 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 30409 >> tcp.seq == 28961 >> 38 0.000627 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#2] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 39 0.000650 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 31857 >> tcp.seq == 30409 >> 40 0.000657 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#3] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 41 0.000672 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 33305 >> tcp.seq == 31857 >> 42 0.000679 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#4] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 43 0.000799 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 34753 >> tcp.seq == 33305 >> 44 0.000806 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#5] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 45 0.000857 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 36201 >> tcp.seq == 34753 >> 46 0.000866 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#6] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 47 0.000939 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 37649 >> tcp.seq == 36201 >> 48 0.000946 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#7] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 49 0.000955 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 39097 >> tcp.seq == 37649 >> 50 0.000963 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#8] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423652 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 51 0.001256 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 40545 >> tcp.seq == 39097 >> 52 0.001268 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#9] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 53 0.001300 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 41993 >> tcp.seq == 40545 >> 54 0.001309 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#10] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 55 0.001352 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 43441 >> tcp.seq == 41993 >> 56 0.001359 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#11] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 57 0.001431 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 44889 >> tcp.seq == 43441 >> 58 0.001439 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#12] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 59 0.001468 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 46337 >> tcp.seq == 44889 >> 60 0.001475 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#13] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 61 0.001492 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 47785 >> tcp.seq == 46337 >> 62 0.001498 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#14] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 63 0.001569 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 49233 >> tcp.seq == 47785 >> 64 0.001576 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#15] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 65 0.001879 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 50681 >> tcp.seq == 49233 >> 66 0.001890 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#16] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 67 0.001925 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 52129 >> tcp.seq == 50681 >> 68 0.001933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#17] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 69 0.002006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 53577 >> tcp.seq == 52129 >> 70 0.002016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#18] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423653 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 71 0.002218 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 55025 >> tcp.seq == 53577 >> 72 0.002225 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#19] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 73 0.002281 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 56473 >> tcp.seq == 55025 >> 74 0.002288 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#20] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 75 0.002343 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 57921 >> tcp.seq == 56473 >> 76 0.002350 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#21] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 77 0.002525 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 59369 >> tcp.seq == 57921 >> 78 0.002535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#22] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 79 0.002580 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 60817 >> tcp.seq == 59369 >> 80 0.002589 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#23] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 81 0.002772 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 62265 >> tcp.seq == 60817 >> 82 0.002779 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#24] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 83 0.002826 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 63713 >> tcp.seq == 62265 >> 84 0.002834 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#25] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 85 0.002889 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 65161 >> tcp.seq == 63713 >> 86 0.002896 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#26] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 87 0.002967 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 66609 >> tcp.seq == 65161 >> 88 0.002974 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#27] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423654 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 89 0.003411 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 68057 >> tcp.seq == 66609 >> 90 0.003419 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#28] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 91 0.003457 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 69505 >> tcp.seq == 68057 >> 92 0.003464 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#29] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 93 0.003557 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 70953 >> tcp.seq == 69505 >> 94 0.003564 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#30] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 95 0.003644 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 72401 >> tcp.seq == 70953 >> 96 0.003653 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#31] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 97 0.003675 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 73849 >> tcp.seq == 72401 >> 98 0.003683 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#32] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 99 0.003728 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 75297 >> tcp.seq == 73849 >> 100 0.003735 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#33] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 101 0.003780 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 76745 >> tcp.seq == 75297 >> 102 0.003792 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#34] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 103 0.003829 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 78193 >> tcp.seq == 76745 >> 104 0.003842 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#35] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 105 0.003874 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 79641 >> tcp.seq == 78193 >> 106 0.003883 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#36] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423655 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 107 0.004165 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 81089 >> tcp.seq == 79641 >> 108 0.004172 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#37] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 109 0.004279 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 82537 >> tcp.seq == 81089 >> 110 0.004287 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#38] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 111 0.004340 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 83985 >> tcp.seq == 82537 >> 112 0.004347 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#39] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 113 0.004400 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 85433 >> tcp.seq == 83985 >> 114 0.004408 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#40] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 115 0.004887 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 86881 >> tcp.seq == 85433 >> 116 0.004894 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#41] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 117 0.004940 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 88329 >> tcp.seq == 86881 >> 118 0.004947 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#42] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 119 0.005006 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 89777 >> tcp.seq == 88329 >> 120 0.005015 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#43] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423656 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 121 0.005140 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 91225 >> tcp.seq == 89777 >> 122 0.005150 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#44] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 123 0.005194 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 92673 >> tcp.seq == 91225 >> 124 0.005203 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#45] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 125 0.005263 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 94121 >> tcp.seq == 92673 >> 126 0.005270 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#46] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 127 0.005413 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 95569 >> tcp.seq == 94121 >> 128 0.005420 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#47] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 129 0.005480 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 97017 >> tcp.seq == 95569 >> 130 0.005487 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#48] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 131 0.005504 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 98465 >> tcp.seq == 97017 >> 132 0.005511 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#49] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 133 0.005528 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 99913 >> tcp.seq == 98465 >> 134 0.005536 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#50] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 135 0.005553 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 101361 >> tcp.seq == 99913 >> 136 0.005561 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#51] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 137 0.005576 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 102809 >> tcp.seq == 101361 >> 138 0.005583 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#52] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 139 0.005627 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 104257 >> tcp.seq == 102809 >> 140 0.005634 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#53] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 141 0.005689 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 105705 >> tcp.seq == 104257 >> 142 0.005696 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#54] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 143 0.005704 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 107153 >> tcp.seq == 105705 >> 144 0.005710 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#55] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 145 0.005716 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 108601 >> tcp.seq == 107153 >> 146 0.005724 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#56] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 147 0.005727 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 110049 >> tcp.seq == 108601 >> 148 0.005734 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#57] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 149 0.005739 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 111497 >> tcp.seq == 110049 >> 150 0.005746 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#58] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 151 0.005756 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 112945 >> tcp.seq == 111497 >> 152 0.005765 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#59] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 153 0.005768 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 114393 >> tcp.seq == 112945 >> 154 0.005777 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#60] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 155 0.005781 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 115841 >> tcp.seq == 114393 >> 156 0.005789 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#61] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423657 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 157 0.006154 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 117289 >> tcp.seq == 115841 >> 158 0.006162 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#62] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 159 0.006200 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 118737 >> tcp.seq == 117289 >> 160 0.006207 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#63] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 161 0.006229 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 120185 >> tcp.seq == 118737 >> 162 0.006237 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#64] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 163 0.006253 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 121633 >> tcp.seq == 120185 >> 164 0.006259 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#65] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 165 0.006278 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 123081 >> tcp.seq == 121633 >> 166 0.006285 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#66] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 167 0.006294 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 124529 >> tcp.seq == 123081 >> 168 0.006300 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#67] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 169 0.006304 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 125977 >> tcp.seq == 124529 >> 170 0.006310 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#68] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 171 0.006315 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 127425 >> tcp.seq == 125977 >> 172 0.006323 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#69] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 173 0.006333 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 128873 >> tcp.seq == 127425 >> 174 0.006340 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#70] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 175 0.006342 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 130321 >> tcp.seq == 128873 >> 176 0.006349 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#71] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 177 0.006358 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 131769 >> tcp.seq == 130321 >> 178 0.006365 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#72] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 179 0.006368 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 133217 >> tcp.seq == 131769 >> 180 0.006375 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#73] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 181 0.006388 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 134665 >> tcp.seq == 133217 >> 182 0.006395 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#74] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 183 0.006435 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 136113 >> tcp.seq == 134665 >> 184 0.006446 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#75] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 185 0.006506 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 137561 >> tcp.seq == 136113 >> 186 0.006516 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#76] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 187 0.006558 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 139009 >> tcp.seq == 137561 >> 188 0.006565 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#77] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 189 0.006568 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 140457 >> tcp.seq == 139009 >> 190 0.006574 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#78] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 191 0.006579 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 141905 >> tcp.seq == 140457 >> 192 0.006585 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#79] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 193 0.006596 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 143353 >> tcp.seq == 141905 >> 194 0.006603 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#80] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 195 0.006606 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 144801 >> tcp.seq == 143353 >> 196 0.006612 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#81] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 197 0.006654 10.10.1.10 -> 10.10.1.20 SSH Encrypted request packet len=1448 tcp.nxtseq == 146249 >> tcp.seq == 144801 >> 198 0.006661 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#82] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560660 SLE=8689 SRE=23169 tcp.seq == 49 >> 199 0.006696 10.10.1.10 -> 10.10.1.20 SSH [TCP Fast Retransmission] Encrypted request packet >> len=1448 tcp.nxtseq == 8689 >> tcp.seq == 7241 >> 200 0.006703 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#83] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423658 >> TSER=261560666 SLE=8689 SRE=23169 tcp.seq == 49 >> 201 0.246652 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 202 0.246664 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#84] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902423897 >> TSER=261560896 SLE=8689 SRE=23169 tcp.seq == 49 >> 203 0.518331 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 204 0.518343 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#85] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902424168 >> TSER=261561156 SLE=8689 SRE=23169 tcp.seq == 49 >> 205 0.852522 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 206 0.852535 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#86] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902424501 >> TSER=261561476 SLE=8689 SRE=23169 tcp.seq == 49 >> 207 1.311920 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 208 1.311933 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#87] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902424958 >> TSER=261561916 SLE=8689 SRE=23169 tcp.seq == 49 >> 209 2.022170 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 210 2.022184 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#88] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902425666 >> TSER=261562596 SLE=8689 SRE=23169 tcp.seq == 49 >> 211 3.233004 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 212 3.233016 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#89] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902426872 >> TSER=261563756 SLE=8689 SRE=23169 tcp.seq == 49 >> 213 5.444340 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 214 5.444353 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#90] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902429074 >> TSER=261565876 SLE=8689 SRE=23169 tcp.seq == 49 >> 215 9.658815 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 216 9.658826 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#91] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902433272 >> TSER=261569916 SLE=8689 SRE=23169 tcp.seq == 49 >> 217 17.882735 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 218 17.882748 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#92] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902441463 >> TSER=261577796 SLE=8689 SRE=23169 tcp.seq == 49 >> 219 34.113339 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 220 34.113352 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#93] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902457629 >> TSER=261593356 SLE=8689 SRE=23169 tcp.seq == 49 >> 221 50.343593 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 222 50.343607 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#94] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902473795 >> TSER=261608916 SLE=8689 SRE=23169 tcp.seq == 49 >> 223 66.567449 10.10.1.10 -> 10.10.1.20 SSH [TCP Retransmission] Encrypted request packet len=1448 >> tcp.nxtseq == 8689 tcp.seq == >> 7241 >> 224 66.567461 10.10.1.20 -> 10.10.1.10 TCP [TCP Dup ACK 34#95] ssh > 28296 [ACK] Seq=49 Ack=7241 >> Win=9283 Len=0 TSV=1902489954 >> TSER=261624476 SLE=8689 SRE=23169 tcp.seq == 49 >> 225 82.793601 10.10.1.10 -> 10.10.1.20 TCP 28296 > ssh [RST, ACK] Seq=146249 Ack=49 Win=4163 Len=0 >> TSV=261640036 TSER=1902489954 >> tcp.seq == 146249 >> >> Just to confirm both machines involved are running 8.2-RELEASE >> with both hardware checksums and offloading disabled using >> -tso -rxcsum -txcsum. >> >> The sender is using em and the receiver using igb. > > > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom > it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, > printing or otherwise disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 10:22:40 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 275B9106568B; Tue, 2 Aug 2011 10:22:40 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 635E98FC13; Tue, 2 Aug 2011 10:22:38 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 11:22:06 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 11:22:06 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014432344.msg; Tue, 02 Aug 2011 11:22:06 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> From: "Steven Hartland" To: "Andre Oppermann" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk> <4E37C0F2.4080004@freebsd.org> Date: Tue, 2 Aug 2011 11:22:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 10:22:40 -0000 ----- Original Message ----- From: "Andre Oppermann" ... >> I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the >> following comment:- >> >> * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat >> * should work but causes packets to be dropped when they shouldn't. >> >> I notice this code is relatively new, so I'm wondering if this may be >> something to do with what we're seeing, possibly still dropping packets >> it shouldn't? >> >> @Lawrence apologies' for the direct mail, but I believe you where the >> original author this particular change so wondered if you may be >> able to shed any light on this? > > You could be onto something here. Please try this patch: > http://people.freebsd.org/~andre/tcp_reass.c-logdebug-20110802.diff > > You can enable the log output with > sysctl net.inet.tcp.log_debug=1 > and report the log output (comes at LOG_DEBUG level). Thanks for the response Andre, I've applied the patch and I'm seeing lots of the following during the test which is:- 1. scp from local host (10.10.1.30) -> tcptest (10.10.1.20) reciever which gets ~ 64MB/s 2. scp from remote host (10.10.1.10) -> tcptest (10.10.1.20) reciever which gets ~ 10MB/s (line has packet loss) Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped, would have drained queue! Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped, would have drained queue! Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.10]:32416 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped, would have drained queue! Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped, would have drained queue! Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit reached, segment dropped Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 11:35:55 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87E481065686; Tue, 2 Aug 2011 11:35:55 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 6954E8FC14; Tue, 2 Aug 2011 11:35:54 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 12:35:21 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 12:35:21 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014433252.msg; Tue, 02 Aug 2011 12:35:20 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Andre Oppermann" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> Date: Tue, 2 Aug 2011 12:35:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 11:35:55 -0000 ----- Original Message ----- From: "Steven Hartland" > ----- Original Message ----- > From: "Andre Oppermann" > ... >>> I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the >>> following comment:- >>> >>> * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat >>> * should work but causes packets to be dropped when they shouldn't. >>> >>> I notice this code is relatively new, so I'm wondering if this may be >>> something to do with what we're seeing, possibly still dropping packets >>> it shouldn't? >>> >>> @Lawrence apologies' for the direct mail, but I believe you where the >>> original author this particular change so wondered if you may be >>> able to shed any light on this? >> >> You could be onto something here. Please try this patch: >> http://people.freebsd.org/~andre/tcp_reass.c-logdebug-20110802.diff >> >> You can enable the log output with >> sysctl net.inet.tcp.log_debug=1 >> and report the log output (comes at LOG_DEBUG level). > > Thanks for the response Andre, I've applied the patch and I'm seeing > lots of the following during the test which is:- > 1. scp from local host (10.10.1.30) -> tcptest (10.10.1.20) reciever > which gets ~ 64MB/s > 2. scp from remote host (10.10.1.10) -> tcptest (10.10.1.20) reciever > which gets ~ 10MB/s (line has packet loss) > > Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit > reached, segment dropped > Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit > reached, segment dropped Hmm, based on this are we seeing something similar to the following? http://www.freebsd.org/cgi/query-pr.cgi?pr=155407 Other potentially useful info:- vmstat -z | head -1 ; vmstat -z | grep -i tcp ITEM SIZE LIMIT USED FREE REQUESTS FAILURES tcp_inpcb: 336, 25608, 115, 556, 707, 0 tcpcb: 880, 25600, 115, 405, 707, 0 tcptw: 72, 5150, 0, 600, 188, 0 tcpreass: 40, 1680, 106, 1574, 185926, 4414 sysctl net.inet.tcp.reass net.inet.tcp.reass.overflows: 0 net.inet.tcp.reass.cursegments: 106 net.inet.tcp.reass.maxsegments: 1680 netstat -s -f inet -p tcp | grep "discarded due" 4414 discarded due to memory problems net.inet.tcp.reass.maxsegments: 1680 sysctl kern.ipc.nmbclusters kern.ipc.nmbclusters: 25600 The default value of nmbclusters on the target machine explains the value of net.inet.tcp.reass.maxsegments which defaults to nmbclusters / 16 Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the tests appears to result in a solid 14MB/s, its still running a full soak test but looking very promising :) So I suppose the question is should maxsegments be larger by default due to the recent changes e.g. - V_tcp_reass_maxseg = nmbclusters / 16; + V_tcp_reass_maxseg = nmbclusters / 8; or is the correct fix something more involved? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 12:33:05 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68606106564A; Tue, 2 Aug 2011 12:33:05 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 91DDB8FC08; Tue, 2 Aug 2011 12:33:04 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 13:32:32 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 13:32:32 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014433717.msg; Tue, 02 Aug 2011 13:32:31 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Andre Oppermann" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> Date: Tue, 2 Aug 2011 13:32:53 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:33:05 -0000 ----- Original Message ----- From: "Steven Hartland" > Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the > tests appears to result in a solid 14MB/s, its still running a > full soak test but looking very promising :) Ok so full test has completed with over 134GB of data transferred without a hitch. I haven't been able to complete this test without error on this target machine before the change to maxsegments; so packets being dropped in the tcp reassembly code due to tcp_reass global zone exhaustion is almost certainly the cause of the stalls we're seeing, which is good news :) I suspect there are a few contributing factors at play causing this. 1. There is known to be a dodgy fibre on the test routing, which is scheduled for cleaning, but is currently causing a small amount of packet loss between the sender and receiver. 2. The target machine has 24 cores and is running 8 queues on the igb interface which could result in the requirement to reorder packets even if they arrived in order on the wire. I say this as disabling msix on igb0, which results in just one queue, did reduce the occurrence rate of the problem. 3. Combining a low latency (<1ms) high throughput connection ~64MB/s with a lower throughput but still relatively high bandwidth ~14MB/s with a 10ms latency. I look forward to hearing peoples thoughts on what the actual fix should be: increased default nmbclusters, decreased nmbclusters => maxsegments divisor, or something else? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 12:56:49 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0D4C1065674; Tue, 2 Aug 2011 12:56:49 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 099518FC28; Tue, 2 Aug 2011 12:56:48 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 13:56:16 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 13:56:16 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014433957.msg; Tue, 02 Aug 2011 13:56:15 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Andre Oppermann" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org><2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> Date: Tue, 2 Aug 2011 13:56:48 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:56:50 -0000 ----- Original Message ----- From: "Steven Hartland" > I look forward to hearing peoples thoughts on what the actual fix > should be: increased default nmbclusters, decreased nmbclusters => > maxsegments divisor, or something else? Another useful piece of information is that even though maxsegments is set as a read only tuneable, it can be changed on the fly by changing kern.ipc.nmbclusters, which seems a bit strange tbh, so possibly an oversight there somewhere. Example:- sysctl net.inet.tcp.reass.maxsegments=16464 sysctl: oid 'net.inet.tcp.reass.maxsegments' is a read only tunable sysctl: Tunable values are set in /boot/loader.conf sysctl net.inet.tcp.reass.maxsegments net.inet.tcp.reass.maxsegments: 1680 sysctl kern.ipc.nmbclusters=262144 kern.ipc.nmbclusters: 25600 -> 262144 sysctl net.inet.tcp.reass.maxsegments net.inet.tcp.reass.maxsegments: 16464 Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 13:36:35 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79E091065675 for ; Tue, 2 Aug 2011 13:36:35 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 004E68FC14 for ; Tue, 2 Aug 2011 13:36:34 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 14:36:02 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 02 Aug 2011 14:36:02 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014434374.msg for ; Tue, 02 Aug 2011 14:36:02 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-net@freebsd.org Message-ID: <8F1656993CD14870B678FCF7CF71C113@multiplay.co.uk> From: "Steven Hartland" To: "Jack Vogel" References: <379885BA631F4C7787C24E00A174B429@multiplay.co.uk> <00F86D895BEA493E8FB82F0CDF42AA04@multiplay.co.uk> Date: Tue, 2 Aug 2011 14:36:35 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, Kevin Oberman Subject: Re: igb enable_aim or flow_control causing tcp stalls? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 13:36:35 -0000 Wanted to update this thread to confirm, as Jack suspected, the issue we where seeing was something much more ingrained that a driver issue. It turned out this problem in the tcp reassembly code which is likely due to insufficient size of net.inet.tcp.reass.maxsegments. The result is even with just 2 decent sized tcp streams a single lost packet can cause an unrecoverable situation resulting in the session stalling and eventually ~60 seconds later being reset by the sender. Increasing nmbclusters which in turn increases maxsegments eliminated the problem e.g. sysctl kern.ipc.nmbclusters=262144 kern.ipc.nmbclusters: 25600 -> 262144 For more information see the thread:- tcp failing to recover from a packet loss under 8.2-RELEASE? http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029491.html Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 21:25:06 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA6811065676 for ; Tue, 2 Aug 2011 21:25:06 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 184B68FC0A for ; Tue, 2 Aug 2011 21:25:05 +0000 (UTC) Received: (qmail 3203 invoked from network); 2 Aug 2011 20:19:37 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Aug 2011 20:19:37 -0000 Message-ID: <4E386B35.20303@freebsd.org> Date: Tue, 02 Aug 2011 23:25:09 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Steven Hartland References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 21:25:06 -0000 On 02.08.2011 14:32, Steven Hartland wrote: > ----- Original Message ----- From: "Steven Hartland" >> Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the >> tests appears to result in a solid 14MB/s, its still running a >> full soak test but looking very promising :) > > Ok so full test has completed with over 134GB of data transferred > without a hitch. > > I haven't been able to complete this test without error on this > target machine before the change to maxsegments; so packets being > dropped in the tcp reassembly code due to tcp_reass global zone > exhaustion is almost certainly the cause of the stalls we're > seeing, which is good news :) The zone exhaustion is part of the problem but not the real cause of the stall you're seeing. When the reassembly zone is exhausted the retransmit that would fill the hole between the socket buffer and the out-of-order segments in the reassembly queue can't be processed anymore. This brings all TCP sessions with data in the reassembly queue to a standstill. > I suspect there are a few contributing factors at play causing > this. > > 1. There is known to be a dodgy fibre on the test routing, which > is scheduled for cleaning, but is currently causing a small amount > of packet loss between the sender and receiver. Packet loss and the use of the reassembly queue. > 2. The target machine has 24 cores and is running 8 queues on > the igb interface which could result in the requirement to reorder > packets even if they arrived in order on the wire. I say this as > disabling msix on igb0, which results in just one queue, did reduce > the occurrence rate of the problem. Maybe this is a compounding factor. > 3. Combining a low latency (<1ms) high throughput connection > ~64MB/s with a lower throughput but still relatively high bandwidth > ~14MB/s with a 10ms latency. What matters here is the socket buffer and receive window size. > I look forward to hearing peoples thoughts on what the actual fix > should be: increased default nmbclusters, decreased nmbclusters => > maxsegments divisor, or something else? These items may or may not need some adjustments. Though it will eventually wedge again. The reassembly queue must be able to process the one missing segment despite having exhausted the global zone limit. I had fixed that at one point in time. It seems that it got lost with the recent changes. Please try this patch: http://people.freebsd.org/~andre/tcp_reass.c-logdebug+missingsegment-20110802.diff Running it with normal amount of nmbclusters and it should still prevent the sessions from stalling. -- Andre From owner-freebsd-net@FreeBSD.ORG Tue Aug 2 23:57:32 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98E1B1065674; Tue, 2 Aug 2011 23:57:32 +0000 (UTC) (envelope-from prvs=119502d6cc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id C40458FC08; Tue, 2 Aug 2011 23:57:31 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Wed, 03 Aug 2011 00:56:59 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 03 Aug 2011 00:56:59 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014442262.msg; Wed, 03 Aug 2011 00:56:58 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=119502d6cc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <3E54CF756D4845298C690C570D80C032@multiplay.co.uk> From: "Steven Hartland" To: "Andre Oppermann" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> <4E386B35.20303@freebsd.org> Date: Wed, 3 Aug 2011 00:57:34 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 23:57:32 -0000 ----- Original Message ----- From: "Andre Oppermann" To: "Steven Hartland" Cc: ; Sent: Tuesday, August 02, 2011 10:25 PM Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? > On 02.08.2011 14:32, Steven Hartland wrote: >> ----- Original Message ----- From: "Steven Hartland" >>> Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the >>> tests appears to result in a solid 14MB/s, its still running a >>> full soak test but looking very promising :) >> >> Ok so full test has completed with over 134GB of data transferred >> without a hitch. >> >> I haven't been able to complete this test without error on this >> target machine before the change to maxsegments; so packets being >> dropped in the tcp reassembly code due to tcp_reass global zone >> exhaustion is almost certainly the cause of the stalls we're >> seeing, which is good news :) > > The zone exhaustion is part of the problem but not the real cause > of the stall you're seeing. When the reassembly zone is exhausted > the retransmit that would fill the hole between the socket buffer > and the out-of-order segments in the reassembly queue can't be > processed anymore. This brings all TCP sessions with data in the > reassembly queue to a standstill. > >> I suspect there are a few contributing factors at play causing >> this. >> >> 1. There is known to be a dodgy fibre on the test routing, which >> is scheduled for cleaning, but is currently causing a small amount >> of packet loss between the sender and receiver. > > Packet loss and the use of the reassembly queue. > >> 2. The target machine has 24 cores and is running 8 queues on >> the igb interface which could result in the requirement to reorder >> packets even if they arrived in order on the wire. I say this as >> disabling msix on igb0, which results in just one queue, did reduce >> the occurrence rate of the problem. > > Maybe this is a compounding factor. > >> 3. Combining a low latency (<1ms) high throughput connection >> ~64MB/s with a lower throughput but still relatively high bandwidth >> ~14MB/s with a 10ms latency. > > What matters here is the socket buffer and receive window size. On the reciever the only none default sysclt's are:- vfs.read_max=32 net.inet.tcp.inflight.enable=0 net.inet.tcp.sendspace=65536 kern.ipc.maxsockbuf=524288 kern.maxfiles=50000 >> I look forward to hearing peoples thoughts on what the actual fix >> should be: increased default nmbclusters, decreased nmbclusters => >> maxsegments divisor, or something else? > > These items may or may not need some adjustments. Though it will > eventually wedge again. The reassembly queue must be able to process > the one missing segment despite having exhausted the global zone limit. > I had fixed that at one point in time. It seems that it got lost with > the recent changes. > > Please try this patch: > http://people.freebsd.org/~andre/tcp_reass.c-logdebug+missingsegment-20110802.diff Would it not make sense to always use stack for the missing segment, as this would avoid the allocation e.g. http://www.multiplaygameservers.com/dropzone/tcp_reass.c-logdebug+missingsegment-20110803.diff Also added a two additional cases to protect against using uma_zfree on the stack based tqs, not 100% clear if these could be triggered or not, but thought I'd check? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Wed Aug 3 10:53:17 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18D6F1065674 for ; Wed, 3 Aug 2011 10:53:17 +0000 (UTC) (envelope-from zeus@relay.ibs.dn.ua) Received: from relay.ibs.dn.ua (relay.ibs.dn.ua [91.216.196.25]) by mx1.freebsd.org (Postfix) with ESMTP id 504B18FC17 for ; Wed, 3 Aug 2011 10:53:15 +0000 (UTC) Received: from relay.ibs.dn.ua (localhost [127.0.0.1]) by relay.ibs.dn.ua with ESMTP id p73AXYQe003225 for ; Wed, 3 Aug 2011 13:33:34 +0300 (EEST) Received: (from zeus@localhost) by relay.ibs.dn.ua (8.14.4/8.14.4/Submit) id p73AXXr0003224 for freebsd-net@freebsd.org; Wed, 3 Aug 2011 13:33:33 +0300 (EEST) Date: Wed, 3 Aug 2011 13:33:33 +0300 From: Zeus V Panchenko To: freebsd-net@freebsd.org Message-ID: <20110803103332.GA98303@relay.ibs.dn.ua> Mail-Followup-To: freebsd-net@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 8.1-RELEASE X-Editor: GNU Emacs 23.2.1 X-Face: iVBORw0KGgoAAAANSUhEUgAAACoAAAAqBAMAAAA37dRoAAAAFVBMVEWjjoiZhHDWzcZuW1U wOT+RcGxziJxEN0lIAAABrklEQVQokV2STXLbMAyFQaraE3a5dzSTfR1IF7CQrM3QuECn9z9DH0 gxzgSyFvr88PBD0uJxoR6BE+e8LtRgohE5ZB50sODP/REbfUnte/z12+llCekLUSKenFIMke6Be WinE8H0RJHSN71rUQp64gFDmtDDhRk0zam3FzpNVFprhwPGaFo6oY9wDBJQ9Qz6EuKyROJjDGa+ uza4VOTa8iHlN58Yv5BF9+4BGl0LA5pUD5xKXg4aQlVZm0co3NKxCGxQpu3aC352Gv3DZONmwQd tkrlaylV3YSew7bWtwAZF/zi9jblmprPoL7ktzeFSxmarVNmWRi+Bmxg7Y7tbGtR8XZUxLTo86G thANsssetjp3POuBvMBRlw6jRa5pKN7yVlP+F2lyiZGSMf5hnSU6eAVupmtfjRcxy0momwpxDnz 06hwnOWvBnUdR8U2/KX7cq26u1Jy5xFZMPOVONRbRUrwey8Qar6cWgf12xSymQuVX0DfYd4R8kN Hg0qCtLeaYZcj8B90M2N0cEX1P0vKSxw7NLy/3X8Qeriusu66jNA37P4Mn5QRTG2hz4d9D/6E3a EX852nwAAAABJRU5ErkJggg== Subject: weird results while ipsec + ipfv_nat (nat before vpn) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: zeus@ibs.dn.ua List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 10:53:17 -0000 Hi, i faced weird for me situation, may somebody agree to help to win it, please? we need to see some http/s resources behind the Cisco PIX IPSEC i'm trying to get working this schema: SCHEMA (`nat before vpn' as i believe): -------------- +-> a.a.0.1/16 LAN | +-> a.a.a.2/24 FreeBSD b.b.b.1 <-> c.c.c.1/24 IPSEC PEER PIX | | + x.x.x.x <-------> y.y.y.y + CONFIGURATION: -------------- > uname -a FreeBSD 8.2-STABLE #3: Tue Aug 2 15:39:33 EEST 2011 i386 > cat /etc/rc.conf ... gateway_enable="YES" cloned_interfaces="gif0" ifconfig_bge0="inet x.x.x.x/25" ifconfig_bge1="inet a.a.a.2/24" ifconfig_gif0="inet b.b.b.1 c.c.c.1 tunnel x.x.x.x y.y.y.y" ipsec_enable="YES" ipsec_program="/usr/local/sbin/setkey" ipsec_file="/usr/local/etc/racoon/setkey.conf" racoon_enable="YES" ipfw_enable="YES" ipfw_nat_enable="YES" ... in kernel i have: options IPSEC options IPSEC_DEBUG device crypto options IPFIREWALL options IPFIREWALL_FORWARD options IPFIREWALL_NAT options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=200 options IPDIVERT options LIBALIAS > cat /usr/local/etc/racoon/setkey.conf flush; spdflush; spdadd b.b.b.1 c.c.c.0/24 any -P out ipsec \ esp/tunnel/x.x.x.x-y.y.y.y/require; spdadd c.c.c.0/24 b.b.b.1 any -P in ipsec \ esp/tunnel/y.y.y.y-x.x.x.x/require; > cat /etc/ipfw.conf ... add 000401 allow udp from x.x.x.x to y.y.y.y isakmp add 000402 allow udp from y.y.y.y to x.x.x.x isakmp add 000403 allow { esp or ipencap } from x.x.x.x to y.y.y.y add 000404 allow { esp or ipencap } from y.y.y.y to x.x.x.x add 00502 nat 100 all from { a.a.1.0/24 or a.a.2.0/24 } to c.c.c.0/24 nat 100 config log if bge1 ip b.b.b.1 reverse WHAT I DO: -------------- 1) trying to ping IPSEC PEER from LAN user@a.a.a.20> ping c.c.c.1 c.c.c.1 reply packets are coming in and are decrypted but replies doesn't reach ping initiator a.a.a.20 box a.a.a.20 reports ping statistics: 450 packets transmitted, 0 packets received, 100.0% packet loss at FreeBSD box i see: user@FreeBSD> tcpdump -n -i gif0 host c.c.c.1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on gif0, link-type NULL (BSD loopback), capture size 96 bytes ... 13:27:18.122542 IP c.c.c.1 > b.b.b.1: ICMP echo request, id 39050, seq 2903, length 64 13:27:19.123275 IP c.c.c.1 > b.b.b.1: ICMP echo request, id 39050, seq 2904, length 64 13:27:20.124517 IP c.c.c.1 > b.b.b.1: ICMP echo request, id 39050, seq 2905, length 64 13:27:21.125568 IP c.c.c.1 > b.b.b.1: ICMP echo request, id 39050, seq 2906, length 64 on WAN i see this user@FreeBSD> tcpdump -n -i bge0 esp ... 00:00:00.635862 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x7), length 132 00:00:00.024467 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x7), length 132 00:00:00.635567 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x8), length 132 00:00:00.024689 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x8), length 132 00:00:00.636724 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x9), length 132 00:00:00.024286 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x9), length 132 so, ipsec and ipfw_nat out works, but where are reply packets disappearing to after coming to gif0 interface? why no backward divert occures? 2) trying to ping IPSEC PEER from FreeBSD box user@b.b.b.1> ping c.c.c.1 everything works since no nat occures ... user@b.b.b.1> tcpdump -n -i gif0 host c.c.c.1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on gif0, link-type NULL (BSD loopback), capture size 96 bytes 13:45:56.759567 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 213, length 64 13:45:57.760745 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 214, length 64 13:45:58.762787 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 215, length 64 13:45:59.765493 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 216, length 64 13:46:00.764619 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 217, length 64 13:46:01.765676 IP c.c.c.1 > b.b.b.1: ICMP echo reply, id 53484, seq 218, length 64 user@b.b.b.1> tcpdump -n -ettt -s0 -i bge0 host y.y.y.y tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vlan11, link-type EN10MB (Ethernet), capture size 65535 bytes 00:00:00.635862 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x7), length 132 00:00:00.024467 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x7), length 132 00:00:00.635567 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x8), length 132 00:00:00.024689 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x8), length 132 00:00:00.636724 ethertype IPv4 (0x0800), length 166: x.x.x.x > y.y.y.y: ESP(spi=0xad597f86,seq=0x9), length 132 00:00:00.024286 ethertype IPv4 (0x0800), length 166: y.y.y.y > x.x.x.x: ESP(spi=0x060bc3e3,seq=0x9), length 132 so, is it possible to get it working? if yes, where is my mistake, please? -- Zeus V. Panchenko JID:zeus@gnu.org.ua GMT+2 (EET) From owner-freebsd-net@FreeBSD.ORG Wed Aug 3 12:21:07 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BE561065672 for ; Wed, 3 Aug 2011 12:21:07 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 465658FC08 for ; Wed, 3 Aug 2011 12:21:07 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 6584C25D3870 for ; Wed, 3 Aug 2011 12:21:06 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 9132CBD3D90 for ; Wed, 3 Aug 2011 12:21:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 0Im4D-dkwcVJ for ; Wed, 3 Aug 2011 12:21:04 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 7E1BDBD3BC1 for ; Wed, 3 Aug 2011 12:21:04 +0000 (UTC) Date: Wed, 3 Aug 2011 12:21:03 +0000 (UTC) From: "Bjoern A. Zeeb" To: freebsd-net@freebsd.org Message-ID: X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: FreeBSD 9.0-BETA1 IPv6-only snapshots available X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2011 12:21:07 -0000 Hi, I have built IPv6-only snapshots for 9.0-BETA1 for everyone interested. I am planning to selectively do further BETAs, and RCs as well if possible. You can find more information at http://www.freebsd.org/ipv6/ and http://wiki.freebsd.org/IPv6Only for a list of mirrors, etc. To do a network install you can use http:///ipv6only-snapshot-9.0-BETA1/(amd64|i386)/ftp/ with your closest mirror. For example: http://people.freebsd.org/~bz/ipv6only/ipv6only-snapshot-9.0-BETA1/amd64/ftp/ In case you have questions or suggestions, please let me know. Bjoern -- Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-net@FreeBSD.ORG Thu Aug 4 05:12:25 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 556DB106564A; Thu, 4 Aug 2011 05:12:25 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (unknown [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 9F73D8FC08; Thu, 4 Aug 2011 05:12:24 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p745CLLO076429; Thu, 4 Aug 2011 12:12:21 +0700 (NOVST) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E3A2A30.8000702@rdtc.ru> Date: Thu, 04 Aug 2011 12:12:16 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: In-Reply-To: Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org Subject: Re: FreeBSD 9.0-BETA1 IPv6-only snapshots available X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 05:12:25 -0000 03.08.2011 19:21, Bjoern A. Zeeb пишет: > Hi, > > I have built IPv6-only snapshots for 9.0-BETA1 for everyone interested. What does it mean, IPv6-only? Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Thu Aug 4 07:04:13 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35D28106566C for ; Thu, 4 Aug 2011 07:04:13 +0000 (UTC) (envelope-from bz@freebsd.org) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id E3C468FC18 for ; Thu, 4 Aug 2011 07:04:12 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 05BDB25D3888; Thu, 4 Aug 2011 07:04:11 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 324D7BD3C42; Thu, 4 Aug 2011 07:04:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 9Bx5k5jPMHpS; Thu, 4 Aug 2011 07:04:10 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 0D1B6BD3BC1; Thu, 4 Aug 2011 07:04:09 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=utf-8 From: "Bjoern A. Zeeb" In-Reply-To: <4E3A2A30.8000702@rdtc.ru> Date: Thu, 4 Aug 2011 07:04:08 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <29A8B8AB-367C-4A88-97EB-671D238CEC37@freebsd.org> References: <4E3A2A30.8000702@rdtc.ru> To: Eugene Grosbein X-Mailer: Apple Mail (2.1084) Cc: freebsd-net@freebsd.org Subject: Re: FreeBSD 9.0-BETA1 IPv6-only snapshots available X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 07:04:13 -0000 On Aug 4, 2011, at 5:12 AM, Eugene Grosbein wrote: > 03.08.2011 19:21, Bjoern A. Zeeb =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >> Hi, >>=20 >> I have built IPv6-only snapshots for 9.0-BETA1 for everyone = interested. >=20 > What does it mean, IPv6-only? No-INET would probably be a better description. Anyway see http://www.freebsd.org/ipv6/ipv6only.html --=20 Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-net@FreeBSD.ORG Thu Aug 4 14:18:02 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADCD1106566B; Thu, 4 Aug 2011 14:18:02 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 23DD68FC08; Thu, 4 Aug 2011 14:18:01 +0000 (UTC) Received: from lstewart-laptop.caia.swin.edu.au (124-171-48-43.dyn.iinet.net.au [124.171.48.43]) by lauren.room52.net (Postfix) with ESMTPSA id 0844B7E84F; Fri, 5 Aug 2011 00:02:26 +1000 (EST) Message-ID: <4E3AA66A.6060605@freebsd.org> Date: Fri, 05 Aug 2011 00:02:18 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110727 Thunderbird/5.0 MIME-Version: 1.0 To: Steven Hartland References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: freebsd-net@freebsd.org, Andre Oppermann , slw@zxy.spb.ru Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 14:18:02 -0000 Hi Steven, Andre and Slawa, Firstly, sorry for the delay diving into this. I just returned from some travel. Comments inline... On 08/02/11 21:35, Steven Hartland wrote: > > ----- Original Message ----- From: "Steven Hartland" >> ----- Original Message ----- From: "Andre Oppermann" >> ... >>>> I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the >>>> following comment:- >>>> >>>> * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat >>>> * should work but causes packets to be dropped when they shouldn't. This is not related to the issues you were experiencing. >>>> I notice this code is relatively new, so I'm wondering if this may be >>>> something to do with what we're seeing, possibly still dropping packets >>>> it shouldn't? >>>> >>>> @Lawrence apologies' for the direct mail, but I believe you where the >>>> original author this particular change so wondered if you may be >>>> able to shed any light on this? Thanks for bringing me in directly, I haven't been keeping up with the mailing lists at all recently. >>> You could be onto something here. Please try this patch: >>> http://people.freebsd.org/~andre/tcp_reass.c-logdebug-20110802.diff >>> >>> You can enable the log output with >>> sysctl net.inet.tcp.log_debug=1 >>> and report the log output (comes at LOG_DEBUG level). >> >> Thanks for the response Andre, I've applied the patch and I'm seeing >> lots of the following during the test which is:- >> 1. scp from local host (10.10.1.30) -> tcptest (10.10.1.20) reciever >> which gets ~ 64MB/s >> 2. scp from remote host (10.10.1.10) -> tcptest (10.10.1.20) reciever >> which gets ~ 10MB/s (line has packet loss) >> >> Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to >> [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit >> reached, segment dropped >> Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to >> [10.10.1.20]:22 tcpflags 0x10; tcp_reass: global zone limit >> reached, segment dropped > > Hmm, based on this are we seeing something similar to the following? > http://www.freebsd.org/cgi/query-pr.cgi?pr=155407 Slawa (CC'd) is the author of PR 155407 and Steven, your problem is the same. I grabbed the PR some time back but haven't found some time to sit down and respond in detail. > Other potentially useful info:- > > vmstat -z | head -1 ; vmstat -z | grep -i tcp > ITEM SIZE LIMIT USED FREE REQUESTS FAILURES > tcp_inpcb: 336, 25608, 115, 556, 707, 0 > tcpcb: 880, 25600, 115, 405, 707, 0 > tcptw: 72, 5150, 0, 600, 188, 0 > tcpreass: 40, 1680, 106, 1574, 185926, 4414 > > sysctl net.inet.tcp.reass > net.inet.tcp.reass.overflows: 0 > net.inet.tcp.reass.cursegments: 106 > net.inet.tcp.reass.maxsegments: 1680 > > netstat -s -f inet -p tcp | grep "discarded due" > 4414 discarded due to memory problems > net.inet.tcp.reass.maxsegments: 1680 > > sysctl kern.ipc.nmbclusters > kern.ipc.nmbclusters: 25600 > > The default value of nmbclusters on the target machine explains > the value of net.inet.tcp.reass.maxsegments which defaults to > nmbclusters / 16 > > Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the > tests appears to result in a solid 14MB/s, its still running a > full soak test but looking very promising :) This is exactly the necessary tuning required to drive high BDP links successfully. The unfortunate problem with my reassembly change was that by removing the global count of reassembly segments and using the uma zone to enforce the restrictions on memory use, we wouldn't necessarily have room for the last segment (particularly if a single flow has a BDP larger than the max size of the reassembly queue - which is the case for you and Slawa). This is bad as Andre explained in his message, as we could stall connections. I hadn't even considered the idea of allocating on the stack as Andre has suggested in his patch, which I believe is an appropriate solution to the the stalling problem assuming the function will never return with the stack allocated tqe still in the reassembly queue. My longer term goal is discussed below. > So I suppose the question is should maxsegments be larger by > default due to the recent changes e.g. > - V_tcp_reass_maxseg = nmbclusters / 16; > + V_tcp_reass_maxseg = nmbclusters / 8; > > or is the correct fix something more involved? I'm not sure if bumping the value is appropriate - we have always expected users to tune their network stack to perform well when used in "unusual" scenarios - a large BDP fibre path still being in the "unusual" category. The real fix which is somewhere down on my todo list is to make all these memory constraints elastic and respond to VM pressure, thus negating the need for a hard limit at all. This would solve many if not most of the TCP tuning problems we currently have with one foul swoop and would greatly reduce the need for tuning in many situations that currently are in the "needs manual tuning" basket. Andre and Steven, I'm a bit too sleepy to properly review your combined proposed changes right now and will follow up in the next few days instead. Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Thu Aug 4 14:31:20 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70817106564A; Thu, 4 Aug 2011 14:31:20 +0000 (UTC) (envelope-from prvs=11971cf4bd=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 87CEC8FC1E; Thu, 4 Aug 2011 14:31:19 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Thu, 04 Aug 2011 15:19:19 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 04 Aug 2011 15:19:19 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50014465425.msg; Thu, 04 Aug 2011 15:19:19 +0100 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=11971cf4bd=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <20229216858044E4881642284F245750@multiplay.co.uk> From: "Steven Hartland" To: "Lawrence Stewart" References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> <4E3AA66A.6060605@freebsd.org> Date: Thu, 4 Aug 2011 15:19:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Cc: freebsd-net@freebsd.org, Andre Oppermann , slw@zxy.spb.ru Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 14:31:20 -0000 ----- Original Message ----- From: "Lawrence Stewart" > Thanks for bringing me in directly, I haven't been keeping up with the > mailing lists at all recently. No problem >> So I suppose the question is should maxsegments be larger by >> default due to the recent changes e.g. >> - V_tcp_reass_maxseg = nmbclusters / 16; >> + V_tcp_reass_maxseg = nmbclusters / 8; >> >> or is the correct fix something more involved? > > I'm not sure if bumping the value is appropriate - we have always > expected users to tune their network stack to perform well when used in > "unusual" scenarios - a large BDP fibre path still being in the > "unusual" category. TBH I wouldn't classify a latency of 7ms @ 100Mbps unusal in the slightest in this day and age. > The real fix which is somewhere down on my todo list is to make all > these memory constraints elastic and respond to VM pressure, thus > negating the need for a hard limit at all. This would solve many if not > most of the TCP tuning problems we currently have with one foul swoop > and would greatly reduce the need for tuning in many situations that > currently are in the "needs manual tuning" basket. This would indeed be a great improvement. > Andre and Steven, I'm a bit too sleepy to properly review your combined > proposed changes right now and will follow up in the next few days instead. No problem, we've increased nmbclusters on all our machines and there now performing fine in the problem scenario so no rush, look forward to your feedback when you've had some sleep :) Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 01:31:47 2011 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F6221065687; Fri, 5 Aug 2011 01:31:47 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 36AE18FC1B; Fri, 5 Aug 2011 01:31:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p751VlZ5022884; Fri, 5 Aug 2011 01:31:47 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p751Vl7o022878; Fri, 5 Aug 2011 01:31:47 GMT (envelope-from linimon) Date: Fri, 5 Aug 2011 01:31:47 GMT Message-Id: <201108050131.p751Vl7o022878@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/159353: [netinet] [patch] conditional call of ifa_del_loopback_route() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 01:31:47 -0000 Old Synopsis: [patch] conditional call of ifa_del_loopback_route() New Synopsis: [netinet] [patch] conditional call of ifa_del_loopback_route() Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Fri Aug 5 01:31:22 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=159353 From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 06:47:43 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FCCF106564A; Fri, 5 Aug 2011 06:47:43 +0000 (UTC) (envelope-from mrossi@swin.edu.au) Received: from gpo4.cc.swin.edu.au (gpo4.cc.swin.edu.au [136.186.1.33]) by mx1.freebsd.org (Postfix) with ESMTP id 5BB3B8FC0A; Fri, 5 Aug 2011 06:47:41 +0000 (UTC) Received: from mrossi.caia.swin.edu.au (mrossi.caia.swin.edu.au [136.186.229.109]) by gpo4.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id p755xW9p002321; Fri, 5 Aug 2011 15:59:48 +1000 Message-ID: <4E3B86C4.7050005@swin.edu.au> Date: Fri, 05 Aug 2011 15:59:32 +1000 From: Mattia Rossi User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110707 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org, freebsd-current@freebsd.org, fbsd@opal.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: resolvconf script overwrites entries in resolv.conf - RDNSS/DNSSL related X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 06:47:43 -0000 Hi all, I've finally patched my 8.2 IPv6 gateway with the RDNSS/DNSSL patches and I'm distributing DNS servers that way now. Works fine, my box running CURRENT picks up the DNS servers and search domains and writes them into /etc/resolv.conf using the resolvconf script. The script anyhow overwrites my previous manual entries in /etc/resolv.conf which I need for my manual IPv4 setup... I think it's an easily fixable bug - haven't looked into it that close atm., but given that the resolvconf script is going to be rewritten/enhanced anyways, that's something to keep in mind. I think that manual entries should always be preferred. Mat From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 06:52:55 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 82EAF1065672; Fri, 5 Aug 2011 06:52:55 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 12-207-105-211.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id B6FA81A8172; Fri, 5 Aug 2011 06:52:54 +0000 (UTC) Message-ID: <4E3B9346.9000101@FreeBSD.org> Date: Thu, 04 Aug 2011 23:52:54 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110706 Thunderbird/5.0 MIME-Version: 1.0 To: Mattia Rossi References: <4E3B86C4.7050005@swin.edu.au> In-Reply-To: <4E3B86C4.7050005@swin.edu.au> X-Enigmail-Version: 1.2pre OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, fbsd@opal.com Subject: Re: resolvconf script overwrites entries in resolv.conf - RDNSS/DNSSL related X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 06:52:55 -0000 On 08/04/2011 22:59, Mattia Rossi wrote: > Hi all, > > I've finally patched my 8.2 IPv6 gateway with the RDNSS/DNSSL patches > and I'm distributing DNS servers that way now. Works fine, my box > running CURRENT picks up the DNS servers and search domains and writes > them into /etc/resolv.conf using the resolvconf script. > > The script anyhow overwrites my previous manual entries in > /etc/resolv.conf which I need for my manual IPv4 setup... > > I think it's an easily fixable bug - haven't looked into it that close > atm., but given that the resolvconf script is going to be > rewritten/enhanced anyways, that's something to keep in mind. > I think that manual entries should always be preferred. Check 'man resolvconf.conf' for information on name_servers_append. It would probably be nice if there was a _prepend equivalent. -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 07:23:08 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11DEC1065670; Fri, 5 Aug 2011 07:23:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 695F68FC16; Fri, 5 Aug 2011 07:23:07 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QpELn-000Hd0-PL; Fri, 05 Aug 2011 10:57:43 +0400 Date: Fri, 5 Aug 2011 10:57:43 +0400 From: Slawa Olhovchenkov To: Lawrence Stewart Message-ID: <20110805065743.GC94016@zxy.spb.ru> References: <1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk> <4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> <4E3AA66A.6060605@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E3AA66A.6060605@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Andre Oppermann , Steven Hartland , freebsd-net@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 07:23:08 -0000 On Fri, Aug 05, 2011 at 12:02:18AM +1000, Lawrence Stewart wrote: > > Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the > > tests appears to result in a solid 14MB/s, its still running a > > full soak test but looking very promising :) > > This is exactly the necessary tuning required to drive high BDP links > successfully. The unfortunate problem with my reassembly change was that > by removing the global count of reassembly segments and using the uma > zone to enforce the restrictions on memory use, we wouldn't necessarily > have room for the last segment (particularly if a single flow has a BDP > larger than the max size of the reassembly queue - which is the case for > you and Slawa). > > This is bad as Andre explained in his message, as we could stall > connections. I hadn't even considered the idea of allocating on the > stack as Andre has suggested in his patch, which I believe is an > appropriate solution to the the stalling problem assuming the function > will never return with the stack allocated tqe still in the reassembly > queue. My longer term goal is discussed below. > > > So I suppose the question is should maxsegments be larger by > > default due to the recent changes e.g. > > - V_tcp_reass_maxseg = nmbclusters / 16; > > + V_tcp_reass_maxseg = nmbclusters / 8; > > > > or is the correct fix something more involved? > > I'm not sure if bumping the value is appropriate - we have always > expected users to tune their network stack to perform well when used in > "unusual" scenarios - a large BDP fibre path still being in the > "unusual" category. > > The real fix which is somewhere down on my todo list is to make all > these memory constraints elastic and respond to VM pressure, thus > negating the need for a hard limit at all. This would solve many if not > most of the TCP tuning problems we currently have with one foul swoop > and would greatly reduce the need for tuning in many situations that > currently are in the "needs manual tuning" basket. Autotunig w/o limits is bad idea. This is way to DoS. May be solved this trouble by preallocation "hidden" element in tqe for segment received in valid order and ready for send to application? T.e. when creating reassembled queue for tcp connection we allocation queue element (with room for data payload), used only when data ready for application. Allocation in queue for not breaking ABI (don't change struct tcpcb). > Andre and Steven, I'm a bit too sleepy to properly review your combined > proposed changes right now and will follow up in the next few days instead. > > Cheers, > Lawrence From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 11:15:25 2011 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBF65106564A; Fri, 5 Aug 2011 11:15:25 +0000 (UTC) (envelope-from fbsd@opal.com) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) by mx1.freebsd.org (Postfix) with ESMTP id 6C00D8FC12; Fri, 5 Aug 2011 11:15:25 +0000 (UTC) Received: from pool-141-154-233-28.bos.east.verizon.net ([141.154.233.28] helo=homobox.opal.com) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1QpHgA-000CIN-AS; Fri, 05 Aug 2011 10:30:58 +0000 Received: from opal.com (localhost [IPv6:::1]) (authenticated bits=0) by homobox.opal.com (8.14.4/8.14.4) with ESMTP id p75AUsKB060669 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 5 Aug 2011 06:30:55 -0400 (EDT) (envelope-from fbsd@opal.com) Received: from shibato.opal.com ([173.52.168.221] helo=shibato.opal.com) with IPv4:587 by opal.com; 5 Aug 2011 06:30:54 -0400 X-Mail-Handler: MailHop Outbound by DynDNS X-Originating-IP: 141.154.233.28 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/mailhop/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+BPHq4u4Es/msrpL4yP/TZ Date: Fri, 5 Aug 2011 12:30:50 +0200 From: "J.R. Oldroyd" To: Doug Barton Message-ID: <20110805123050.1aa49e0c@shibato.opal.com> In-Reply-To: <4E3B9346.9000101@FreeBSD.org> References: <4E3B86C4.7050005@swin.edu.au> <4E3B9346.9000101@FreeBSD.org> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.20.1; amd64-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/tUxfInpl02g50qhPN=E=1j4"; protocol="application/pgp-signature" Cc: freebsd-net@FreeBSD.org, freebsd-current@FreeBSD.org, Mattia Rossi Subject: Re: resolvconf script overwrites entries in resolv.conf - RDNSS/DNSSL related X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 11:15:25 -0000 --Sig_/tUxfInpl02g50qhPN=E=1j4 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Thu, 04 Aug 2011 23:52:54 -0700, Doug Barton wrote: > > On 08/04/2011 22:59, Mattia Rossi wrote: > > I've finally patched my 8.2 IPv6 gateway with the RDNSS/DNSSL patches > > The script anyhow overwrites my previous manual entries in > > /etc/resolv.conf which I need for my manual IPv4 setup... > >=20 > Check 'man resolvconf.conf' for information on name_servers_append. It > would probably be nice if there was a _prepend equivalent. >=20 Mattia, when you say you have the patches, which ones? To be clear, the RDNSS/DNSSL support that was committed to head was very heavily modified from that which I proposed and which is on my web site. In particular, the resolvconf(8) tool that I offered was not used at all; the version in head is the openresolv tool by Roy Marples. Doug's response is in regard to the resolvconf version in head. FWIW, the resolvconf version from my site will use the most recent nameservers received by from either dhclient-script or rtadvd but you can also add "static" entries using standard resolv.conf syntax in the /var/db/resolv.db file; see the man page with that version. I will update my RDNSS/DNSSL web page now to add a warning that my patches have been superseded and that folk should use the committed versions where possible. -jr --Sig_/tUxfInpl02g50qhPN=E=1j4 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAk47xlsACgkQls33urr0k4keggCbBUnLoI+7VPLwFlKhqigBXYLq x44AniPXNDxibndiHW2NxNwgygJwbkPe =1AtI -----END PGP SIGNATURE----- --Sig_/tUxfInpl02g50qhPN=E=1j4-- From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 14:41:43 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 855CE1065670 for ; Fri, 5 Aug 2011 14:41:43 +0000 (UTC) (envelope-from direzione@amcservizi.net) Received: from aa002msb.fastweb.it (aa002msb.fastweb.it [85.18.95.81]) by mx1.freebsd.org (Postfix) with ESMTP id 0D7A18FC0C for ; Fri, 5 Aug 2011 14:41:42 +0000 (UTC) Received: from lorenzo52e46d1 (10.74.44.229) by aa002msb.fastweb.it (8.5.016.6) id 4E1C180501C79BE3 for freebsd-net@freebsd.org; Fri, 5 Aug 2011 16:30:07 +0200 From: "AmC Servizi - Direzione" To: Date: Fri, 5 Aug 2011 16:30:09 +0200 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcxTfC1k0mkEXEw/TdqY+Mv/+3Z4tA== Content-Language: it Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: conferma message X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: direzione@amcservizi.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 14:41:43 -0000 Welcome to the freebsd-net@freebsd.org mailing list! To post to this list, send your email to: freebsd-net@freebsd.org General information about the mailing list is at: http://lists.freebsd.org/mailman/listinfo/freebsd-net If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: http://lists.freebsd.org/mailman/options/freebsd-net/direzionetecnica%40amcl eaning.it You can also make such adjustments via email by sending a message to: freebsd-net-request@freebsd.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe. It is: veoxvi Normally, Mailman will remind you of your freebsd.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you. From owner-freebsd-net@FreeBSD.ORG Fri Aug 5 17:29:39 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A477106564A for ; Fri, 5 Aug 2011 17:29:39 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id A645A8FC16 for ; Fri, 5 Aug 2011 17:29:38 +0000 (UTC) Received: (qmail 17070 invoked from network); 5 Aug 2011 16:23:37 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 5 Aug 2011 16:23:37 -0000 Message-ID: <4E3C2886.9070100@freebsd.org> Date: Fri, 05 Aug 2011 19:29:42 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Matthew Cini Sarreo References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: ESP Raw Socket: Returned IP packet incorrect X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2011 17:29:39 -0000 On 11.07.2011 17:26, Matthew Cini Sarreo wrote: > Hello all; > > I have recently encountered a problem when using raw sockets on FreeBSD 8 > (8.0-RELEASE) when using ESP raw sockets. > > I have created a raw esp socket using: > socket(AF_INET, SOCK_RAW, 50); > which works fine. However, when there is a packet on the socket, recvfrom() > returns a packet where the length bytes in the IP header are incorrect; they > are swapped (MSB is placed in the LSB and vice-versa) > > tcpdump shows the following: > > tcpdump: listening on le0, link-type EN10MB (Ethernet), capture size 96 > bytes > 15:00:53.993810 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ESP > (50), length 120) > 10.0.251.228> 10.0.252.231: ESP(spi=0xa0534f17,seq=0x3), length 100 > 0x0000: 4500 0078 0000 4000 4032 2d88 0a00 fbe4 > 0x0010: 0a00 fce7 a053 4f17 0000 0003 6885 8abd > 0x0020: 2222 5ded 44dc 842f 3081 8fa3 bde4 2265 > 0x0030: 7438 2bf4 049c 664b 7dc4 44ef 1f6f 5e7d > 0x0040: b8c1 482f 8c3b f488 a19a 3d9a d5fe ed9d > 0x0050: b1c2 > > > However, recvfrom() returns the following buffer: > 4500 6400 0000 0040 4032 2D88 0A00 FBE4 > 0A00 FCE7 A053 4F17 0000 0003 6885 8ABD > 2222 5DED 44DC 842F 3081 8FA3 BDE4 2265 > 7438 2BF4 049C 664B 7DC4 44EF 1F6F 5E7D > B8C1 482F 8C3B F488 A19A 3D9A D5FE ED9D > B1C2 > > As it is easy to see, the length is not correct (bytes 2 and 3 are 0x6400 > instead of 0x0064) and it does not correspond to the value returned by > recvfrom(). > > Is this a known issue? Am I missing some options for raw sockets that are > required for FreeBSD? I have attempted this on a socket to a TUN interface > (not with an ESP socket) and the buffer had the proper length; it seems to > only happen with ESP. This code runs fine on multiple Linux distributions > and on Windows; it was only noticed with FreeBSD. Could it be that there is > some other ESP application running and interfering (I have not installed > any; don't know if there are by default and I'm quite new to any of the > BSDs)? The problem is with all raw sockets. Contrary to the statement in ip(4) "Incoming packets are received with IP header and options intact" and other popular OSes the ip_len field in the IP header has the IP header length already deducted (line 770 in ip_input.c). For normal in-kernel implemented protocols this is fine but raw sockets it is clearly wrong. The fix is pretty easy and just adds the header length back in raw_input() in raw_ip.c. Please test this patch: http://people.freebsd.org/~andre/raw_ip-header-length-20110805.diff -- Andre