From owner-freebsd-net@FreeBSD.ORG Sun Aug 16 01:29:10 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60DBC1065691; Sun, 16 Aug 2009 01:29:10 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id 3C7878FC3D; Sun, 16 Aug 2009 01:29:10 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rmac.psg.com) by ran.psg.com with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1McUYO-000OsS-GL; Sun, 16 Aug 2009 01:29:00 +0000 Received: from rmac.local.psg.com (localhost [127.0.0.1]) by rmac.psg.com (Postfix) with ESMTP id 094EF29543AE; Sun, 16 Aug 2009 10:29:00 +0900 (JST) Date: Sun, 16 Aug 2009 10:29:00 +0900 Message-ID: From: Randy Bush To: sthaug@nethelp.no In-Reply-To: <20090815.214022.41662662.sthaug@nethelp.no> References: <4A8601CE.5030205@delphij.net> <4A86C782.5030808@freebsd.org> <4A86F2BE.4050203@elischer.org> <20090815.214022.41662662.sthaug@nethelp.no> User-Agent: Wanderlust/2.15.5 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Sun, 16 Aug 2009 02:03:52 +0000 Cc: qing.li@bluecoat.com, brooks@freebsd.org, d@delphij.net, freebsd-net@freebsd.org, qingli@freebsd.org, andre@freebsd.org, bu7cher@yandex.ru, julian@elischer.org, bzeeb-lists@lists.zabbadoz.net Subject: Re: [Take 2] Re: RFC: interface description X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2009 01:29:10 -0000 >> From my perspective, putting it in a separate db outside the kernel >> kind of defeats the purpose. I thought the first patches had the >> right idea. though for me the current ability to rename an interface >> is good enough. I mean is you can cal your interface "Sydney0" or >> "Melbourne2" that is really enough.. > Having read the discussion, I agree that the description should be > in the kernel. However, being a router geek the ability to rename > an interface to "Sydney0" or "Melbourne2" is not at all enough. For > the routers & switches I work with we really want a description of > at least 50 characters - and it's important to be able to include > space. also a router geek. but for the sake of simplicity, i am quite willing to s/\ /_/ or whatever. randy From owner-freebsd-net@FreeBSD.ORG Sun Aug 16 16:02:27 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0FCE106568D; Sun, 16 Aug 2009 16:02:27 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A80AE8FC41; Sun, 16 Aug 2009 16:02:27 +0000 (UTC) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7GG2ReX052473; Sun, 16 Aug 2009 16:02:27 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7GG2RWG052469; Sun, 16 Aug 2009 16:02:27 GMT (envelope-from linimon) Date: Sun, 16 Aug 2009 16:02:27 GMT Message-Id: <200908161602.n7GG2RWG052469@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: bin/137841: [patch] wpa_supplicant(8) cannot verify SHA256 signed certificates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2009 16:02:27 -0000 Old Synopsis: wpa_supplicant cannot verify SHA256 signed certificates New Synopsis: [patch] wpa_supplicant(8) cannot verify SHA256 signed certificates Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Aug 16 16:01:47 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=137841 From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 11:07:00 2009 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F616106568C for ; Mon, 17 Aug 2009 11:07:00 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 2C60E8FC68 for ; Mon, 17 Aug 2009 11:07:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HB70BQ075884 for ; Mon, 17 Aug 2009 11:07:00 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HB6xXn075880 for freebsd-net@FreeBSD.org; Mon, 17 Aug 2009 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Aug 2009 11:06:59 GMT Message-Id: <200908171106.n7HB6xXn075880@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 11:07:00 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed o kern/137795 net [sctp] [panic] mtx_lock() of destroyed mutex o kern/137775 net [netgraph] [patch] Add XMIT_FAILOVER to ng_one2many o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137592 net [ath] panic - 7-STABLE (Aug 7, 2009 UTC) crashes on ne o bin/137484 net [patch] Integer overflow in wpa_supplicant(8) base64 e o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137317 net [tcp] logs full of syncache problems o kern/137292 net [ste] DFE-580TX not working properly o kern/137279 net [bge] [panic] Page fault (fatal trap 12) NFS server w/ o kern/137170 net [ath] atheros AR9285 not recognised o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136943 net [wpi] [lor] wpi0_com_lock / wpi0 o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136876 net [bge] bge will not resume properly after suspend o kern/136836 net [ath] atheros card stops functioning after about 12 ho o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/136482 net [age] Attansic L1 Gigabit Ethernet recieves multicasts o kern/136168 net [em] em driver initialization fails on Intel 5000PSL m o kern/135836 net [bce] bce BCM5709 Watchdog after warm boot - ok after o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/135222 net [igb] low speed routing between two igb interfaces o kern/135067 net [patch] [fib] Incorrect KASSERTs in sys/net/route.c o kern/134931 net [route] [fib] Route messages sent to all socket listen o kern/134658 net [bce] bce driver fails on PowerEdge m610 blade. o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134401 net [msk] [panic] Kernel Fatal trap 12: page fault while i o kern/134369 net [route] [ip6] IPV6 in Head broken for routing table up o kern/134168 net [ral] ral driver problem on RT2525 2.4GHz transceiver o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/134079 net [em] "em0: Invalid MAC address" in FreeBSD-Current ( 8 o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133902 net [tun] Killing tun0 iface ssh tunnel causes Panic Strin o kern/133736 net [udp] ip_id not protected ... o kern/133613 net [wpi] [panic] kernel panic in wpi(4) o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133328 net [bge] [panic] Kernel panics with Windows7 client o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre o kern/133218 net [carp] [hang] use of carp(4) causes system to freeze o kern/133204 net [msk] msk driver timeouts o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132991 net [bge] if_bge low performance problem f bin/132911 net ip6fw(8): argument type of fill_icmptypes is wrong and o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132885 net [wlan] 802.1x broken after SVN rev 189592 o conf/132851 net [fib] [patch] allow to setup fib for service running f o kern/132832 net [netinet] [patch] tcp_output() might generate invalid o bin/132798 net [patch] ggatec(8): ggated/ggatec connection slowdown p o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132722 net [ath] Wifi ath0 associates fine with AP, but DHCP or I o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132669 net [xl] 3c905-TX send DUP! in reply on ping (sometime) o kern/132625 net [iwn] iwn drivers don't support setting country o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132285 net [carp] alias gives incorrect hash in dmesg o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o conf/132179 net [patch] /etc/network.subr: ipv6 rtsol on incorrect wla o kern/132107 net [carp] carp(4) advskew setting ignored when carp IP us o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o kern/131549 net ifconfig(8) can't clear 'monitor' mode on the wireless o kern/131536 net [netinet] [patch] kernel does allow manipulation of su o bin/131365 net route(8): route add changes interpretation of network o kern/131162 net [ath] Atheros driver bugginess and kernel crashes o kern/131153 net [iwi] iwi doesn't see a wireless network f kern/131087 net [ipw] [panic] ipw / iwi - no sent/received packets; iw f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour o kern/129750 net [ath] Atheros AR5006 exits on "cannot map register spa f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129580 net [ndis] Netgear WG311v3 (ndis) causes kenel trap at boo o kern/129517 net [ipsec] [panic] double fault / stack overflow o kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129352 net [xl] [patch] xl0 watchdog timeout o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o kern/129135 net [vge] vge driver on a VIA mini-ITX not working o bin/128954 net ifconfig(8) deletes valid routes o kern/128917 net [wpi] [panic] if_wpi and wpa+tkip causing kernel panic o kern/128884 net [msk] if_msk page fault while in kernel mode o kern/128840 net [igb] page fault under load with igb/LRO o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128598 net [bluetooth] WARNING: attempt to net_add_domain(bluetoo o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o conf/128334 net [request] use wpa_cli in the "WPA DHCP" situation o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127928 net [tcp] [patch] TCP bandwidth gets squeezed every time t o kern/127834 net [ixgbe] [patch] wrong error counting o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) s kern/127587 net [bge] [request] if_bge(4) doesn't support BCM576X fami f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/127102 net [wpi] Intel 3945ABG low throughput o kern/127057 net [udp] Unable to send UDP packet via IPv6 socket to IPv o kern/127050 net [carp] ipv6 does not work on carp interfaces [regressi o kern/126945 net [carp] CARP interface destruction with ifconfig destro o kern/126924 net [an] [patch] printf -> device_printf and simplify prob o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o bin/126822 net wpa_supplicant(8): WPA PSK does not work in adhoc mode o kern/126714 net [carp] CARP interface renaming makes system no longer o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126688 net [ixgbe] [patch] 1.4.7 ixgbe driver panic with 4GB and o kern/126475 net [ath] [panic] ath pcmcia card inevitably panics under o kern/126339 net [ipw] ipw driver drops the connection o kern/126214 net [ath] txpower problem with Atheros wifi card o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125816 net [carp] [if_bridge] carp stuck in init when using bridg f kern/125502 net [ral] ifconfig ral0 scan produces no output unless in o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124767 net [iwi] Wireless connection using iwi0 driver (Intel 220 o kern/124753 net [ieee80211] net80211 discards power-save queue packets o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124127 net [msk] watchdog timeout (missed Tx interrupts) -- recov o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. p kern/123961 net [vr] [patch] Allow vr interface to handle vlans o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o bin/123633 net ifconfig(8) doesn't set inet and ether address in one f kern/123617 net [tcp] breaking connection when client downloading file o kern/123603 net [tcp] tcp_do_segment and Received duplicate SYN o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o kern/123429 net [nfe] [hang] "ifconfig nfe up" causes a hard system lo o kern/123347 net [bge] bge1: watchdog timeout -- linkstate changed to D o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123256 net [wpi] panic: blockable sleep lock with wpi(4) f kern/123172 net [bce] Watchdog timeout problems with if_bce o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices o kern/122928 net [em] interface watchdog timeouts and stops receiving p f kern/122839 net [multicast] FreeBSD 7 multicast routing problem p kern/122794 net [lagg] Kernel panic after brings lagg(8) up if NICs ar o kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122772 net [em] em0 taskq panic, tcp reassembly bug causes radix o kern/122743 net [mbuf] [panic] vm_page_unwire: invalid wire count: 0 o kern/122697 net [ath] Atheros card is not well supported o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122551 net [bge] Broadcom 5715S no carrier on HP BL460c blade usi o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal f kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122195 net [ed] Alignment problems in if_ed o kern/122058 net [em] [panic] Panic on em1: taskq o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup [reg o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw o kern/121872 net [wpi] driver fails to attach on a fujitsu-siemens s711 s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121706 net [netinet] [patch] "rtfree: 0xc4383870 has 1 refs" emit o kern/121624 net [em] [regression] Intel em WOL fails after upgrade to o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] ppp(8): fix local stack overflow in ppp o kern/121298 net [em] [panic] Fatal trap 12: page fault while in kernel o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/121080 net [bge] IPv6 NUD problem on multi address config on bge0 o kern/120966 net [rum] kernel panic with if_rum and WPA encryption p docs/120945 net [patch] ip6(4) man page lacks documentation for TCLASS o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o kern/120232 net [nfe] [patch] Bring in nfe(4) to RELENG_6 o kern/120130 net [carp] [panic] carp causes kernel panics in any conste o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr a bin/118987 net ifconfig(8): ifconfig -l (address_family) does not wor o sparc/118932 net [panic] 7.0-BETA4/sparc-64 kernel panic in rip_output a kern/118879 net [bge] [patch] bge has checksum problems on the 5703 ch o kern/118727 net [netgraph] [patch] [request] add new ng_pf module a kern/118238 net [bce] [patch] bce driver shows "no carrier" on Intel S s kern/117717 net [panic] Kernel panic with Bittorrent client. o kern/117448 net [carp] 6.2 kernel crash [regression] o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o kern/117043 net [em] Intel PWLA8492MT Dual-Port Network adapter EEPROM o kern/116837 net [tun] [panic] [patch] ifconfig tunX destroy: panic o kern/116747 net [ndis] FreeBSD 7.0-CURRENT crash with Dell TrueMobile o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116328 net [bge]: Solid hang with bge interface o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113895 net [xl] xl0 fails on 6.2-RELEASE but worked fine on 5.5-R o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o kern/112570 net [bge] packet loss with bge driver on BCM5704 chipset o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111457 net [ral] ral(4) freeze o kern/110140 net [ipw] ipw fails under load o kern/109733 net [bge] bge link state issues [regression] o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o kern/109308 net [pppd] [panic] Multiple panics kernel ppp suspected [r o kern/109251 net [re] [patch] if_re cardbus card won't attach o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/108542 net [bce] Huge network latencies with 6.2-RELEASE / STABLE o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o kern/107850 net [bce] bce driver link negotiation is faulty o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/106243 net [nve] double fault panic in if_nve.c on high loads o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/105348 net [ath] ath device stopps TX o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104485 net [bge] Broadcom BCM5704C: Intermittent on newer chip ve o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working f bin/97392 net ppp(8) hangs instead terminating o kern/97306 net [netgraph] NG_L2TP locks after connection with failed f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/96030 net [bfe] [patch] Install hangs with Broadcomm 440x NIC in o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear s kern/94863 net [bge] [patch] hack to get bge(4) working on IBM e326m o kern/94162 net [bge] 6.x kenel stale with bge(4) o kern/93886 net [ath] Atheros/D-Link DWL-G650 long delay to associate f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi f kern/92552 net A serious bug in most network drivers from 5.X to 6.X s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/92090 net [bge] bge0: watchdog timeout -- resetting o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91594 net [em] FreeBSD > 5.4 w/ACPI fails to detect Intel Pro/10 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/90890 net [vr] Problems with network: vr0: tx shutdown timeout s kern/90086 net [hang] 5.4p8 on supermicro P8SCT hangs during boot if f kern/88082 net [ath] [panic] cts protection for ath0 causes panic o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87506 net [vr] [patch] Fix alias support on vr interfaces s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o kern/85266 net [xe] [patch] xe(4) driver does not recognise Xircom XE o kern/84202 net [ed] [patch] Holtek HT80232 PCI NIC recognition on Fre o bin/82975 net route change does not parse classfull network as given o kern/82497 net [vge] vge(4) on AMD64 only works when loaded late, not f kern/81644 net [vge] vge(4) does not work properly when loaded as a K s kern/81147 net [net] [patch] em0 reinitialization while adding aliase o kern/80853 net [ed] [patch] add support for Compex RL2000/ISA in PnP o kern/79895 net [ipf] 5.4-RC2 breaks ipfilter NAT when using netgraph f kern/79262 net [dc] Adaptec ANA-6922 not fully supported o bin/79228 net [patch] extend arp(8) to be able to create blackhole r o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if p kern/77913 net [wi] [patch] Add the APDL-325 WLAN pccard to wi(4) o kern/77341 net [ip6] problems with IPV6 implementation o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time f kern/73538 net [bge] problem with the Broadcom BCM5788 Gigabit Ethern o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/64556 net [sis] if_sis short cable fix problems with NetGear FA3 s kern/60293 net [patch] FreeBSD arp poison patch o kern/54383 net [nfs] [patch] NFS root configurations without dynamic f i386/45773 net [bge] Softboot causes autoconf failure on Broadcom 570 s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr s kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/35442 net [sis] [patch] Problem transmitting runts in if_sis dri o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c o conf/23063 net [arp] [patch] for static ARP tables in rc.network 323 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 16:10:05 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DE0D106568B for ; Mon, 17 Aug 2009 16:10:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D842E8FC52 for ; Mon, 17 Aug 2009 16:10:04 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HGA4GD014267 for ; Mon, 17 Aug 2009 16:10:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HGA4l1014266; Mon, 17 Aug 2009 16:10:04 GMT (envelope-from gnats) Date: Mon, 17 Aug 2009 16:10:04 GMT Message-Id: <200908171610.n7HGA4l1014266@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Michael Tuexen Cc: Subject: Re: kern/137795: [sctp] [panic] mtx_lock() of destroyed mutex X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Michael Tuexen List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 16:10:05 -0000 The following reply was made to PR kern/137795; it has been noted by GNATS. From: Michael Tuexen To: bug-followup@FreeBSD.org, bruce@cran.org.uk Cc: Subject: Re: kern/137795: [sctp] [panic] mtx_lock() of destroyed mutex Date: Mon, 17 Aug 2009 18:07:45 +0200 Hi Bruce, is there running a (discard) server at 192.168.1.80, port 2345? Best regards Michael From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 19:26:48 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F2DE106568D; Mon, 17 Aug 2009 19:26:48 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 76C078FC62; Mon, 17 Aug 2009 19:26:48 +0000 (UTC) Received: from freefall.freebsd.org (remko@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HJQm7O067266; Mon, 17 Aug 2009 19:26:48 GMT (envelope-from remko@freefall.freebsd.org) Received: (from remko@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HJQmSI067262; Mon, 17 Aug 2009 19:26:48 GMT (envelope-from remko) Date: Mon, 17 Aug 2009 19:26:48 GMT Message-Id: <200908171926.n7HJQmSI067262@freefall.freebsd.org> To: remko@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: remko@FreeBSD.org Cc: Subject: Re: kern/137776: [rum] panic in rum(4) driver on 8.0-BETA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 19:26:48 -0000 Synopsis: [rum] panic in rum(4) driver on 8.0-BETA2 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: remko Responsible-Changed-When: Mon Aug 17 19:26:38 UTC 2009 Responsible-Changed-Why: Reassin to networking team http://www.freebsd.org/cgi/query-pr.cgi?pr=137776 From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 19:50:03 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41DBD106568F for ; Mon, 17 Aug 2009 19:50:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 164728FC66 for ; Mon, 17 Aug 2009 19:50:03 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HJo2Tl083360 for ; Mon, 17 Aug 2009 19:50:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HJo2EB083359; Mon, 17 Aug 2009 19:50:02 GMT (envelope-from gnats) Date: Mon, 17 Aug 2009 19:50:02 GMT Message-Id: <200908171950.n7HJo2EB083359@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Michael Tuexen Cc: Subject: Re: kern/137795: [sctp] [panic] mtx_lock() of destroyed mutex X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Michael Tuexen List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 19:50:03 -0000 The following reply was made to PR kern/137795; it has been noted by GNATS. From: Michael Tuexen To: bug-followup@FreeBSD.org, bruce@cran.org.uk Cc: Subject: Re: kern/137795: [sctp] [panic] mtx_lock() of destroyed mutex Date: Mon, 17 Aug 2009 21:40:33 +0200 Hi Bruce, OK, with a server available it runs fine, but I can reproduce a crash when there is no server available. I'll have a look into the problem. Thanks for reporting the issue. Best regards Michael From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 20:17:22 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA7BB1065690; Mon, 17 Aug 2009 20:17:22 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C10D08FC7A; Mon, 17 Aug 2009 20:17:22 +0000 (UTC) Received: from freefall.freebsd.org (emaste@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HKHM6u005838; Mon, 17 Aug 2009 20:17:22 GMT (envelope-from emaste@freefall.freebsd.org) Received: (from emaste@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HKHMoG005834; Mon, 17 Aug 2009 20:17:22 GMT (envelope-from emaste) Date: Mon, 17 Aug 2009 20:17:22 GMT Message-Id: <200908172017.n7HKHMoG005834@freefall.freebsd.org> To: robhass@gmail.com, emaste@FreeBSD.org, freebsd-net@FreeBSD.org From: emaste@FreeBSD.org Cc: Subject: Re: kern/122794: [lagg] Kernel panic after brings lagg(8) up if NICs are not bringed up before X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 20:17:23 -0000 Synopsis: [lagg] Kernel panic after brings lagg(8) up if NICs are not bringed up before State-Changed-From-To: patched->closed State-Changed-By: emaste State-Changed-When: Mon Aug 17 20:16:50 UTC 2009 State-Changed-Why: I see that thompsa has MFC'd this to 7 & 6. http://www.freebsd.org/cgi/query-pr.cgi?pr=122794 From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 20:40:12 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87B63106568B for ; Mon, 17 Aug 2009 20:40:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC9A8FC51 for ; Mon, 17 Aug 2009 20:40:12 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HKeCr9027849 for ; Mon, 17 Aug 2009 20:40:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HKeCO3027846; Mon, 17 Aug 2009 20:40:12 GMT (envelope-from gnats) Date: Mon, 17 Aug 2009 20:40:12 GMT Message-Id: <200908172040.n7HKeCO3027846@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Ed Maste Cc: Subject: Re: kern/122780: [lagg] tcpdump on lagg interface during high pps wedges netcode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ed Maste List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 20:40:12 -0000 The following reply was made to PR kern/122780; it has been noted by GNATS. From: Ed Maste To: bug-followup@FreeBSD.org, paul@gtcomm.net Cc: Subject: Re: kern/122780: [lagg] tcpdump on lagg interface during high pps wedges netcode Date: Mon, 17 Aug 2009 16:14:13 -0400 Can you retest on recent 7.x or HEAD? I think there have been BPF changes to address this issue. -Ed From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 22:28:33 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC992106568D; Mon, 17 Aug 2009 22:28:33 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 839648FC69; Mon, 17 Aug 2009 22:28:33 +0000 (UTC) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7HMSXrI010503; Mon, 17 Aug 2009 22:28:33 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7HMSXHt010499; Mon, 17 Aug 2009 22:28:33 GMT (envelope-from linimon) Date: Mon, 17 Aug 2009 22:28:33 GMT Message-Id: <200908172228.n7HMSXHt010499@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/137881: [netgraph] [panic] ng_pppoe fatal trap 12 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 22:28:33 -0000 Old Synopsis: [netgraph] ng_pppoe fatal trap 12 New Synopsis: [netgraph] [panic] ng_pppoe fatal trap 12 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Mon Aug 17 22:27:58 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=137881 From owner-freebsd-net@FreeBSD.ORG Mon Aug 17 22:47:22 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35C29106568D for ; Mon, 17 Aug 2009 22:47:22 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id EFC2D8FC3D for ; Mon, 17 Aug 2009 22:47:21 +0000 (UTC) Received: by vws10 with SMTP id 10so2780129vws.7 for ; Mon, 17 Aug 2009 15:47:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.18.75 with SMTP id v11mr5599206vca.95.1250547898764; Mon, 17 Aug 2009 15:24:58 -0700 (PDT) Date: Mon, 17 Aug 2009 16:24:58 -0600 Message-ID: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> From: Manish Vachharajani To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 22:47:22 -0000 I've been doing some performance testing on freebsd 7.2 and noticed that the ixgbe driver does not report missed packets as dropped when queried via netstat -id (the ixgbe driver in the 8.0 beta has a similar issue). A missed packet is a packet that was correctly received by the NIC but because it was out of descriptors and internal memory, the packet had to be dropped by the NIC itself -- a hardware register counts such events. Instead the driver only reports drops that are due to the driver itself, i.e., the NIC DMAed the packet to memory but the driver had to drop something because it was out of mbufs. Is the miss count reported elsewhere (besides via a kernel printf from the driver)? At the end of the email I give a stats dump from the driver and from netstat, note the number of packets dropped is 0 as reported by netstat but the Missed field in the dmesg output shows many missed packets. >From my perspective, it is disconcerting to see performance degradation on the link, along with TCP ack retransmits, packet reordering, etc. (on a point-to-point link with no switch in between) but then see no drops reported by netstat because the driver didn't drop the packet, the NIC did. The fix should be straight-forward and I'll gladly make a patch assuming that it is indeed a bug and not a conscious design choice. Here is the relevant netstat output Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll Drop ix0 1500 00:30:48:94:60:ec 0 0 1 0 0 0 ix0 1500 192.168.105.0 192.168.105.2 0 - 0 - - - ix1 1500 00:30:48:94:60:ed 11M 0 6.1M 0 0 0 ix1 1500 192.168.5.0 192.168.5.2 10M - 6.1M - - - And here is the dmesg output after doing a sysctl dev.ix.1.stats=1 ix1: Std Mbuf Failed = 0 ix1: Missed Packets = 413872 ix1: Receive length errors = 0 ix1: Crc errors = 0 ix1: Driver dropped packets = 0 ix1: watchdog timeouts = 0 ix1: XON Rcvd = 616428212235 ix1: XON Xmtd = 0 ix1: XOFF Rcvd = 616428212235 ix1: XOFF Xmtd = 0 ix1: Total Packets Rcvd = 12424533 ix1: Good Packets Rcvd = 12010661 ix1: Good Packets Xmtd = 6419128 ix1: TSO Transmissions = 0 Manish -- Manish Vachharajani manishv@lineratesystems.com From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 02:51:20 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34EBD1065736 for ; Tue, 18 Aug 2009 02:51:20 +0000 (UTC) (envelope-from glen.j.barber@gmail.com) Received: from mail-fx0-f205.google.com (mail-fx0-f205.google.com [209.85.220.205]) by mx1.freebsd.org (Postfix) with ESMTP id BB4AF8FC45 for ; Tue, 18 Aug 2009 02:51:19 +0000 (UTC) Received: by fxm1 with SMTP id 1so2649676fxm.7 for ; Mon, 17 Aug 2009 19:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=J0R/ixCgKgLOuJvzMtu/q++e5E3YG3jJolaC5nvug6g=; b=TmjkkS9Z+rp+tjh75sHqfyhdekMSk97qIo/PlTpDlmFQ9T8krLiIsVp+K8gCvSEoGr zjCCQQfSlXM2q/9Eqjk/oOn/mMEN78OY20Szot8TFzM3OPROk56Xp6+MguaU8xL6fSVv CZmEtd3KA3d+3LlzQxWEvEbgviOcwNRm2KovI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mFr00d3RY43b7qi/whEvqzQ2Z7WCxIOabVKryj08gEMdAu/20cdzT2fdTX4SWhUUFg HAaSdEgbGxAP9BNp+dU7AdQ/ayCaxZJC7Pod7i84CLQeo035doG7GTA/ynoLLh8xj5Ew VKm7JZ9XNHWLr8pKG5hiwkqhDtCiqcH1JIRf0= MIME-Version: 1.0 Received: by 10.223.144.143 with SMTP id z15mr1023632fau.33.1250563872257; Mon, 17 Aug 2009 19:51:12 -0700 (PDT) In-Reply-To: <11167f520908142010g278e0cd7q3acaf989b75ac69a@mail.gmail.com> References: <20090815004549.GA70621@weongyo.local> <11167f520908142010g278e0cd7q3acaf989b75ac69a@mail.gmail.com> Date: Mon, 17 Aug 2009 22:51:12 -0400 Message-ID: <4ad871310908171951n6295ef18x3ce2d790969ca162@mail.gmail.com> From: Glen Barber To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: 8. Current and Intel 5100 AGN wifi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 02:51:20 -0000 On Fri, Aug 14, 2009 at 11:10 PM, Sam Fourman Jr. wrote= : > On Fri, Aug 14, 2009 at 7:45 PM, Weongyo Jeong w= rote: >> On Sun, Aug 09, 2009 at 02:41:20PM +0100, Alireza Torabi wrote: >>> folks, >>> >>> Can anyone please shet any lights on this. are iwn & iwnfw =A0going to >>> support 5100 agn wifi? >>> I've updated mine to 8. Current with iwn & iwnfw and still not going >>> anywhere with the card. >> >> AFAIK no one working on it. >> >> regards, >> Weongyo Jeong > > I have a spare 5100 agn to donate if someone want to work on a driver > Hi, I don't have a spare card to donate unfortunately, but I'm willing (and hopefully able) to test patches on 8-STABLE. --=20 Glen Barber From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 20:00:14 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96E3F1065691 for ; Tue, 18 Aug 2009 20:00:14 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 84F038FC5B for ; Tue, 18 Aug 2009 20:00:14 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7IK04ha049426 for ; Tue, 18 Aug 2009 20:00:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7IK04wi049425; Tue, 18 Aug 2009 20:00:04 GMT (envelope-from gnats) Date: Tue, 18 Aug 2009 20:00:04 GMT Message-Id: <200908182000.n7IK04wi049425@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/137795: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 20:00:14 -0000 The following reply was made to PR kern/137795; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/137795: commit references a PR Date: Tue, 18 Aug 2009 19:59:03 +0000 (UTC) Author: tuexen Date: Tue Aug 18 19:58:49 2009 New Revision: 196364 URL: http://svn.freebsd.org/changeset/base/196364 Log: Fix a crash when using one-to-one stlye socket in non-blocking mode and there is no listening server. PR: 137795 Approved by: re, rrs (mentor) MFC after:immediately. Modified: head/sys/netinet/sctp_output.c Modified: head/sys/netinet/sctp_output.c ============================================================================== --- head/sys/netinet/sctp_output.c Tue Aug 18 16:23:09 2009 (r196363) +++ head/sys/netinet/sctp_output.c Tue Aug 18 19:58:49 2009 (r196364) @@ -12464,7 +12464,8 @@ sctp_lower_sosend(struct socket *so, error = ENOTCONN; goto out_unlocked; } - hold_tcblock = 0; + SCTP_TCB_LOCK(stcb); + hold_tcblock = 1; SCTP_INP_RUNLOCK(inp); if (addr) { /* Must locate the net structure if addr given */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 20:10:02 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFC38106568F for ; Tue, 18 Aug 2009 20:10:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BE38F8FC62 for ; Tue, 18 Aug 2009 20:10:02 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7IKA2A9056359 for ; Tue, 18 Aug 2009 20:10:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7IKA2mf056358; Tue, 18 Aug 2009 20:10:02 GMT (envelope-from gnats) Date: Tue, 18 Aug 2009 20:10:02 GMT Message-Id: <200908182010.n7IKA2mf056358@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/137795: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 20:10:02 -0000 The following reply was made to PR kern/137795; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/137795: commit references a PR Date: Tue, 18 Aug 2009 20:06:16 +0000 (UTC) Author: tuexen Date: Tue Aug 18 20:06:00 2009 New Revision: 196365 URL: http://svn.freebsd.org/changeset/base/196365 Log: Fix a panic when using one-to-one style sockets in non-blocking mode and there is no listening server. PR: 137795 Approved by: re, rrs (mentor) Modified: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) stable/8/sys/netinet/sctp_output.c Modified: stable/8/sys/netinet/sctp_output.c ============================================================================== --- stable/8/sys/netinet/sctp_output.c Tue Aug 18 19:58:49 2009 (r196364) +++ stable/8/sys/netinet/sctp_output.c Tue Aug 18 20:06:00 2009 (r196365) @@ -12464,7 +12464,8 @@ sctp_lower_sosend(struct socket *so, error = ENOTCONN; goto out_unlocked; } - hold_tcblock = 0; + SCTP_TCB_LOCK(stcb); + hold_tcblock = 1; SCTP_INP_RUNLOCK(inp); if (addr) { /* Must locate the net structure if addr given */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 20:54:37 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 29033106568C for ; Tue, 18 Aug 2009 20:54:37 +0000 (UTC) (envelope-from alireza.torabi@gmail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id 84BEA8FC43 for ; Tue, 18 Aug 2009 20:54:35 +0000 (UTC) Received: by ewy5 with SMTP id 5so422314ewy.36 for ; Tue, 18 Aug 2009 13:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=w7kvjbLyFJFiFR5jQ+x6lPMTqYp0nRiBRcVEFxSixXg=; b=MN1ISaucLsvCHjb6A6H43pm4MgrOV18QvM3+aq/6KWXiYlujuUOc+8oOuRH0YXe8NX ABCnJtpF8Ngo3QMTdlt9eK3EoS5KcCBe2CJupywJGqyxA4uESx6jfLJKiEbi7AVbbfj4 iIzXE3o+A3sbuuj1/A8nVvp04Ef+A6LQ1NJcc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=Ey9oTGPfO3hg9NE9lBjc9FlyEGLtNMZq6cIPBzyqJXXxHhOwvhOcNz6PJJtjOhdZRD HqqSNCKbyJ98gK88/dJBotvVbEHFmprdTow4G9/j+KkLHMGNH70hvqZusFvIYrqfANnN XMg9XTS71KwNRQGR3wBwoRcnXTJm9z4ionxw4= MIME-Version: 1.0 Received: by 10.216.12.85 with SMTP id 63mr1438724wey.89.1250628874857; Tue, 18 Aug 2009 13:54:34 -0700 (PDT) In-Reply-To: <4ad871310908171951n6295ef18x3ce2d790969ca162@mail.gmail.com> References: <20090815004549.GA70621@weongyo.local> <11167f520908142010g278e0cd7q3acaf989b75ac69a@mail.gmail.com> <4ad871310908171951n6295ef18x3ce2d790969ca162@mail.gmail.com> Date: Tue, 18 Aug 2009 21:54:34 +0100 Message-ID: From: Alireza Torabi To: Glen Barber , freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: 8. Current and Intel 5100 AGN wifi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 20:54:37 -0000 So will I On 18/08/2009, Glen Barber wrote: > On Fri, Aug 14, 2009 at 11:10 PM, Sam Fourman Jr. wro= te: >> On Fri, Aug 14, 2009 at 7:45 PM, Weongyo Jeong >> wrote: >>> On Sun, Aug 09, 2009 at 02:41:20PM +0100, Alireza Torabi wrote: >>>> folks, >>>> >>>> Can anyone please shet any lights on this. are iwn & iwnfw =A0going to >>>> support 5100 agn wifi? >>>> I've updated mine to 8. Current with iwn & iwnfw and still not going >>>> anywhere with the card. >>> >>> AFAIK no one working on it. >>> >>> regards, >>> Weongyo Jeong >> >> I have a spare 5100 agn to donate if someone want to work on a driver >> > > Hi, > > I don't have a spare card to donate unfortunately, but I'm willing > (and hopefully able) to test patches on 8-STABLE. > > > -- > Glen Barber > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 Sent from my mobile device From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 21:05:24 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C93D91065672; Tue, 18 Aug 2009 21:05:24 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 21CF48FC45; Tue, 18 Aug 2009 21:05:16 +0000 (UTC) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 8F5715C06F; Wed, 19 Aug 2009 05:05:10 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 3B74F55CDC94; Wed, 19 Aug 2009 05:05:10 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id Mb-qDex1uhlT; Wed, 19 Aug 2009 05:04:11 +0800 (CST) Received: from charlie.delphij.net (adsl-76-237-33-62.dsl.pltn13.sbcglobal.net [76.237.33.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 9451255CDC82; Wed, 19 Aug 2009 05:03:59 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type; b=DvB6xfedbIueLqIBf2+zT7JUEMrVF0VR9KtnXRQiQBLClNT9ASZ+Q2sBpaUl5mPrb Q0u0eh/uIgTbJNVxmtWuA== Message-ID: <4A8B1729.8070503@delphij.net> Date: Tue, 18 Aug 2009 14:03:37 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.22 (X11/20090803) MIME-Version: 1.0 To: Jack Vogel References: <5D267A3F22FD854F8F48B3D2B523819339EC3813D2@IRVEXCHCCR01.corp.ad.broadcom.com> <692150.91493.qm@web63906.mail.re1.yahoo.com> <2a41acea0908060905n69edf1dfkbf993d9f8a4edf37@mail.gmail.com> In-Reply-To: <2a41acea0908060905n69edf1dfkbf993d9f8a4edf37@mail.gmail.com> X-Enigmail-Version: 0.96.0 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------050902070402030906080103" Cc: Barney Cordoba , David Christensen , "d@delphij.net" , "freebsd-net@freebsd.org" , Julian Elischer , Jack F Vogel , yongari@FreeBSD.org Subject: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 21:05:25 -0000 This is a multi-part message in MIME format. --------------050902070402030906080103 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Jack, I have looked into the code history and found that sys/dev/em/if_em.c,v 1.119 has introduced the arp_ifinit() call in order to fix the problem that if_em won't send ARP when IP address is changed. I think we can further improve it as attached, say, only do it when IFF_NOARP is not set. This should have no effect for usual configuration but fix the problem when NOARP is the desired behavior. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkqLFykACgkQi+vbBBjt66CMFQCeOkESwsgDAbqe5PCtiMulaU1E lIAAoIm0LDJ6qHuR8jyo7dXFi/9iYA22 =9E54 -----END PGP SIGNATURE----- --------------050902070402030906080103 Content-Type: text/plain; name="e1000-noarp-fix.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="e1000-noarp-fix.diff" Index: if_igb.c =================================================================== --- if_igb.c (revision 196363) +++ if_igb.c (working copy) @@ -952,7 +952,8 @@ igb_ioctl(struct ifnet *ifp, u_long command, caddr igb_init_locked(adapter); IGB_CORE_UNLOCK(adapter); } - arp_ifinit(ifp, ifa); + if (!(ifp->if_flags & IFF_NOARP)) + arp_ifinit(ifp, ifa); } else #endif error = ether_ioctl(ifp, command, data); Index: if_em.c =================================================================== --- if_em.c (revision 196363) +++ if_em.c (working copy) @@ -1204,7 +1204,8 @@ em_ioctl(struct ifnet *ifp, u_long command, caddr_ em_init_locked(adapter); EM_CORE_UNLOCK(adapter); } - arp_ifinit(ifp, ifa); + if (!(ifp->if_flags & IFF_NOARP)) + arp_ifinit(ifp, ifa); } else #endif error = ether_ioctl(ifp, command, data); --------------050902070402030906080103-- From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 21:34:12 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 938511065691; Tue, 18 Aug 2009 21:34:12 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-yw0-f199.google.com (mail-yw0-f199.google.com [209.85.211.199]) by mx1.freebsd.org (Postfix) with ESMTP id 081688FC68; Tue, 18 Aug 2009 21:34:11 +0000 (UTC) Received: by ywh37 with SMTP id 37so5692572ywh.28 for ; Tue, 18 Aug 2009 14:34:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=ccIYJ1kICc4J4mYNOnxYwZzLwdV/Va2xkWRJjQA7yTk=; b=foayCMngJV0E6dbGQEeMVgawyqVYa7Zwp7F6jQKhZzogxePBzA9YiVQC6Txjo9WxCA oML12CJIPFTp0qR32p9yfTGFe4zYQq+uIR4s3mdVpvThpw6h3n5ZcPuIv2KmlDuaSgvr JNlMkf9A9SdVl2y7y7x2WTl9lZODjb2MKTyWU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CRHRJY0IojbSmDfC5wFzaF1kl1n0rZu2EatZH8okuSEwu9fD0mIwiS+cid/Z838F4y 0RDd2kkhJnxUwWJitwBHruMjcMUsgVnDGSATtFEX7nuTnoybsfvuF3QQiCyPx4OHFGUd rf6yXSsXil5kbOz+aguVZkeywXjQ0u+Qzp+zQ= MIME-Version: 1.0 Received: by 10.101.80.14 with SMTP id h14mr6096028anl.129.1250631249560; Tue, 18 Aug 2009 14:34:09 -0700 (PDT) In-Reply-To: <4A8B1729.8070503@delphij.net> References: <5D267A3F22FD854F8F48B3D2B523819339EC3813D2@IRVEXCHCCR01.corp.ad.broadcom.com> <692150.91493.qm@web63906.mail.re1.yahoo.com> <2a41acea0908060905n69edf1dfkbf993d9f8a4edf37@mail.gmail.com> <4A8B1729.8070503@delphij.net> Date: Tue, 18 Aug 2009 14:34:09 -0700 Message-ID: <2a41acea0908181434l7a8119ffxbe03a5d312947757@mail.gmail.com> From: Jack Vogel To: d@delphij.net, Sam Leffler Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Barney Cordoba , David Christensen , "freebsd-net@freebsd.org" , Julian Elischer , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 21:34:12 -0000 Yes, this would work, although talking to Sam about this, he said the whole reason for this hack in the driver was some other brokenness in the if code, and he said he was going to fix that, just not before 8 RELEASE. So, I would rather fix it upstream as it where, and then take this hack out of the code altogether instead of refining the hack :) Just my .02 Jack On Tue, Aug 18, 2009 at 2:03 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, Jack, > > I have looked into the code history and found that sys/dev/em/if_em.c,v > 1.119 has introduced the arp_ifinit() call in order to fix the problem > that if_em won't send ARP when IP address is changed. > > I think we can further improve it as attached, say, only do it when > IFF_NOARP is not set. This should have no effect for usual > configuration but fix the problem when NOARP is the desired behavior. > > Cheers, > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.12 (FreeBSD) > > iEYEARECAAYFAkqLFykACgkQi+vbBBjt66CMFQCeOkESwsgDAbqe5PCtiMulaU1E > lIAAoIm0LDJ6qHuR8jyo7dXFi/9iYA22 > =9E54 > -----END PGP SIGNATURE----- > > Index: if_igb.c > =================================================================== > --- if_igb.c (revision 196363) > +++ if_igb.c (working copy) > @@ -952,7 +952,8 @@ igb_ioctl(struct ifnet *ifp, u_long command, caddr > igb_init_locked(adapter); > IGB_CORE_UNLOCK(adapter); > } > - arp_ifinit(ifp, ifa); > + if (!(ifp->if_flags & IFF_NOARP)) > + arp_ifinit(ifp, ifa); > } else > #endif > error = ether_ioctl(ifp, command, data); > Index: if_em.c > =================================================================== > --- if_em.c (revision 196363) > +++ if_em.c (working copy) > @@ -1204,7 +1204,8 @@ em_ioctl(struct ifnet *ifp, u_long command, caddr_ > em_init_locked(adapter); > EM_CORE_UNLOCK(adapter); > } > - arp_ifinit(ifp, ifa); > + if (!(ifp->if_flags & IFF_NOARP)) > + arp_ifinit(ifp, ifa); > } else > #endif > error = ether_ioctl(ifp, command, data); > > From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 21:49:58 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12CDC106568D; Tue, 18 Aug 2009 21:49:58 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.25]) by mx1.freebsd.org (Postfix) with ESMTP id 892298FC52; Tue, 18 Aug 2009 21:49:57 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 3so1317104qwe.7 for ; Tue, 18 Aug 2009 14:49:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=dROb6XgKCn8igHZENjsePND0XdXC8QbaNULH8hzgpfo=; b=uLBsPbva/jUFS7tgiwQNF8xLskQG3ne7LZB+b+CUl64PFYtuDf+4g4MVyN6coXjHso g/mM2s3NdwRnVW5Z22YoSyF/HnOc5sknVaqSiUnr9nlDKa3mt4tZRtJhLahtmbi12kys f8cuZbXcd1i6kQPZiObn2GcRbtxQ5OhrHn8EU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=HM8Rb9i5Uh5QUCJufnJoaw2h55MhviKFpIfuo6dSWGyb9GlC9pn9wMschREYwL4cqO zc39RvskJ1ssQfDbb5szcm5N4W81j7c9kgYYZucfiq7pvlrxot4tMszNMM2m01Qb2jYq ciQk0ZQ/sR3q5/IVApbj+2CqTSVOblC9jK+Q4= Received: by 10.224.36.142 with SMTP id t14mr5961412qad.17.1250632196494; Tue, 18 Aug 2009 14:49:56 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 8sm1893828qwj.6.2009.08.18.14.49.53 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 18 Aug 2009 14:49:55 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 18 Aug 2009 14:49:14 -0700 From: Pyun YongHyeon Date: Tue, 18 Aug 2009 14:49:14 -0700 To: Xin LI Message-ID: <20090818214914.GC15025@michelle.cdnetworks.com> References: <5D267A3F22FD854F8F48B3D2B523819339EC3813D2@IRVEXCHCCR01.corp.ad.broadcom.com> <692150.91493.qm@web63906.mail.re1.yahoo.com> <2a41acea0908060905n69edf1dfkbf993d9f8a4edf37@mail.gmail.com> <4A8B1729.8070503@delphij.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4A8B1729.8070503@delphij.net> User-Agent: Mutt/1.4.2.3i Cc: Barney Cordoba , David Christensen , "d@delphij.net" , "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org, Julian Elischer Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 21:49:58 -0000 On Tue, Aug 18, 2009 at 02:03:37PM -0700, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, Jack, > > I have looked into the code history and found that sys/dev/em/if_em.c,v > 1.119 has introduced the arp_ifinit() call in order to fix the problem > that if_em won't send ARP when IP address is changed. > > I think we can further improve it as attached, say, only do it when > IFF_NOARP is not set. This should have no effect for usual > configuration but fix the problem when NOARP is the desired behavior. > That change was introduced by me. I guess the root cause of the problem was long initialization time of hardware which in turn resulted in unbearable boot time when multiple-alias addresses are assigned to em(4). I don't remember details,though. Since we're in the release cycle, the change you suggested would be quick fix for 8.0. I think em(4)/igb(4) should remove SIOCSIFADDR handling in driver which is layering violation. From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:03:26 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE72A1065672; Tue, 18 Aug 2009 22:03:26 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id 78C858FC66; Tue, 18 Aug 2009 22:03:26 +0000 (UTC) Received: by an-out-0708.google.com with SMTP id d14so1494838and.13 for ; Tue, 18 Aug 2009 15:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=4FVkt6Kj9Y/ewrMqJDyPK7FI5Jrbq9Mlb4uzZQyifz4=; b=LdDwIQ+EEkCOzTYwYcUk3t8Dalji2M5xYk+U2i3ceubDwK40REdF1Dr8tzOjdLIG30 4TtsTSzCGwlZ+yqFSsT4RR51LHLOjbwVRZM5M6rZXlCgkfBFu1gI1vOZbh+zhoUJLjv8 lChuFJ/IVBkWSF8cwFaPboP98A8pQ3f7lT5v4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=JOOKxKoGzak2cp7MyFgoQOZBpTR69snnML2JPfvWGNOzZ8LVS1y3dpc25t4d8HJTXR 22w8id6gvOCqziL2pgDSRkvywMH39yV3HeUcb3GIJhdoqyIoTPffL34f9ohSFx54rDTe E8lFmPch3rAnMGkpuxEr/kVFfN5y+ssBjpZFg= MIME-Version: 1.0 Received: by 10.101.15.6 with SMTP id s6mr6157220ani.120.1250633005338; Tue, 18 Aug 2009 15:03:25 -0700 (PDT) In-Reply-To: <20090818214914.GC15025@michelle.cdnetworks.com> References: <5D267A3F22FD854F8F48B3D2B523819339EC3813D2@IRVEXCHCCR01.corp.ad.broadcom.com> <692150.91493.qm@web63906.mail.re1.yahoo.com> <2a41acea0908060905n69edf1dfkbf993d9f8a4edf37@mail.gmail.com> <4A8B1729.8070503@delphij.net> <20090818214914.GC15025@michelle.cdnetworks.com> Date: Tue, 18 Aug 2009 15:03:25 -0700 Message-ID: <2a41acea0908181503jbb5b335q870e0d2eecbfce05@mail.gmail.com> From: Jack Vogel To: pyunyh@gmail.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Barney Cordoba , David Christensen , "d@delphij.net" , "freebsd-net@freebsd.org" , Julian Elischer , Jack F Vogel , Xin LI , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:03:27 -0000 Yes, I knew it was all your fault :) I was asking about this last week on IRC, without this code in place it will always end up doing a re-init, it should not but if it didn't then another driver broke according to Sam, think it was re. Should have the root cause fixed, but I'm ok with this temporary enhanced hack for now. Jack On Tue, Aug 18, 2009 at 2:49 PM, Pyun YongHyeon wrote: > On Tue, Aug 18, 2009 at 02:03:37PM -0700, Xin LI wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Hi, Jack, > > > > I have looked into the code history and found that sys/dev/em/if_em.c,v > > 1.119 has introduced the arp_ifinit() call in order to fix the problem > > that if_em won't send ARP when IP address is changed. > > > > I think we can further improve it as attached, say, only do it when > > IFF_NOARP is not set. This should have no effect for usual > > configuration but fix the problem when NOARP is the desired behavior. > > > > That change was introduced by me. I guess the root cause of the > problem was long initialization time of hardware which in turn > resulted in unbearable boot time when multiple-alias addresses are > assigned to em(4). I don't remember details,though. > > Since we're in the release cycle, the change you suggested would be > quick fix for 8.0. I think em(4)/igb(4) should remove SIOCSIFADDR > handling in driver which is layering violation. > From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:16:07 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76F9D106568B for ; Tue, 18 Aug 2009 22:16:07 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63907.mail.re1.yahoo.com (web63907.mail.re1.yahoo.com [69.147.97.122]) by mx1.freebsd.org (Postfix) with SMTP id 30D6D8FC6B for ; Tue, 18 Aug 2009 22:16:06 +0000 (UTC) Received: (qmail 52563 invoked by uid 60001); 18 Aug 2009 22:16:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250633766; bh=JIhdzu6PSvZHVTDDszqd1jHyLPaIIM6s9MUXLm3h0vQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=cmeVSnKVFqYOde5wfu+iEoksTZXy3/MyLNjKvh3rZywmKDoMsZzr2EtOOMvrHmmjIaC0vUcYBpPtpq5KFvfBMTygn9Gk+wf8Vtz/xH1viDLh799rRl8c2YqydI4X/Pmxo7xaP3E9leuywRvgYumzV8hl74umnAqC542EJAPRxLQ= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=DqaibM4/mlFo0f69sxX9uYv05w/HECrRuhzM9JqJdzzWdsw5GIpWFJL0BKisGMMXOQk7o1KTKZeIIDSk87QWp7rsWMfS96/YWXrPZDbd0XjHRo+bANEO8PZ2KZLW03vtKGNbo6jZT7PakFa6oPsPCQSBFKTpXSb38/fuLHgcGKw=; Message-ID: <373149.52091.qm@web63907.mail.re1.yahoo.com> X-YMail-OSG: JwOWzIAVM1nL_S3QvTKG5grU2hdWiMAGXiW9rcxg_k5bm4PckhBfvCgckyWz8hNiyDzCCno2Tpq.WZ2qSPUUEp4jDjlRw4sLdRCE8KxX9DhLRY35aVJd.MOg7onGahvOdyJfDD7RW_GyiVoU578K2_mX50W07ZzJ0kKbvtKDpUPEWwSfYxSCyl7I7WnIzym1hlanhe3ObKVjV4h6ucZAA3BG9rED9cjOaRGlhfWqQZbrFZDsrip1igWx72eNW0Ha4kuoKp04njmkQsxNhKqFYDUgG6k- Received: from [66.176.162.245] by web63907.mail.re1.yahoo.com via HTTP; Tue, 18 Aug 2009 15:16:06 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.2 Date: Tue, 18 Aug 2009 15:16:06 -0700 (PDT) From: Barney Cordoba To: freebsd-net@freebsd.org, Manish Vachharajani In-Reply-To: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:16:07 -0000 =0A=0A--- On Mon, 8/17/09, Manish Vachharajani wrote:=0A=0A> From: Manish Vachharajani =0A>= Subject: Dropped vs. missed packets in the ixgbe driver=0A> To: freebsd-ne= t@freebsd.org=0A> Date: Monday, August 17, 2009, 6:24 PM=0A> I've been doin= g some performance=0A> testing on freebsd 7.2 and noticed=0A> that the ixgb= e driver does not report missed packets as=0A> dropped when=0A> queried via= netstat -id (the ixgbe driver in the 8.0=A0=0A> beta has a=0A> similar iss= ue).=A0 A missed packet is a packet that was=0A> correctly=0A> received by = the NIC but because it was out of descriptors=0A> and internal=0A> memory, = the packet had to be dropped by the NIC itself -- a=0A> hardware=0A> regist= er counts such events.=A0 Instead the driver only=0A> reports drops=0A> tha= t are due to the driver itself, i.e., the NIC DMAed the=0A> packet to=0A> m= emory but the driver had to drop something because it was=0A> out of=0A> mb= ufs.=A0 Is the miss count reported elsewhere (besides=0A> via a kernel=0A> = printf from the driver)?=A0 At the end of the email I=0A> give a stats dump= =0A> from the driver and from netstat, note the number of=0A> packets dropp= ed=0A> is 0 as reported by netstat but the Missed field in the=0A> dmesg ou= tput=0A> shows many missed packets.=0A> =0A> >From my perspective, it is di= sconcerting to see=0A> performance=0A> degradation on the link, along with = TCP ack retransmits,=0A> packet=0A> reordering, etc. (on a point-to-point l= ink with no switch=0A> in between)=0A> but then see no drops reported by ne= tstat because the=0A> driver didn't=0A> drop the packet, the NIC did.=A0 Th= e fix should be=0A> straight-forward and=0A> I'll gladly make a patch assum= ing that it is indeed a bug=0A> and not a=0A> conscious design choice.=0A> = =0A> Here is the relevant netstat output=0A> =0A> Name=A0 =A0 Mtu Network= =A0 =A0=0A> =A0=A0=A0Address=A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 Ipkts Ierrs=A0 = =A0 Opkts=0A> Oerrs=A0 Coll Drop=0A> ix0=A0 =A0 1500 =A0 =A0 =A0=0A= > 00:30:48:94:60:ec=A0 =A0 =A0 =A0 0=A0=0A> =A0=A0=A00=A0 =A0 =A0 =A0 1=0A>= 0=A0 =A0=A0=A00=A0 =A0 0=0A> ix0=A0 =A0 1500 192.168.105.0 192.168.105.2= =A0=0A> =A0 =A0 =A0 =A0 =A0 0=A0=0A> =A0=A0=A0-=A0 =A0 =A0 =A0 0=0A> -=A0 = =A0=A0=A0-=A0 =A0 -=0A> ix1=A0 =A0 1500 =A0 =A0 =A0=0A> 00:30:48:94= :60:ed=A0 =A0 =A0 11M=A0=0A> =A0=A0=A00=A0 =A0=A0=A06.1M=0A> 0=A0 =A0=A0= =A00=A0 =A0 0=0A> ix1=A0 =A0 1500=0A> 192.168.5.0=A0=A0=A0192.168.5.2=A0 = =A0 =A0=0A> =A0 =A0 =A0 10M=A0 =A0=A0=A0-=A0=0A> =A0=A0=A06.1M=0A> -=A0 = =A0=A0=A0-=A0 =A0 -=0A> =0A> And here is the dmesg output after doing a sys= ctl=0A> dev.ix.1.stats=3D1=0A> =0A> ix1: Std Mbuf Failed =3D 0=0A> ix1: Mis= sed Packets =3D 413872=0A> ix1: Receive length errors =3D 0=0A> ix1: Crc er= rors =3D 0=0A> ix1: Driver dropped packets =3D 0=0A> ix1: watchdog timeouts= =3D 0=0A> ix1: XON Rcvd =3D 616428212235=0A> ix1: XON Xmtd =3D 0=0A> ix1: = XOFF Rcvd =3D 616428212235=0A> ix1: XOFF Xmtd =3D 0=0A> ix1: Total Packets = Rcvd =3D 12424533=0A> ix1: Good Packets Rcvd =3D 12010661=0A> ix1: Good Pac= kets Xmtd =3D 6419128=0A> ix1: TSO Transmissions =3D 0=0A> =0A> Manish=0A= =0Athe debug sysctl show more interesting info. Don't get too excited=0Aabo= ut doing performance testing with that driver. Its not designed=0Ato be any= higher in performance than any of the other intel drivers.=0A=0ABarney=0A= =0A=0A From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:32:29 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7D36106568F for ; Tue, 18 Aug 2009 22:32:28 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n4.bullet.mail.ac4.yahoo.com (n4.bullet.mail.ac4.yahoo.com [76.13.13.28]) by mx1.freebsd.org (Postfix) with SMTP id 89BF78FC45 for ; Tue, 18 Aug 2009 22:32:28 +0000 (UTC) Received: from [76.13.13.25] by n4.bullet.mail.ac4.yahoo.com with NNFMP; 18 Aug 2009 22:19:36 -0000 Received: from [76.13.10.175] by t4.bullet.mail.ac4.yahoo.com with NNFMP; 18 Aug 2009 22:19:36 -0000 Received: from [127.0.0.1] by omp116.mail.ac4.yahoo.com with NNFMP; 18 Aug 2009 22:19:36 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 604978.63872.bm@omp116.mail.ac4.yahoo.com Received: (qmail 21521 invoked by uid 60001); 18 Aug 2009 22:19:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250633976; bh=GPBduJ1ifKYvdDMBAqKMI/o2T8UPnBjc/nW5GU8f/30=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Yv5LLiu2xZqTJimVmPrHYDDBexlcarJ5Bd/ri195p5cfeSlQlnTSDw767m6lp5+sx8skJKpv7coaKQ8yUDnkg4Dj414XMy703S0wRLUcUPBE1dO7tQbL4UGXsKuROC08HMEV0lexyLJsSir17RAq7c4NETs5Ky/8pgIkn7P8OCo= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=bSh7eW9T991/AekDcmuJxJCZKIkkoTGRNGncNFtSSp88FzMhMeCVHN3adGjz2oylvA/is7EUqk/XMv4QA+z+jOZyGBiOWmlnFwCvc3QIa/5C4WRh26+Carmz9NR/eDWr6iw823zpmcF0TTN6B78H4d9GY0W26ljYuSlujndn0GM=; Message-ID: <527700.21341.qm@web63902.mail.re1.yahoo.com> X-YMail-OSG: 9O.kGtAVM1kwGEC_kq4oWhrH6X8XYrnZCiExkCfu1voV_yWVu8uRl.7P0jUE1iukDNuUm4Yg2H9McAV1FyyUDGT3IpIWfg196D5I.DN4MjWek.RH4UjTdOi33TJFN5wsu9XVlSHhrsx1zdEVhnOqrAgl9.F2UXSMKEHPLvmKItrjtpJTlfhCFwIHGSIaGXnQPnNMrFaRU6uqEkl2SCbc2grJAgSi8mundQgfn4BEBpXo7yIVzUEZ14NJvb68MSfTPWo29lfbSqKIcZ0w2raojnWY2H9qxMEAKuft.DI3VoILOvGPt8ZmG_bUoebayg-- Received: from [66.176.162.245] by web63902.mail.re1.yahoo.com via HTTP; Tue, 18 Aug 2009 15:19:36 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Tue, 18 Aug 2009 15:19:36 -0700 (PDT) From: Barney Cordoba To: Xin LI , pyunyh@gmail.com In-Reply-To: <20090818214914.GC15025@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: David Christensen , "d@delphij.net" , Julian Elischer , "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:32:29 -0000 =0A=0A--- On Tue, 8/18/09, Pyun YongHyeon wrote:=0A=0A> = From: Pyun YongHyeon =0A> Subject: Re: [PATCH] Fix for e1= 000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP fl= ag]=0A> To: "Xin LI" =0A> Cc: "Barney Cordoba" , "David Christensen" , "d@delphij= ..net" , "freebsd-net@freebsd.org" ,= "Jack Vogel" , "Jack F Vogel" , yongar= i@freebsd.org, "Julian Elischer" =0A> Date: Tuesday, A= ugust 18, 2009, 5:49 PM=0A> On Tue, Aug 18, 2009 at 02:03:37PM=0A> -0700, X= in LI wrote:=0A> > -----BEGIN PGP SIGNED MESSAGE-----=0A> > Hash: SHA1=0A> = > =0A> > Hi, Jack,=0A> > =0A> > I have looked into the code history and fou= nd that=0A> sys/dev/em/if_em.c,v=0A> > 1.119 has introduced the arp_ifinit(= ) call in order to=0A> fix the problem=0A> > that if_em won't send ARP when= IP address is changed.=0A> > =0A> > I think we can further improve it as a= ttached, say,=0A> only do it when=0A> > IFF_NOARP is not set.=A0 This shoul= d have no effect=0A> for usual=0A> > configuration but fix the problem when= NOARP is the=0A> desired behavior.=0A> > =0A> =0A> That change was introdu= ced by me. I guess the root cause of=0A> the=0A> problem was long initializ= ation time of hardware which in=0A> turn=0A> resulted in unbearable boot ti= me when multiple-alias=0A> addresses are=0A> assigned to em(4). I don't rem= ember details,though.=0A> =0A> Since we're in the release cycle, the change= you suggested=0A> would be=0A> quick fix for 8.0. I think em(4)/igb(4) sho= uld remove=0A> SIOCSIFADDR=0A> handling in driver which is layering violati= on.=0A=0AThere are 2 kinds of programmers; those who do things "correctly',= =0Aand those that do things that work. =0A=0A99.99999% of the people will b= e using ARPs, so don't be silly and=0Abreak the driver to solve a case that= almost no-one cares about please.=0A=0ABarney=0A=0A=0A From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:35:08 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F1EC106568D for ; Tue, 18 Aug 2009 22:35:08 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 260768FC59 for ; Tue, 18 Aug 2009 22:35:07 +0000 (UTC) Received: by vws10 with SMTP id 10so3427004vws.7 for ; Tue, 18 Aug 2009 15:35:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.82.2 with SMTP id z2mr7903159vck.0.1250634904901; Tue, 18 Aug 2009 15:35:04 -0700 (PDT) In-Reply-To: <373149.52091.qm@web63907.mail.re1.yahoo.com> References: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> <373149.52091.qm@web63907.mail.re1.yahoo.com> Date: Tue, 18 Aug 2009 16:35:04 -0600 Message-ID: <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> From: Manish Vachharajani To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:35:08 -0000 Indeed the debugging info is also interesting. However, I'd like to get some data from netstat when the driver drops frames. When looking at the bge driver source, it appears that all input drops at the NIC are reported as input errors. It appears that the intel drivers (e1000, ixgb, and ixgbe) don't report that number outside of the debug and stats printfs at all, and this seems broken. What I want to know is if I have just missed where these are reported. So, in a nutshell, the question is: should these drivers be reporting miss events as input errors in the ifnet struct as the bge driver does, or as drops in the ifnet struct, was there some conscious decision not to report miss events anywhere outside the debug and stats info, or am I just being silly and not seeing where the numbers are reported? Also, don't worry on the performance front, we are also looking at the driver in FreeBSD 8.0 :) which supports RSS to help performance scaling, though we have some interesting data there that I'll post about once I confirm that the numbers are indeed correct and not a tuning or setup problem. Manish > --- On Mon, 8/17/09, Manish Vachharajani wr= ote: > >> From: Manish Vachharajani >> Subject: Dropped vs. missed packets in the ixgbe driver >> To: freebsd-net@freebsd.org >> Date: Monday, August 17, 2009, 6:24 PM >> I've been doing some performance >> testing on freebsd 7.2 and noticed >> that the ixgbe driver does not report missed packets as >> dropped when >> queried via netstat -id (the ixgbe driver in the 8.0 >> beta has a >> similar issue).=A0 A missed packet is a packet that was >> correctly >> received by the NIC but because it was out of descriptors >> and internal >> memory, the packet had to be dropped by the NIC itself -- a >> hardware >> register counts such events.=A0 Instead the driver only >> reports drops >> that are due to the driver itself, i.e., the NIC DMAed the >> packet to >> memory but the driver had to drop something because it was >> out of >> mbufs.=A0 Is the miss count reported elsewhere (besides >> via a kernel >> printf from the driver)?=A0 At the end of the email I >> give a stats dump >> from the driver and from netstat, note the number of >> packets dropped >> is 0 as reported by netstat but the Missed field in the >> dmesg output >> shows many missed packets. >> >> >From my perspective, it is disconcerting to see >> performance >> degradation on the link, along with TCP ack retransmits, >> packet >> reordering, etc. (on a point-to-point link with no switch >> in between) >> but then see no drops reported by netstat because the >> driver didn't >> drop the packet, the NIC did.=A0 The fix should be >> straight-forward and >> I'll gladly make a patch assuming that it is indeed a bug >> and not a >> conscious design choice. >> >> Here is the relevant netstat output >> >> Name=A0 =A0 Mtu Network >> =A0=A0=A0Address >> =A0 =A0 Ipkts Ierrs=A0 =A0 Opkts >> Oerrs=A0 Coll Drop >> ix0=A0 =A0 1500 >> 00:30:48:94:60:ec=A0 =A0 =A0 =A0 0 >> =A0=A0=A00=A0 =A0 =A0 =A0 1 >> =A00=A0 =A0=A0=A00=A0 =A0 0 >> ix0=A0 =A0 1500 192.168.105.0 192.168.105.2 >> =A0 =A0 =A0 =A0 =A0 0 >> =A0=A0=A0-=A0 =A0 =A0 =A0 0 >> =A0-=A0 =A0=A0=A0-=A0 =A0 - >> ix1=A0 =A0 1500 >> 00:30:48:94:60:ed=A0 =A0 =A0 11M >> =A0=A0=A00=A0 =A0=A0=A06.1M >> =A00=A0 =A0=A0=A00=A0 =A0 0 >> ix1=A0 =A0 1500 >> 192.168.5.0=A0=A0=A0192.168.5.2 >> =A0 =A0 =A0 10M=A0 =A0=A0=A0- >> =A0=A0=A06.1M >> =A0-=A0 =A0=A0=A0-=A0 =A0 - >> >> And here is the dmesg output after doing a sysctl >> dev.ix.1.stats=3D1 >> >> ix1: Std Mbuf Failed =3D 0 >> ix1: Missed Packets =3D 413872 >> ix1: Receive length errors =3D 0 >> ix1: Crc errors =3D 0 >> ix1: Driver dropped packets =3D 0 >> ix1: watchdog timeouts =3D 0 >> ix1: XON Rcvd =3D 616428212235 >> ix1: XON Xmtd =3D 0 >> ix1: XOFF Rcvd =3D 616428212235 >> ix1: XOFF Xmtd =3D 0 >> ix1: Total Packets Rcvd =3D 12424533 >> ix1: Good Packets Rcvd =3D 12010661 >> ix1: Good Packets Xmtd =3D 6419128 >> ix1: TSO Transmissions =3D 0 >> >> Manish > > the debug sysctl show more interesting info. Don't get too excited > about doing performance testing with that driver. Its not designed > to be any higher in performance than any of the other intel drivers. > > Barney > > > > --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:51:20 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CF5C106568B; Tue, 18 Aug 2009 22:51:20 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5DAE18FC64; Tue, 18 Aug 2009 22:51:18 +0000 (UTC) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 6894A5C06F; Wed, 19 Aug 2009 06:51:16 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 2DC8A55CDC93; Wed, 19 Aug 2009 06:51:16 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id OsXNu-VMTdCN; Wed, 19 Aug 2009 06:50:22 +0800 (CST) Received: from charlie.delphij.net (adsl-76-237-33-62.dsl.pltn13.sbcglobal.net [76.237.33.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 6202855CDC88; Wed, 19 Aug 2009 06:50:11 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=AUEOSaaD+NfOGzbHy3noZ2kFRcNnofy1xJclK+BShPaX/7bgg6gy/AS7hlwb2SvKk V802FhXSCsO8S0okOJ8mA== Message-ID: <4A8B3011.6070104@delphij.net> Date: Tue, 18 Aug 2009 15:49:53 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.22 (X11/20090803) MIME-Version: 1.0 To: Barney Cordoba References: <527700.21341.qm@web63902.mail.re1.yahoo.com> In-Reply-To: <527700.21341.qm@web63902.mail.re1.yahoo.com> X-Enigmail-Version: 0.96.0 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, David Christensen , "d@delphij.net" , Julian Elischer , "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:51:20 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Barney Cordoba wrote: > > --- On Tue, 8/18/09, Pyun YongHyeon wrote: > >> From: Pyun YongHyeon >> Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] >> To: "Xin LI" >> Cc: "Barney Cordoba" , "David Christensen" , "d@delphij..net" , "freebsd-net@freebsd.org" , "Jack Vogel" , "Jack F Vogel" , yongari@freebsd.org, "Julian Elischer" >> Date: Tuesday, August 18, 2009, 5:49 PM >> On Tue, Aug 18, 2009 at 02:03:37PM >> -0700, Xin LI wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Hi, Jack, >>> >>> I have looked into the code history and found that >> sys/dev/em/if_em.c,v >>> 1.119 has introduced the arp_ifinit() call in order to >> fix the problem >>> that if_em won't send ARP when IP address is changed. >>> >>> I think we can further improve it as attached, say, >> only do it when >>> IFF_NOARP is not set. This should have no effect >> for usual >>> configuration but fix the problem when NOARP is the >> desired behavior. >> That change was introduced by me. I guess the root cause of >> the >> problem was long initialization time of hardware which in >> turn >> resulted in unbearable boot time when multiple-alias >> addresses are >> assigned to em(4). I don't remember details,though. >> >> Since we're in the release cycle, the change you suggested >> would be >> quick fix for 8.0. I think em(4)/igb(4) should remove >> SIOCSIFADDR >> handling in driver which is layering violation. > > There are 2 kinds of programmers; those who do things "correctly', > and those that do things that work. > > 99.99999% of the people will be using ARPs, so don't be silly and > break the driver to solve a case that almost no-one cares about please. I see no reason how you have reached the "99.99999%" conclusion. My employer for instance, has several millions of dollars worth of hardware purchase every year, and, we do care about DSR, or NOARP being working or not. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkqLMBEACgkQi+vbBBjt66DtJACcCuMIEljhYtKT/B9xP18HYzLD gMwAmwQpJiVSzFJzgXoNggWdRF/kj2Qs =ROT8 -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:55:46 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DE66106568F for ; Tue, 18 Aug 2009 22:55:46 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outX.internet-mail-service.net (outx.internet-mail-service.net [216.240.47.247]) by mx1.freebsd.org (Postfix) with ESMTP id E0AF88FC16 for ; Tue, 18 Aug 2009 22:55:45 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 68D09B3F80; Tue, 18 Aug 2009 15:55:46 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 106742D6011; Tue, 18 Aug 2009 15:55:44 -0700 (PDT) Message-ID: <4A8B316F.3030408@elischer.org> Date: Tue, 18 Aug 2009 15:55:43 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: d@delphij.net References: <527700.21341.qm@web63902.mail.re1.yahoo.com> <4A8B3011.6070104@delphij.net> In-Reply-To: <4A8B3011.6070104@delphij.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, Barney Cordoba , David Christensen , "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:55:46 -0000 Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Barney Cordoba wrote: >> --- On Tue, 8/18/09, Pyun YongHyeon wrote: >> >>> From: Pyun YongHyeon >>> Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] >>> To: "Xin LI" >>> Cc: "Barney Cordoba" , "David Christensen" , "d@delphij..net" , "freebsd-net@freebsd.org" , "Jack Vogel" , "Jack F Vogel" , yongari@freebsd.org, "Julian Elischer" >>> Date: Tuesday, August 18, 2009, 5:49 PM >>> On Tue, Aug 18, 2009 at 02:03:37PM >>> -0700, Xin LI wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> Hi, Jack, >>>> >>>> I have looked into the code history and found that >>> sys/dev/em/if_em.c,v >>>> 1.119 has introduced the arp_ifinit() call in order to >>> fix the problem >>>> that if_em won't send ARP when IP address is changed. >>>> >>>> I think we can further improve it as attached, say, >>> only do it when >>>> IFF_NOARP is not set. This should have no effect >>> for usual >>>> configuration but fix the problem when NOARP is the >>> desired behavior. >>> That change was introduced by me. I guess the root cause of >>> the >>> problem was long initialization time of hardware which in >>> turn >>> resulted in unbearable boot time when multiple-alias >>> addresses are >>> assigned to em(4). I don't remember details,though. >>> >>> Since we're in the release cycle, the change you suggested >>> would be >>> quick fix for 8.0. I think em(4)/igb(4) should remove >>> SIOCSIFADDR >>> handling in driver which is layering violation. >> There are 2 kinds of programmers; those who do things "correctly', >> and those that do things that work. >> >> 99.99999% of the people will be using ARPs, so don't be silly and >> break the driver to solve a case that almost no-one cares about please. > Cisco.Ironport runs 50% (2 out of 4) of their em interfaces in noarp mode. please keep noarp working! > I see no reason how you have reached the "99.99999%" conclusion. My > employer for instance, has several millions of dollars worth of hardware > purchase every year, and, we do care about DSR, or NOARP being working > or not. > > Cheers, > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.12 (FreeBSD) > > iEYEARECAAYFAkqLMBEACgkQi+vbBBjt66DtJACcCuMIEljhYtKT/B9xP18HYzLD > gMwAmwQpJiVSzFJzgXoNggWdRF/kj2Qs > =ROT8 > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 02:55:23 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9337106568B for ; Wed, 19 Aug 2009 02:55:23 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-gx0-f227.google.com (mail-gx0-f227.google.com [209.85.217.227]) by mx1.freebsd.org (Postfix) with ESMTP id 8FC868FC16 for ; Wed, 19 Aug 2009 02:55:23 +0000 (UTC) Received: by gxk27 with SMTP id 27so5331410gxk.12 for ; Tue, 18 Aug 2009 19:55:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Nqm2WqL82KaHD5yi9V+Lm0oTeLd/gzVjHLyoovmn+9s=; b=kHCQkJS1w/RLzCnCey8zhU86SDgQ0Bm+vCbtyoZ/XTvAWaaf41gnLxKXRpzGNIOnQl wuErnsVmRAKjY71jfDhrpMK7UKWmrxgDq/dEdRJ4srS9rVvmxgXmxPYEEB6EyVKf34Fn m+dNwAALQnzlAWxuK2S8anxhY2s+6eLU8i+XA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=oQ0YoCTlMAkJMk09wfUlCiqg+8veaGS/14haBgso8he/NOR+tu+WJ6VcEhMBSwy20c aRhLC0Q6KwzgiWGuikpgAsU5OOb7D/5QlUVZBsT55EMIZZM66W7+dLvJq9Vm5Ch/O9Kq sdfFdouCuvDpBBQMrd2fdHCxdDBstARBvPg3s= MIME-Version: 1.0 Received: by 10.150.235.17 with SMTP id i17mr9403995ybh.200.1250650522830; Tue, 18 Aug 2009 19:55:22 -0700 (PDT) In-Reply-To: References: <2a41acea0905020803s63b69b1awb39538f000f5bd5a@mail.gmail.com> <2a41acea0906261725x57e6903br9f3f42b55f3a3d30@mail.gmail.com> <688430.20427.qm@web37906.mail.mud.yahoo.com> <2a41acea0906280952s23d6553ep42fcfd4671561c3a@mail.gmail.com> <2a41acea0907071722p7992bea0s281399cb0baecd90@mail.gmail.com> Date: Tue, 18 Aug 2009 21:55:22 -0500 Message-ID: <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> From: "Sam Fourman Jr." To: Mark Atkinson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Regression: em driver in -CURRENT, "Invalid MAC address" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 02:55:24 -0000 On Mon, Jul 13, 2009 at 12:25 PM, Mark Atkinson wrote: > Jack Vogel wrote: >> In case you hadn't seen it, the code that fixes this is now checked into >> the tip, so the latest em driver should work for you. > > I upgraded the machine in question this morning and it appears to be > working, thanks! Jack Would you be able to commit this patch to em(4) I have several machines that do not work on FreeBSD 8 BETA2 With the 82543 em(4) based card. I copied the same approach you took on e1000_82542.c Thank you Sam Fourman Jr. pciconf -v -l |grep -A4 -e "^em" em0@pci0:14:13:0: class=0x020000 card=0x10038086 chip=0x10018086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82543GC Gigabit Ethernet Adapter (Fiber)' class = network subclass = ethernet I tested this patch and it works on today's source tree **** e1000_82543.c Wed Nov 26 17:57:23 2008 --- /root/e1000_82543.c Tue Aug 18 21:23:00 2009 *************** *** 75,80 **** --- 75,81 ---- u16 count); static bool e1000_tbi_compatibility_enabled_82543(struct e1000_hw *hw); static void e1000_set_tbi_sbp_82543(struct e1000_hw *hw, bool state); + static s32 e1000_read_mac_addr_82543(struct e1000_hw *hw); /** * e1000_init_phy_params_82543 - Init PHY func ptrs. *************** *** 246,251 **** --- 247,254 ---- mac->ops.clear_vfta = e1000_clear_vfta_generic; /* setting MTA */ mac->ops.mta_set = e1000_mta_set_82543; + /* read mac address */ + mac->ops.read_mac_addr = e1000_read_mac_addr_82543; /* turn on/off LED */ mac->ops.led_on = e1000_led_on_82543; mac->ops.led_off = e1000_led_off_82543; *************** *** 1600,1602 **** --- 1603,1638 ---- E1000_READ_REG(hw, E1000_TSCTC); E1000_READ_REG(hw, E1000_TSCTFC); } + + /** + * e1000_read_mac_addr_82543 - Read device MAC address + * @hw: pointer to the HW structure + * Same approach was taken for the 82542 + * + * Reads the device MAC address from the EEPROM and stores the value. + **/ + static s32 e1000_read_mac_addr_82543(struct e1000_hw *hw) + { + s32 ret_val = E1000_SUCCESS; + u16 offset, nvm_data, i; + + DEBUGFUNC("e1000_read_mac_addr"); + + for (i = 0; i < ETH_ADDR_LEN; i += 2) { From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 06:06:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B8C7106568D for ; Wed, 19 Aug 2009 06:06:41 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.242]) by mx1.freebsd.org (Postfix) with ESMTP id 522C88FC61 for ; Wed, 19 Aug 2009 06:06:41 +0000 (UTC) Received: by an-out-0708.google.com with SMTP id d14so1569880and.13 for ; Tue, 18 Aug 2009 23:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=rdBsZqFRYlGV/gceMK0nnZb1IR6mqIo96q0RC5bDZsg=; b=PhbJP3rrDVlRvIFw4cnG4IF736AjHqAV6uF+Oz8wCqIOsbo16GJ+I34EnaDpr7S3nr tWRufdfqUGXIjC9WzdEvjKBQaEGKYtJFihu0mcOlHWVTSpETqtdsS6gBxnbpCOxRJcYJ SJZDqmyD2ffoC508maqNJRpM7XkimsY3EGuIg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=eTI5cACYcZi/HnxK7fbDjATpJSsL1HwqkcdAg4sYNbzRqTo6/xzDK3hPGTvccIEQTp A493rJg57DzApbNY/O8w+1TSzXfz8HLsZRXYPfmRZQYTLZ46b59fXYhXaTnA2EeMMwCj vw8C68OcH3yvOQToJrzlPqGh/CDM9G1kgRehM= MIME-Version: 1.0 Received: by 10.100.40.17 with SMTP id n17mr6523942ann.187.1250662000582; Tue, 18 Aug 2009 23:06:40 -0700 (PDT) In-Reply-To: <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> References: <2a41acea0905020803s63b69b1awb39538f000f5bd5a@mail.gmail.com> <2a41acea0906261725x57e6903br9f3f42b55f3a3d30@mail.gmail.com> <688430.20427.qm@web37906.mail.mud.yahoo.com> <2a41acea0906280952s23d6553ep42fcfd4671561c3a@mail.gmail.com> <2a41acea0907071722p7992bea0s281399cb0baecd90@mail.gmail.com> <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> Date: Tue, 18 Aug 2009 23:06:40 -0700 Message-ID: <2a41acea0908182306h740194bcw4de78b6093998e88@mail.gmail.com> From: Jack Vogel To: "Sam Fourman Jr." Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Mark Atkinson Subject: Re: Regression: em driver in -CURRENT, "Invalid MAC address" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 06:06:41 -0000 Hmmm, that's odd, I had the validation group do a thorough test of older adapters, and that one did not seem to have a problem, let me see if we can repro the issue tomorrow, but, in principle I have no problem doing this. Jack On Tue, Aug 18, 2009 at 7:55 PM, Sam Fourman Jr. wrote: > On Mon, Jul 13, 2009 at 12:25 PM, Mark Atkinson wrote: > > Jack Vogel wrote: > >> In case you hadn't seen it, the code that fixes this is now checked into > >> the tip, so the latest em driver should work for you. > > > > I upgraded the machine in question this morning and it appears to be > > working, thanks! > > > Jack > > Would you be able to commit this patch to em(4) I have several > machines that do not work on FreeBSD 8 BETA2 > With the 82543 em(4) based card. I copied the same approach you took > on e1000_82542.c > > Thank you > > Sam Fourman Jr. > > > pciconf -v -l |grep -A4 -e "^em" > > em0@pci0:14:13:0: class=0x020000 card=0x10038086 chip=0x10018086 > rev=0x02 hdr=0x00 > vendor = 'Intel Corporation' > device = '82543GC Gigabit Ethernet Adapter (Fiber)' > class = network > subclass = ethernet > > > > I tested this patch and it works on today's source tree > > > **** e1000_82543.c Wed Nov 26 17:57:23 2008 > --- /root/e1000_82543.c Tue Aug 18 21:23:00 2009 > *************** > *** 75,80 **** > --- 75,81 ---- > u16 count); > static bool e1000_tbi_compatibility_enabled_82543(struct e1000_hw *hw); > static void e1000_set_tbi_sbp_82543(struct e1000_hw *hw, bool state); > + static s32 e1000_read_mac_addr_82543(struct e1000_hw *hw); > > /** > * e1000_init_phy_params_82543 - Init PHY func ptrs. > *************** > *** 246,251 **** > --- 247,254 ---- > mac->ops.clear_vfta = e1000_clear_vfta_generic; > /* setting MTA */ > mac->ops.mta_set = e1000_mta_set_82543; > + /* read mac address */ > + mac->ops.read_mac_addr = e1000_read_mac_addr_82543; > /* turn on/off LED */ > mac->ops.led_on = e1000_led_on_82543; > mac->ops.led_off = e1000_led_off_82543; > *************** > *** 1600,1602 **** > --- 1603,1638 ---- > E1000_READ_REG(hw, E1000_TSCTC); > E1000_READ_REG(hw, E1000_TSCTFC); > } > + > + /** > + * e1000_read_mac_addr_82543 - Read device MAC address > + * @hw: pointer to the HW structure > + * Same approach was taken for the 82542 > + * > + * Reads the device MAC address from the EEPROM and stores the value. > + **/ > + static s32 e1000_read_mac_addr_82543(struct e1000_hw *hw) > + { > + s32 ret_val = E1000_SUCCESS; > + u16 offset, nvm_data, i; > + > + DEBUGFUNC("e1000_read_mac_addr"); > + > + for (i = 0; i < ETH_ADDR_LEN; i += 2) { > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 06:58:04 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3F83106568C for ; Wed, 19 Aug 2009 06:58:03 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-yx0-f181.google.com (mail-yx0-f181.google.com [209.85.210.181]) by mx1.freebsd.org (Postfix) with ESMTP id A82858FC55 for ; Wed, 19 Aug 2009 06:58:03 +0000 (UTC) Received: by yxe11 with SMTP id 11so5482684yxe.3 for ; Tue, 18 Aug 2009 23:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=WCtZutyxpJ8+Y7sEmaxxaUZiL4v5YdMUuGmUsthfczw=; b=ICnG5St6UVstpe6px++HtRWYYYxGRKpx1Cf+SUezbyz1fGP8ICdWkjmvv9AQUQjw+D rCK/EaTyuuurfwxDQORYWyfSGsejD1NxnXZxtE9wcpkIWg0agaGDvhBtF5jRRZxXc5yu eV4TszMkcLVoOy9enhtxJGC5zE9h+XETR4Mc4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=YQs6FqHtcWV2oS8tDVBA0M5VnxD/8RIPJXZLBkbvvvvbGIF997gjgvexgtIuUpN7E2 7gu1bIxTPMwqDYxUxGpMMg2QdxKD550OX63zfu4DKsAbZNBFpDNHNqK0ACvIPjBiYItC 7XIJ1OqzGSTrlSFZc7x3pzLAk+gy8CvBb6G3E= MIME-Version: 1.0 Received: by 10.150.61.3 with SMTP id j3mr9754116yba.76.1250665082949; Tue, 18 Aug 2009 23:58:02 -0700 (PDT) In-Reply-To: <2a41acea0908182306h740194bcw4de78b6093998e88@mail.gmail.com> References: <2a41acea0905020803s63b69b1awb39538f000f5bd5a@mail.gmail.com> <2a41acea0906261725x57e6903br9f3f42b55f3a3d30@mail.gmail.com> <688430.20427.qm@web37906.mail.mud.yahoo.com> <2a41acea0906280952s23d6553ep42fcfd4671561c3a@mail.gmail.com> <2a41acea0907071722p7992bea0s281399cb0baecd90@mail.gmail.com> <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> <2a41acea0908182306h740194bcw4de78b6093998e88@mail.gmail.com> Date: Wed, 19 Aug 2009 01:58:02 -0500 Message-ID: <11167f520908182358hc7a176as962788b505b87148@mail.gmail.com> From: "Sam Fourman Jr." To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Mark Atkinson Subject: Re: Regression: em driver in -CURRENT, "Invalid MAC address" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 06:58:04 -0000 On Wed, Aug 19, 2009 at 1:06 AM, Jack Vogel wrote: > Hmmm, that's odd, I had the validation group do a thorough test > of older adapters, and that one did not seem to have a problem, > let me see if we can repro the issue tomorrow, but, in principle I > have no problem doing this. > > Jack I can provide root ssh access to these with a serial console and public IP's if you need I have over 100 of these (old dell P3 2450's with whatever em(4) fibre nic dell shipped) sitting on the shelf collecting dust. Just send me a heads up and I will have someone set one up for you right away. Sam Fourman Jr. Fourman Networks From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 09:24:53 2009 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F348A106568D for ; Wed, 19 Aug 2009 09:24:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail01.syd.optusnet.com.au (mail01.syd.optusnet.com.au [211.29.132.182]) by mx1.freebsd.org (Postfix) with ESMTP id 89AF58FC43 for ; Wed, 19 Aug 2009 09:24:52 +0000 (UTC) Received: from c122-106-152-1.carlnfd1.nsw.optusnet.com.au (c122-106-152-1.carlnfd1.nsw.optusnet.com.au [122.106.152.1]) by mail01.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n7J9OkMp024397 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 19 Aug 2009 19:24:48 +1000 Date: Wed, 19 Aug 2009 19:24:46 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Manish Vachharajani In-Reply-To: <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> Message-ID: <20090819183756.Y35058@delplex.bde.org> References: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> <373149.52091.qm@web63907.mail.re1.yahoo.com> <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@FreeBSD.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 09:24:53 -0000 On Tue, 18 Aug 2009, Manish Vachharajani wrote: > So, in a nutshell, the question is: should these drivers be reporting > miss events as input errors in the ifnet struct as the bge driver > does, or as drops in the ifnet struct, was there some conscious > decision not to report miss events anywhere outside the debug and > stats info, or am I just being silly and not seeing where the numbers > are reported? Certainly they should be no worse than bge in this area. Even bge has problems for the 5705_PLUS versions. PLUS really means MINUS; 5705- hardware is dumbed down so the IFIN_DROPS register is almost unusable, but since the hardware is so bad drops are more likely than with better hardware. The unusablility involves the register being only 8 (?) bits wide and being reset on every read, so you have to read it often to ensure that it doesn't wrap, but reading it (or any PCI register) is very inefficient so the the read that is done often enough to work (in bge_rxeof()) is only done if the "notyet" non-option is configured. Resetting on every read of this and most or all other statistic registers on 5705- hardware also completely breaks most or all bge statistics in the bge statistics sysctl, due to the way sysctl(3) is implemented: sysctl(3) always calls the sysctl syscall twice and uses the results of the second call; both calls do a read at the lowest level, so with registers that are reset on every read, the first call resets the registers and the second call usually reads zero. No history is kept in the sysctl, so the sysctl also clobbers the statistics that are maintained at the non-sysctl level (only collisions and ifin drops for 5705-). The non-sysctl level understands the reset and does keep history, but this is defeated if the sysctl is used. There may also still be a generic problem with intrq drops. The default ip intrq length (sysctl net.inet.ip.intr_queue_maxlen) was too small by default (32 IIRC). Now it is larger by default (256), but 256 is still small if you have multiple NICs with rx ring sizes of hundreds or thousands. Direct dispatch reduces this problem. Further, if an intrq drop actually occurs, then it is only reported in generic ip statistics (net.inet.ip.intr_queue_drops); there is no sign of it in ierrors and no way to determine which interface it happened on. I use 1024 to ensure no drops with a single bge NIC. There is still a related design problem for intrq drops: packets that will be dropped should not even be passed to upper layers, to avoid unnecessary extra load on already-overloaded systems. There is related inefficiency of IFF_MONITOR mode: checking for this should be the very first thing in ether_input(), or at least before asking for cache misses by looking at packet headers, but the check is after mounds of code and at least 1 likely cache miss (for initializing etype unecessarily early). Intrq drops would be efficient if they occurred near the start of ether_input() too, but they occur up much further up the stack than the check for monitor mode. Bruce From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 12:52:27 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D320106568E for ; Wed, 19 Aug 2009 12:52:27 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56404.mail.re3.yahoo.com (web56404.mail.re3.yahoo.com [216.252.111.83]) by mx1.freebsd.org (Postfix) with SMTP id E72A78FC45 for ; Wed, 19 Aug 2009 12:52:26 +0000 (UTC) Received: (qmail 68884 invoked by uid 60001); 19 Aug 2009 12:52:25 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250686345; bh=bV997k2S18D5gNt1TS6mPTjat8J2xSm8IfhnBaNpu24=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=L17en75bOoyTolz+zLZp7QvXu0hky1xj54KZgFIb90SOf/v3TXhQn/v+KvLc4jOAd4fO81QDsWLNfDBeleN4eVB06w3hq9xRxU/Oq/77565D5Eoym70WUg1UJs9tBVQnbrdOpNVxWmuiCLzkVqh/1diKo87QyjH9NZGbqTwRJzs= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=PzamJ1g2wFh4ccP4nBShMQhwT3YqhJz6gLSQB+FC3lD58IgzwxoUmapLB0x+zYle4FcNpDZ0J12Yr0yljMEBrBZtZmLcCwD696XFmGt/cXOavQ93VoBqfvUXcPlXwem8OvU29jzVipd70K+Pm3XTtJII2UGfiqqBH/L/RIgwBVM=; Message-ID: <24727.68667.qm@web56404.mail.re3.yahoo.com> X-YMail-OSG: B7jIdG8VM1m6b_mH9wRlQDX1.mAAwbEGQeo1yzWWid2glcPORGu_F9EYsxDlV8N0MfowIuymZwjEKSqGmWNhcDNg7KnXLT6mfqo3d6qmMXF8EYuEEtFtwYGhGvedbFmaS.zdRYAmupYisIiwxu4aao4hAapFqxrau6TUuXVsb0BAS6PLhMzz5Zmd1EaVHZD7Ob7RP4tXhFVE7rfWgn8dcbjfeXA34F92hNTQkrxi.ZBiLcaop6ASzh9itDlETdKWPd1HNW9RPAfpqC357PDarHPtm1tXZqCn2dDNw2wSdu57E7E6n3YGBMSJyNNvzpnhrTAFqZ8H26D9dNBSsp4To7fYcOoPaNsfOs0- Received: from [91.200.96.83] by web56404.mail.re3.yahoo.com via HTTP; Wed, 19 Aug 2009 05:52:23 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Wed, 19 Aug 2009 05:52:23 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: =?utf-8?B?0JTQvNC40YLRgNC40Lkg0JfQsNC80YPRgNCw0LXQsg==?= In-Reply-To: <001401ca1f4d$e96a2170$1e010a0a@in72.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: RE: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 12:52:27 -0000 Greetings.=0A=0A--- On Mon, 8/17/09, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8= =D0=B9 =D0=97=D0=B0=D0=BC=D1=83=D1=80=D0=B0=D0=B5=D0=B2 wrote:=0A=0A> From: =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=97= =D0=B0=D0=BC=D1=83=D1=80=D0=B0=D0=B5=D0=B2 =0A> Sub= ject: RE: em driver input errors=0A> To: alexpalias-bsdnet@yahoo.com=0A> Cc= : freebsd-net@freebsd.org=0A> Date: Monday, August 17, 2009, 6:17 PM=0A> = =0A> =0A> >/boot/loader.conf:=0A> >hw.em.rxd=3D4096=0A> >hw.em.txd=3D4096= =0A> why you are using this=0A> values? try default (without =0A> this line= s in loader.conf)=0A=0AAs said in my original email, I was getting way more= errors with the defaults.=0A =C2=A0=0A> > Witout the above we=0A> were see= ing way more =0A> errors, now they are reduced, but still come in bursts of= =0A> over 1000 errors on =0A> em0.=0A> >Still seeing errros,=0A> after some= searching the =0A> mailing lists we also added:=0A> ># the four lines belo= w=0A> are repeated for em1, =0A> em2, =0A> em3=0A> >dev.em.0.rx_int_delay= =3D0=0A> >dev.em.0.rx_abs_int_delay=3D0=0A> >dev.em.0.tx_int_delay=3D0=0A> = >dev.em.0.tx_abs_int_delay=3D0=0A> try to increase=0A> rx_int_delay to 600 = and =0A> rx_abs_int_delay to 1000, tx_*_delay without changes ->=0A> by def= ault =0A> (100?)=0A=0AThanks for the suggestion.=0AFrom a "clean" box:=0Ade= v.em.0.rx_int_delay: 0=0Adev.em.0.tx_int_delay: 66=0Adev.em.0.rx_abs_int_de= lay: 66=0Adev.em.0.tx_abs_int_delay: 66=0A=0AI reset all the values (errors= still appearing), then tried your suggestion (rx_int_delay=3D600, rx_abs_i= nt_delay=3D1000). This has reduced the number of interrupts for em0 (from = about 7200/sec to around 6500/sec). After some time, I started getting err= ors again. But that has made me try this also:=0A=0Adev.em.0.tx_int_delay= =3D600=0Adev.em.0.tx_abs_int_delay=3D1000=0A=0AMeaning using your suggested= values for tx too. Now em0 is seeing about 1800 interrupts/second, which = is way better, but after some time I saw errors again...=0A=0AFrom the outp= ut of "netstat -nI em0 -w 5":=0A=0A input (em0) = output=0A packets errs bytes packets errs bytes colls= =0A 87267 0 50372599 106931 0 81598993 0=0A 864= 96 0 50990332 105467 0 80064657 0=0A 81726 3056 = 49876613 99080 0 73273640 0=0A 90425 0 59172531 = 105299 0 77110096 0=0A 120292 0 70369292 109597 = 0 78626248 0=0A... a few minutes pass with zero errors ...=0A 8= 9646 0 56951878 111240 0 86493393 0=0A 86031 0 = 53549721 108695 0 83592747 0=0A 77760 3054 48505562 = 96912 0 73185576 0=0A 87508 0 56116394 106094 = 0 79130608 0=0A 89031 0 56490982 103039 0 773= 98567 0=0A=0AWhat's interesting is that I'm seeing errors in a 80k pack= ets/5 sec (so around 16k packets/s) zone, but no errors at 120k packets/5se= c (24kpps).=0A=0A=0ACurrently, I've set the delay to 600 and abs_delay to 1= 000 on all interfaces (em0, em1, em2, em3), thus reducing the number of int= errupts.=0AI'm currently seeing (in systat -vmstat 2):=0AAround 1800 irqs/s= for em0, 1800 for em1, 1800 for em2, under 10/s for em3=0AAround 2000 irqs= /s for cpu0:time, 2000 more for cpu1:time, 2000 for cpu2:time and 2000 for = cpu3:time.=0A=0AInterrupts total (as reported by systat): around 13500/sec= ond. I would estimate the old IRQ load at around 30000-35000/second, which= doesn't seem too much to me, for a dual xeon machine.=0A=C2=A0=0A> >kern.i= pc.nmbclusters=3D655360=0A> no need. see netstat=0A> -m=0A=0AThanks, but as= I said, I did try almost *EVERYTHING* I could without rebooting. Includin= g this.=0A=0ASpeaking of which, I did compile the kernel with "options DEVI= CE_POLLING", but enabling polling only made the errors appear more often, a= nd in greater numbers.=0A=0A> P.S. change copper cable,=0A> turn off the fl= ow-control =0A> (if is on) =0A=0AThere are 4 em interfaces on this machine,= with new cat6 cables. 2 more em interfaces on another machine that was se= eing the same errors (the old router), on different cables. And 2 more em = interfaces on another machine that's in production, also with new cables. = The input errors (as debugged by sysctl dev.em.0.stats=3D1 -> read dmesg) a= re only 2 because of CRC errors, as opposed to around 2.500.000 from other = causes. I tend to feel the cable isn't the problem.=0A=0AFlow control is o= ff, I just checked. I forgot about that one, thanks for reminding me.=0A= =0A=0AThank you for your help=0AAlex From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 16:07:30 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AE831065690 for ; Wed, 19 Aug 2009 16:07:30 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id B1F1F8FC67 for ; Wed, 19 Aug 2009 16:07:29 +0000 (UTC) Received: by an-out-0708.google.com with SMTP id d14so1664235and.13 for ; Wed, 19 Aug 2009 09:07:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=CnTz+ihsKZVCQbPtOPt+i8ziq1C/Uakn0qVK5I/lbqQ=; b=ATqm4KVA7RkKJM3YoaDz8XALM6LuJliR7qeT7k8TwoVFQmTYTTJZzOrIVFf4IUif/2 lQnwy1Czgl1bJed61v9+1kyb26yZU2DduGE3w9IVtnxuS8rhrAMm7QUGlG4xq8dnEIDO lPa0rco8uZE0HmXKLYg9A/WB/OvDal86uULKk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Fo3j6vtD6dg21XAEWXNv7KMgf/6xMbnhVzAa8or2itBB3eMaSCaPNfyxWcwNzoBNOq vSeemXVjAoMIOM72jZED4OI65eSe1WK0oaMsGvTwiJ0QaDezc5C1r/gC7GbvaBEk8iZX c3QWghz1yEtP1I6MgVo6D+3d1kRKYy1DYbE68= MIME-Version: 1.0 Received: by 10.101.9.10 with SMTP id m10mr7345174ani.130.1250698049130; Wed, 19 Aug 2009 09:07:29 -0700 (PDT) In-Reply-To: <11167f520908182358hc7a176as962788b505b87148@mail.gmail.com> References: <2a41acea0906261725x57e6903br9f3f42b55f3a3d30@mail.gmail.com> <688430.20427.qm@web37906.mail.mud.yahoo.com> <2a41acea0906280952s23d6553ep42fcfd4671561c3a@mail.gmail.com> <2a41acea0907071722p7992bea0s281399cb0baecd90@mail.gmail.com> <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> <2a41acea0908182306h740194bcw4de78b6093998e88@mail.gmail.com> <11167f520908182358hc7a176as962788b505b87148@mail.gmail.com> Date: Wed, 19 Aug 2009 09:07:29 -0700 Message-ID: <2a41acea0908190907g366fd2bft3a65c8565e3d859@mail.gmail.com> From: Jack Vogel To: "Sam Fourman Jr." Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Mark Atkinson Subject: Re: Regression: em driver in -CURRENT, "Invalid MAC address" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 16:07:30 -0000 That should not be necessary. I will have someone look into this today. Jack On Tue, Aug 18, 2009 at 11:58 PM, Sam Fourman Jr. wrote: > On Wed, Aug 19, 2009 at 1:06 AM, Jack Vogel wrote: > > Hmmm, that's odd, I had the validation group do a thorough test > > of older adapters, and that one did not seem to have a problem, > > let me see if we can repro the issue tomorrow, but, in principle I > > have no problem doing this. > > > > Jack > > I can provide root ssh access to these with a serial console and > public IP's if you need > I have over 100 of these (old dell P3 2450's with whatever em(4) fibre > nic dell shipped) sitting on the shelf collecting dust. > > Just send me a heads up and I will have someone set one up for you right > away. > > Sam Fourman Jr. > Fourman Networks > From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 16:27:03 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04292106568B for ; Wed, 19 Aug 2009 16:27:03 +0000 (UTC) (envelope-from gigabyte.tmn@gmail.com) Received: from mail-bw0-f206.google.com (mail-bw0-f206.google.com [209.85.218.206]) by mx1.freebsd.org (Postfix) with ESMTP id 6F9CF8FC57 for ; Wed, 19 Aug 2009 16:27:02 +0000 (UTC) Received: by bwz2 with SMTP id 2so3564177bwz.43 for ; Wed, 19 Aug 2009 09:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:reply-to:from:to :cc:references:subject:date:mime-version:content-type :content-transfer-encoding:x-priority:x-msmail-priority:x-mailer :x-mimeole; bh=89tqJjg/nAcwlKTtc8RxidFsL4Zyo2sclaihu3qOHYg=; b=KeZBDKck5s4B462YSkviWYfnjdqIOfJZgRrJf3Zt7o3r1zeDOFEVXUu1SRrcalm0A5 gt45fEUayT+DvpOoh4sN4Q5aeoDTarVnjM3bYcvSJs1C06hFs4ExAh+B5LlFsjOSDoq+ vDYzpgmqCBTSkE3vA41ajm+e5g1ue28JOlr7c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:reply-to:from:to:cc:references:subject:date:mime-version :content-type:content-transfer-encoding:x-priority:x-msmail-priority :x-mailer:x-mimeole; b=rUfbgsWwelwE6Ob6HLQGFJWImgESQoeMpD2unC5IfocNPwyRkzyES/4XNFEGqt84ED M2fIMX8X+h23kk3yOJt8vtlZr9d588/ANTMVX5jTfRGWNuTeAcSo1cQnEtCtR4X/i0y4 d8/OcerTFdL6hEnyt0jt3vHrpYUwYu4MdXgVI= Received: by 10.204.155.73 with SMTP id r9mr5092113bkw.101.1250699221348; Wed, 19 Aug 2009 09:27:01 -0700 (PDT) Received: from dm (152.dynamic-n192.r72.info [91.211.192.152]) by mx.google.com with ESMTPS id p9sm435474fkb.7.2009.08.19.09.27.00 (version=SSLv3 cipher=RC4-MD5); Wed, 19 Aug 2009 09:27:00 -0700 (PDT) Message-ID: <000e01ca20e9$e19caa10$1e010a0a@in72.ru> From: =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JfQsNC80YPRgNCw0LXQsg==?= To: References: <24727.68667.qm@web56404.mail.re3.yahoo.com> Date: Wed, 19 Aug 2009 22:27:00 +0600 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 X-Mailman-Approved-At: Wed, 19 Aug 2009 16:34:46 +0000 Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JfQsNC80YPRgNCw0LXQsg==?= List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 16:27:03 -0000 Hello Alex. What sheduler are you using? ULE or 4BSD Have you NIC IRQ sharing with other hardware? What HZ value? 1000? >Thanks for the suggestion. >From a "clean" box: >dev.em.0.rx_int_delay: 0 >dev.em.0.tx_int_delay: 66 >dev.em.0.rx_abs_int_delay: 66 >dev.em.0.tx_abs_int_delay: 66 >I reset all the values (errors still appearing), then tried your suggestion >(rx_int_delay=600, rx_abs_int_delay=1000). This has reduced the number of > >interrupts for em0 (from about 7200/sec to around 6500/sec). After some >time, I started getting errors again. mmm, try the maximum value 67108, what hapens... > But that has made me try this also: >dev.em.0.tx_int_delay=600 >dev.em.0.tx_abs_int_delay=1000 I think it's a bad idea, but don't know because: >Meaning using your suggested values for tx too. Now em0 is seeing about >1800 interrupts/second, which is way better, but after some time I saw >errors >again... >From the output of "netstat -nI em0 -w 5": maybe mistake, did you meen "netstat -w5 em0" ? I have PPPoE concenrator based on S3000AHV motherboard with Core2Quad 6600 and four (to load all cores in CPU) Intel PCI-E x1 and PCI-E x4 NIC's My load: bras1 [/usr/home/dm]# netstat -w5 em0 input (Total) output packets errs bytes packets errs bytes colls 943831 0 803741196 932221 0 766771487 0 ^C bras1 [/usr/home/dm]# netstat -w1 -Iem0 input (em0) output packets errs bytes packets errs bytes colls 24067 0 20593033 17152 0 17361755 0 ^C bras1 [/usr/home/dm]# netstat -w1 -Ilagg0 input (lagg0) output packets errs bytes packets errs bytes colls 47085 0 38454150 46708 0 38128482 0 44888 0 36087138 44714 0 35985529 0 49607 0 40467232 49326 0 40227456 0 ^C bras1 [/usr/home/dm]# netstat -w5 -Ilagg0 input (lagg0) output packets errs bytes packets errs bytes colls 230260 0 187650240 228911 0 186485136 0 238023 0 194650670 236648 0 193471650 0 218424 0 175576014 216860 0 174282762 0 ^C The lagg0 interface includes em0, em1, em2, em3 for lacp protocol, and comunicates with cisco 2960G switch. vmstat -i says: interrupt total rate irq4: sio0 95234 0 irq19: atapci1 8430157 1 cpu0: timer 1275549106 258 irq256: em0 2329917460 472 irq257: em1 645070135 130 irq258: em2 3527395550 715 irq259: em3 3923746474 795 cpu1: timer 1275548822 258 cpu3: timer 1275548798 258 cpu2: timer 1275548865 258 Total 15536850601 3149 And i have't any problems. I think i select the good hardware. > input (em0) output > packets errs bytes packets errs bytes colls > 87267 0 50372599 106931 0 81598993 0 > 86496 0 50990332 105467 0 80064657 0 > 81726 3056 49876613 99080 0 73273640 0 > 90425 0 59172531 105299 0 77110096 0 > 120292 0 70369292 109597 0 78626248 0 >... a few minutes pass with zero errors ... > 89646 0 56951878 111240 0 86493393 0 > 86031 0 53549721 108695 0 83592747 0 > 77760 3054 48505562 96912 0 73185576 0 > 87508 0 56116394 106094 0 79130608 0 > 89031 0 56490982 103039 0 77398567 0 >What's interesting is that I'm seeing errors in a 80k packets/5 sec (so >around 16k packets/s) zone, but no errors at 120k packets/5sec (24kpps). Yes, it's not normaly. >Interrupts total (as reported by systat): around 13500/second. I would >estimate the old IRQ load at around 30000-35000/second, which doesn't seem >too >much to me, for a dual xeon machine. I think it depends by motherbord, what full hardware specification are you using? with chips names >Speaking of which, I did compile the kernel with "options DEVICE_POLLING", >but enabling polling only made the errors appear more often, and in greater > >numbers. I don't use polling on FBSD 7.x, it's usable on FBSD older versions > - 1 x dual-port gigabit interface, PCI-X Maybe I have this card. And it works unstable, i don't remember what happens, but i seen by tcpdump "truncated IP, missing XX bytes" Good luck. From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 17:52:26 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65CC3106568B for ; Wed, 19 Aug 2009 17:52:26 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id 03C328FC71 for ; Wed, 19 Aug 2009 17:52:25 +0000 (UTC) Received: (qmail 93309 invoked from network); 19 Aug 2009 17:25:44 -0000 Received: from unknown (HELO ?128.122.51.108?) (spawk@128.122.51.108) by acm.poly.edu with AES256-SHA encrypted SMTP; 19 Aug 2009 17:25:44 -0000 Message-ID: <4A8C3557.20002@acm.poly.edu> Date: Wed, 19 Aug 2009 13:24:39 -0400 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.19 (X11/20090108) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: kmem_map too small panics with Soekris/Atheros access point X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 17:52:26 -0000 Ahoy. I've got two Soekris Net5501s with Atheros 5212 cards in them, acting as access points. They are both running 7.2-RELEASE and at times each one has up to 30 machines associated with it. Relevant information about them can be found at "http://acm.poly.edu/~spawk/ap/". After a few days, they panic with: panic: kmem_malloc(32768): kmem_map too small: 86142976 total allocated Indeed, vm.kmem_size on each of them is 86228992. I had nowhere to write a core dump to, but the root issue seems to be that all kernel memory was exhausted, which sounds like a memory leak somewhere. Is that a known problem with that release and hardware configuration? -Boris From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 18:14:34 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47963106568E for ; Wed, 19 Aug 2009 18:14:34 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n3a.bullet.mail.ac4.yahoo.com (n3a.bullet.mail.ac4.yahoo.com [76.13.13.66]) by mx1.freebsd.org (Postfix) with SMTP id E97958FC65 for ; Wed, 19 Aug 2009 18:14:33 +0000 (UTC) Received: from [76.13.13.25] by n3.bullet.mail.ac4.yahoo.com with NNFMP; 19 Aug 2009 18:14:33 -0000 Received: from [76.13.10.183] by t4.bullet.mail.ac4.yahoo.com with NNFMP; 19 Aug 2009 18:14:33 -0000 Received: from [127.0.0.1] by omp124.mail.ac4.yahoo.com with NNFMP; 19 Aug 2009 18:14:33 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 316750.39893.bm@omp124.mail.ac4.yahoo.com Received: (qmail 42257 invoked by uid 60001); 19 Aug 2009 18:14:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250705673; bh=9vM9JXyk+Q653jWtUZybvIYxS5FjTgxxFpoYCniJDNc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=QZgl4jcOrcsX+1E1UwWIrDsbxnDrV5O6cy2eT6p65Jm04oFiQl+AhAN01k26hmrLA9q5xZeIf31VvopO8/4sGVVZFh+uPVIomL0BIMXmHApHrKhX9PNjTD/wzq6jha/n9uu43QPOPxokcqZmK4SgedkeTmsvdZVn73U+9m+ysos= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=hNPVrC0uxgCYm7dZOi6CGJRheU7fMXD3Jtb8HnSxWh061nVpNA9+govTmBVEyMJjgQBN9l+ZMs/W+ZmWLnZDAaMG7ysu3Slvzjs5teDnRhgRfq30+w2++KS/VNlD3NU48nctDNvVu5tObVi01x9rt3FLPrhKHprMo00MsoKutGo=; Message-ID: <121870.41829.qm@web63905.mail.re1.yahoo.com> X-YMail-OSG: OoysDLUVM1nXmW5mFnFnUc08IWIr0a4DFAk4i_xjquCEIk78IYZhb5tQCf3jhGYn7xTPZfl.rq4EoNqCXji6x9cKzge6CwcEold2gnFsnpW9FXRomBneymjG723VT_fVd0AZWn8qK_xYsebrMvIPTznS.XcoVVNnk5h1zTlG2T9MC6U5yvZMs666unrEE07.enobi9fsYzihRhWoGs89.wdXuWfI2QMKo5TXr2tjer71lg6kdCnUBzYsPkSEQr3V4QfgbbAcssKLepzrr8NGFuvPzRiGYHNEn40ZN5GzcXjAgnLXPMMAXR43uSEELJI87duuSXcv.lcfJQ5fj19bezH7.A-- Received: from [66.176.162.245] by web63905.mail.re1.yahoo.com via HTTP; Wed, 19 Aug 2009 11:14:32 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Wed, 19 Aug 2009 11:14:32 -0700 (PDT) From: Barney Cordoba To: d@delphij.net, Julian Elischer In-Reply-To: <4A8B316F.3030408@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: pyunyh@gmail.com, David Christensen , "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 18:14:34 -0000 =0A=0A--- On Tue, 8/18/09, Julian Elischer wrote:=0A= =0A> From: Julian Elischer =0A> Subject: Re: [PATCH] F= ix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of= NOARP flag]=0A> To: d@delphij.net=0A> Cc: pyunyh@gmail.com, "Barney Cordob= a" , "David Christensen" , = "freebsd-net@freebsd.org" , "Jack Vogel" , "Jack F Vogel" , yongari@freebsd.org=0A> Date:= Tuesday, August 18, 2009, 6:55 PM=0A> Xin LI wrote:=0A> > -----BEGIN PGP S= IGNED MESSAGE-----=0A> > Hash: SHA1=0A> > =0A> > Barney Cordoba wrote:=0A> = >> --- On Tue, 8/18/09, Pyun YongHyeon =0A> wrote:=0A> >>= =0A> >>> From: Pyun YongHyeon =0A> >>> Subject: Re: [PATC= H] Fix for e1000 (em/igb)=0A> NOARP issue [Was Re: em(4): sending ARP regar= dless of NOARP=0A> flag]=0A> >>> To: "Xin LI" =0A> >>>= Cc: "Barney Cordoba" ,=0A> "David Christensen" <= davidch@broadcom.com>,=0A> "d@delphij..net"=0A> ,=0A> "freeb= sd-net@freebsd.org"=0A> ,=0A> "Jack Vogel" ,=0A> "Jack F Vogel" ,=0A> yongari@freebsd.org= ,=0A> "Julian Elischer" =0A> >>> Date: Tuesday, August= 18, 2009, 5:49 PM=0A> >>> On Tue, Aug 18, 2009 at 02:03:37PM=0A> >>> -0700= , Xin LI wrote:=0A> >>>> -----BEGIN PGP SIGNED MESSAGE-----=0A> >>>> Hash: = SHA1=0A> >>>>=0A> >>>> Hi, Jack,=0A> >>>>=0A> >>>> I have looked into the c= ode history and=0A> found that=0A> >>> sys/dev/em/if_em.c,v=0A> >>>> 1.119 = has introduced the arp_ifinit() call=0A> in order to=0A> >>> fix the proble= m=0A> >>>> that if_em won't send ARP when IP address=0A> is changed.=0A> >>= >>=0A> >>>> I think we can further improve it as=0A> attached, say,=0A> >>>= only do it when=0A> >>>> IFF_NOARP is not set.=A0 This should=0A> have no = effect=0A> >>> for usual=0A> >>>> configuration but fix the problem when=0A= > NOARP is the=0A> >>> desired behavior.=0A> >>> That change was introduced= by me. I guess the=0A> root cause of=0A> >>> the=0A> >>> problem was long = initialization time of=0A> hardware which in=0A> >>> turn=0A> >>> resulted = in unbearable boot time when=0A> multiple-alias=0A> >>> addresses are=0A> >= >> assigned to em(4). I don't remember=0A> details,though.=0A> >>>=0A> >>> = Since we're in the release cycle, the change=0A> you suggested=0A> >>> woul= d be=0A> >>> quick fix for 8.0. I think em(4)/igb(4) should=0A> remove=0A> = >>> SIOCSIFADDR=0A> >>> handling in driver which is layering=0A> violation.= =0A> >> There are 2 kinds of programmers; those who do=0A> things "correctl= y',=0A> >> and those that do things that work. =0A> >>=0A> >> 99.99999% of = the people will be using ARPs, so=0A> don't be silly and=0A> >> break the d= river to solve a case that almost=0A> no-one cares about please.=0A> > =0A>= =0A> Cisco.Ironport=A0 runs 50% (2 out of 4) of their em=0A> interfaces in= noarp =0A> mode.=0A=0A=0AAh, are they running Jack's drivers unmodified? S= eems unlikely.=0A=0ANOARP does work. Does your network catch on fire if the= interface sends=0Aan ARP out? Does equipment start failing like dominos? = =0A=0AMy point was don't make ARPs not work (the reason the "hack" is in=0A= there is to make something work better) to preserve some fantasy of=0A"laye= ring" that went out with the 8-track player. The check for =0Athe NOARP fla= g is a better solution until the subsystem works the =0Away its supposed to= work.=0A=0ABarney=0A=0A=0A From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 18:43:00 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F70106568B for ; Wed, 19 Aug 2009 18:43:00 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-yw0-f199.google.com (mail-yw0-f199.google.com [209.85.211.199]) by mx1.freebsd.org (Postfix) with ESMTP id CD90A8FC59 for ; Wed, 19 Aug 2009 18:42:59 +0000 (UTC) Received: by ywh37 with SMTP id 37so6536940ywh.28 for ; Wed, 19 Aug 2009 11:42:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=pZBBq8IcpV72JCJ8uNXC4tute7U4lNV611QOt8ITNOg=; b=S9CG7CepJe8n728QCMJwW/B6gaDTPyhDeYvbWT1W541mFY84sUhoSAYTcuKkx0S/ya ViN208dBjMJxRCdEXrYaVWM4ehHN+csyITw5Oihgb57/tNKXcC5UryemTpMQ74UcfHAJ 09eXI3H5xJHtVHz4ZP2GM2XO66N+Xp6Yl3rZo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=OKk2eOY+gM7d6ACFLAi11AASv2/QstvvUR1sPzgLNN1H6Dt8Y9gy2XjEI74hoFEC5G rEMxKIJoEAGtNPd/bzCrGbA4wtzc8Dsm2CvD/DfnSY0StNntngLCz+rjcuCe3q7P2PyZ kl2H0Irl2UF03DXK5jIPMnYmQcvhKkCqobmIw= MIME-Version: 1.0 Received: by 10.100.40.17 with SMTP id n17mr7500806ann.187.1250707379170; Wed, 19 Aug 2009 11:42:59 -0700 (PDT) In-Reply-To: <2a41acea0908190907g366fd2bft3a65c8565e3d859@mail.gmail.com> References: <2a41acea0906261725x57e6903br9f3f42b55f3a3d30@mail.gmail.com> <688430.20427.qm@web37906.mail.mud.yahoo.com> <2a41acea0906280952s23d6553ep42fcfd4671561c3a@mail.gmail.com> <2a41acea0907071722p7992bea0s281399cb0baecd90@mail.gmail.com> <11167f520908181955x3cc90a26k3ca37fd45cfaabf3@mail.gmail.com> <2a41acea0908182306h740194bcw4de78b6093998e88@mail.gmail.com> <11167f520908182358hc7a176as962788b505b87148@mail.gmail.com> <2a41acea0908190907g366fd2bft3a65c8565e3d859@mail.gmail.com> Date: Wed, 19 Aug 2009 11:42:59 -0700 Message-ID: <2a41acea0908191142k453b4211n15aea5904e1b21a0@mail.gmail.com> From: Jack Vogel To: "Sam Fourman Jr." Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Mark Atkinson Subject: Re: Regression: em driver in -CURRENT, "Invalid MAC address" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 18:43:00 -0000 We've repro'd this, it wasnt caught before because he only tested copper. I will get the change submitted. Jack On Wed, Aug 19, 2009 at 9:07 AM, Jack Vogel wrote: > That should not be necessary. I will have someone look into this today. > > Jack > > > > On Tue, Aug 18, 2009 at 11:58 PM, Sam Fourman Jr. wrote: > >> On Wed, Aug 19, 2009 at 1:06 AM, Jack Vogel wrote: >> > Hmmm, that's odd, I had the validation group do a thorough test >> > of older adapters, and that one did not seem to have a problem, >> > let me see if we can repro the issue tomorrow, but, in principle I >> > have no problem doing this. >> > >> > Jack >> >> I can provide root ssh access to these with a serial console and >> public IP's if you need >> I have over 100 of these (old dell P3 2450's with whatever em(4) fibre >> nic dell shipped) sitting on the shelf collecting dust. >> >> Just send me a heads up and I will have someone set one up for you right >> away. >> >> Sam Fourman Jr. >> Fourman Networks >> > > From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 18:43:06 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86B96106568F for ; Wed, 19 Aug 2009 18:43:06 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n72.bullet.mail.sp1.yahoo.com (n72.bullet.mail.sp1.yahoo.com [98.136.44.34]) by mx1.freebsd.org (Postfix) with SMTP id 5B64C8FC3D for ; Wed, 19 Aug 2009 18:43:06 +0000 (UTC) Received: from [69.147.84.144] by n72.bullet.mail.sp1.yahoo.com with NNFMP; 19 Aug 2009 18:29:31 -0000 Received: from [68.142.200.227] by t6.bullet.mail.sp1.yahoo.com with NNFMP; 19 Aug 2009 18:29:31 -0000 Received: from [76.13.13.26] by t8.bullet.mud.yahoo.com with NNFMP; 19 Aug 2009 18:29:31 -0000 Received: from [76.13.10.172] by t3.bullet.mail.ac4.yahoo.com with NNFMP; 19 Aug 2009 18:29:31 -0000 Received: from [127.0.0.1] by omp113.mail.ac4.yahoo.com with NNFMP; 19 Aug 2009 18:29:31 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 91495.56173.bm@omp113.mail.ac4.yahoo.com Received: (qmail 18673 invoked by uid 60001); 19 Aug 2009 18:29:31 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250706570; bh=QHn+CP2H9zm0O4TzcunGKCRV4eXa0t0nBTGZZ698X8Q=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=6ZOkiEGMBAtCRsf6mJKNigTRQy8G9mnFrv7Aajbwj8Hw78+kEGYSK3ezDQVElTaNm0SFU6nDC27vpjrceLNrjt/+PBzX6/wrKxFKmCUqJBC7yfefunbPzEsedmM4e92brqaX9cRZ6pGmOBnm5ELJtTcZY6/48KxQ6cjbUFYgB00= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=56y1BQ1s2fyBi0Z3iDS5QyvI5VD6xQZC6yJ9W6ROkgI6Z8zSdpkjRX3cQfXdaAWo/gZvoeMi7W+Hel3QRQkWiPYNHpIxE5Y+PFoX6sFePKq7zpjzASRGENoFJoOhxZGIs6dJ+A4zi6YNawt/O7MB70Yg4YytRNp0a5RyokL4mqQ=; Message-ID: <822688.18516.qm@web63903.mail.re1.yahoo.com> X-YMail-OSG: Fa_qbmwVM1m8N.ByR4nyFvFXdxtgrwYUoUJxjRsq54aDfN38DaPrbB8kQVMrmpc9bndHAxn93Ez350UifizXuN0X.89HTZ5Lxjrlp_rTnQCYZp_MF_jVrzTYOx98rxgi6I0Tb0bgkdRTNXZ1Kx74DWm6e8FOXduhyQC.P23MwbFZwP5.OQwgmUve5iwhriU8ZyI.4XrnEe5XRcOWp2s3z0PouccUWxfis8qFBbGUnK1YtzRA_JyD2LsSZlydsA_JYGz63R50OPCd5mf0Dk0ha3HVC5L3lvFysiRFUOrw2aD8OUHyMDIeKVSWpR94L9_DDliXiLsYDhaPngwze_CB_2gRCUHvSu_rlEVi9g2c Received: from [66.176.162.245] by web63903.mail.re1.yahoo.com via HTTP; Wed, 19 Aug 2009 11:29:30 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Wed, 19 Aug 2009 11:29:30 -0700 (PDT) From: Barney Cordoba To: Manish Vachharajani In-Reply-To: <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 18:43:06 -0000 =0A=0A--- On Tue, 8/18/09, Manish Vachharajani wrote:=0A=0A> From: Manish Vachharajani =0A>= Subject: Re: Dropped vs. missed packets in the ixgbe driver=0A> To: "Barne= y Cordoba" =0A> Cc: freebsd-net@freebsd.org=0A> D= ate: Tuesday, August 18, 2009, 6:35 PM=0A> Indeed the debugging info is als= o=0A> interesting.=A0 However, I'd like to=0A> get some data from netstat w= hen the driver drops=0A> frames.=A0 When looking=0A> at the bge driver sour= ce, it appears that all input drops=0A> at the NIC=0A> are reported as inpu= t errors.=A0 It appears that the=0A> intel drivers=0A> (e1000, ixgb, and ix= gbe) don't report that number outside=0A> of the debug=0A> and stats printf= s at all, and this seems broken.=A0 What=0A> I want to know=0A> is if I hav= e just missed where these are reported.=0A> =0A> So, in a nutshell, the que= stion is:=A0 should these=0A> drivers be reporting=0A> miss events as input= errors in the ifnet struct as the bge=0A> driver=0A> does, or as drops in = the ifnet struct, was there some=0A> conscious=0A> decision not to report m= iss events anywhere outside the=0A> debug and=0A> stats info, or am I just = being silly and not seeing where=0A> the numbers=0A> are reported?=0A> =0A>= Also, don't worry on the performance front, we are also=0A> looking at the= =0A> driver in FreeBSD 8.0 :) which supports RSS to help=0A> performance=0A= > scaling,=A0 though we have some interesting data there=0A> that I'll post= =0A> about once I confirm that the numbers are indeed correct=0A> and not a= =0A> tuning or setup problem.=0A> =0A> Manish=0A> =0A> > --- On Mon, 8/17/0= 9, Manish Vachharajani =0A> wrote:=0A> >=0A> >= > From: Manish Vachharajani =0A> >> Subject: D= ropped vs. missed packets in the ixgbe=0A> driver=0A> >> To: freebsd-net@fr= eebsd.org=0A> >> Date: Monday, August 17, 2009, 6:24 PM=0A> >> I've been do= ing some performance=0A> >> testing on freebsd 7.2 and noticed=0A> >> that = the ixgbe driver does not report missed=0A> packets as=0A> >> dropped when= =0A> >> queried via netstat -id (the ixgbe driver in the=0A> 8.0=0A> >> bet= a has a=0A> >> similar issue).=A0 A missed packet is a packet that=0A> was= =0A> >> correctly=0A> >> received by the NIC but because it was out of=0A> = descriptors=0A> >> and internal=0A> >> memory, the packet had to be dropped= by the NIC=0A> itself -- a=0A> >> hardware=0A> >> register counts such eve= nts.=A0 Instead the driver=0A> only=0A> >> reports drops=0A> >> that are du= e to the driver itself, i.e., the NIC=0A> DMAed the=0A> >> packet to=0A> >>= memory but the driver had to drop something=0A> because it was=0A> >> out = of=0A> >> mbufs.=A0 Is the miss count reported elsewhere=0A> (besides=0A> >= > via a kernel=0A> >> printf from the driver)?=A0 At the end of the email= =0A> I=0A> >> give a stats dump=0A> >> from the driver and from netstat, no= te the number=0A> of=0A> >> packets dropped=0A> >> is 0 as reported by nets= tat but the Missed field=0A> in the=0A> >> dmesg output=0A> >> shows many m= issed packets.=0A> >>=0A> >> >From my perspective, it is disconcerting to= =0A> see=0A> >> performance=0A> >> degradation on the link, along with TCP = ack=0A> retransmits,=0A> >> packet=0A> >> reordering, etc. (on a point-to-p= oint link with no=0A> switch=0A> >> in between)=0A> >> but then see no drop= s reported by netstat because=0A> the=0A> >> driver didn't=0A> >> drop the = packet, the NIC did.=A0 The fix should be=0A> >> straight-forward and=0A> >= > I'll gladly make a patch assuming that it is=0A> indeed a bug=0A> >> and = not a=0A> >> conscious design choice.=0A> >>=0A> >> Here is the relevant ne= tstat output=0A> >>=0A> >> Name=A0 =A0 Mtu Network=0A> >> =A0=A0=A0Address= =0A> >> =A0 =A0 Ipkts Ierrs=A0 =A0 Opkts=0A> >> Oerrs=A0 Coll Drop=0A> >> i= x0=A0 =A0 1500 =0A> >> 00:30:48:94:60:ec=A0 =A0 =A0 =A0 0=0A> >> = =A0=A0=A00=A0 =A0 =A0 =A0 1=0A> >> =A00=A0 =A0=A0=A00=A0 =A0 0=0A> >> ix0= =A0 =A0 1500 192.168.105.0 192.168.105.2=0A> >> =A0 =A0 =A0 =A0 =A0 0=0A> >= > =A0=A0=A0-=A0 =A0 =A0 =A0 0=0A> >> =A0-=A0 =A0=A0=A0-=A0 =A0 -=0A=0A=0Aif= you look in ixgbe_update_stats_counters at the bottom:=0A=0A ifp->i= f_ierrors =3D missed_rx + adapter->stats.crcerrs +=0A adapte= r->stats.rlec;=0A=0Athe errors are added in. =0A=0ABC=0A> >> 00:30:48:94:60= :ed=A0 =A0 =A0 11M=0A> >> =A0=A0=A00=A0 =A0=A0=A06.1M=0A> >> =A00=A0 =A0=A0= =A00=A0 =A0 0=0A> >> ix1=A0 =A0 1500=0A> >> 192.168.5.0=A0=A0=A0192.168.5.2= =0A> >> =A0 =A0 =A0 10M=A0 =A0=A0=A0-=0A> >> =A0=A0=A06.1M=0A> >> =A0-=A0 = =A0=A0=A0-=A0 =A0 -=0A> >>=0A> >> And here is the dmesg output after doing = a sysctl=0A> >> dev.ix.1.stats=3D1=0A> >>=0A> >> ix1: Std Mbuf Failed =3D 0= =0A> >> ix1: Missed Packets =3D 413872=0A> >> ix1: Receive length errors = =3D 0=0A> >> ix1: Crc errors =3D 0=0A> >> ix1: Driver dropped packets =3D 0= =0A> >> ix1: watchdog timeouts =3D 0=0A> >> ix1: XON Rcvd =3D 616428212235= =0A> >> ix1: XON Xmtd =3D 0=0A> >> ix1: XOFF Rcvd =3D 616428212235=0A> >> i= x1: XOFF Xmtd =3D 0=0A> >> ix1: Total Packets Rcvd =3D 12424533=0A> >> ix1:= Good Packets Rcvd =3D 12010661=0A> >> ix1: Good Packets Xmtd =3D 6419128= =0A> >> ix1: TSO Transmissions =3D 0=0A> >>=0A> >> Manish=0A> >=0A> > the d= ebug sysctl show more interesting info. Don't get=0A> too excited=0A> > abo= ut doing performance testing with that driver. Its=0A> not designed=0A> > t= o be any higher in performance than any of the other=0A> intel drivers.=0A>= >=0A> > Barney=0A> >=0A> >=0A> >=0A> >=0A> =0A> =0A> =0A> -- =0A> Manish V= achharajani=0A> Founder=0A> LineRate Systems=0A> manishv@lineratesystems.co= m=0A> (609)635-9531 M=0A> _______________________________________________= =0A> freebsd-net@freebsd.org=0A> mailing list=0A> http://lists.freebsd.org/= mailman/listinfo/freebsd-net=0A> To unsubscribe, send any mail to "freebsd-= net-unsubscribe@freebsd.org"=0A> =0A=0A=0A From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 18:46:06 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1C01106568C for ; Wed, 19 Aug 2009 18:46:06 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 700438FC51 for ; Wed, 19 Aug 2009 18:46:06 +0000 (UTC) Received: by vws10 with SMTP id 10so3941702vws.7 for ; Wed, 19 Aug 2009 11:46:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.58.208 with SMTP id i16mr9469031vch.20.1250707562441; Wed, 19 Aug 2009 11:46:02 -0700 (PDT) In-Reply-To: <822688.18516.qm@web63903.mail.re1.yahoo.com> References: <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> <822688.18516.qm@web63903.mail.re1.yahoo.com> Date: Wed, 19 Aug 2009 12:46:02 -0600 Message-ID: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> From: Manish Vachharajani To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 18:46:06 -0000 Agreed, the errors are reported but missed packets are not. The question is, is the correct fix to just add stats.mpc[0] to if_ierrors in that line or to add it to if_iqdrops. The fix is easy once we agree on what the correct behavior is. Manish > Barney wrote: > > if you look in ixgbe_update_stats_counters at the bottom: > > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + adapter->stats.crcerrs + > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; > > the errors are added in. > > BC --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 19:34:07 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AA5D106568C for ; Wed, 19 Aug 2009 19:34:07 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outT.internet-mail-service.net (outt.internet-mail-service.net [216.240.47.243]) by mx1.freebsd.org (Postfix) with ESMTP id 784D78FC52 for ; Wed, 19 Aug 2009 19:34:07 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id C4D5096720; Wed, 19 Aug 2009 12:34:06 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id D6BDF2D6017; Wed, 19 Aug 2009 12:34:05 -0700 (PDT) Message-ID: <4A8C53AD.1040102@elischer.org> Date: Wed, 19 Aug 2009 12:34:05 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: Barney Cordoba References: <121870.41829.qm@web63905.mail.re1.yahoo.com> In-Reply-To: <121870.41829.qm@web63905.mail.re1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, David Christensen , d@delphij.net, "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 19:34:07 -0000 Barney Cordoba wrote: > > --- On Tue, 8/18/09, Julian Elischer wrote: > >> From: Julian Elischer >> Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] >> To: d@delphij.net >> Cc: pyunyh@gmail.com, "Barney Cordoba" , "David Christensen" , "freebsd-net@freebsd.org" , "Jack Vogel" , "Jack F Vogel" , yongari@freebsd.org >> Date: Tuesday, August 18, 2009, 6:55 PM >> Xin LI wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Barney Cordoba wrote: >>>> --- On Tue, 8/18/09, Pyun YongHyeon >> wrote: >>>>> From: Pyun YongHyeon >>>>> Subject: Re: [PATCH] Fix for e1000 (em/igb) >> NOARP issue [Was Re: em(4): sending ARP regardless of NOARP >> flag] >>>>> To: "Xin LI" >>>>> Cc: "Barney Cordoba" , >> "David Christensen" , >> "d@delphij..net" >> , >> "freebsd-net@freebsd.org" >> , >> "Jack Vogel" , >> "Jack F Vogel" , >> yongari@freebsd.org, >> "Julian Elischer" >>>>> Date: Tuesday, August 18, 2009, 5:49 PM >>>>> On Tue, Aug 18, 2009 at 02:03:37PM >>>>> -0700, Xin LI wrote: >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> Hi, Jack, >>>>>> >>>>>> I have looked into the code history and >> found that >>>>> sys/dev/em/if_em.c,v >>>>>> 1.119 has introduced the arp_ifinit() call >> in order to >>>>> fix the problem >>>>>> that if_em won't send ARP when IP address >> is changed. >>>>>> I think we can further improve it as >> attached, say, >>>>> only do it when >>>>>> IFF_NOARP is not set. This should >> have no effect >>>>> for usual >>>>>> configuration but fix the problem when >> NOARP is the >>>>> desired behavior. >>>>> That change was introduced by me. I guess the >> root cause of >>>>> the >>>>> problem was long initialization time of >> hardware which in >>>>> turn >>>>> resulted in unbearable boot time when >> multiple-alias >>>>> addresses are >>>>> assigned to em(4). I don't remember >> details,though. >>>>> Since we're in the release cycle, the change >> you suggested >>>>> would be >>>>> quick fix for 8.0. I think em(4)/igb(4) should >> remove >>>>> SIOCSIFADDR >>>>> handling in driver which is layering >> violation. >>>> There are 2 kinds of programmers; those who do >> things "correctly', >>>> and those that do things that work. >>>> >>>> 99.99999% of the people will be using ARPs, so >> don't be silly and >>>> break the driver to solve a case that almost >> no-one cares about please. >> Cisco.Ironport runs 50% (2 out of 4) of their em >> interfaces in noarp >> mode. > > > Ah, are they running Jack's drivers unmodified? Seems unlikely. well they will be when they go to 8. They stay a revision or two back for stability reasons of course. why wouldn't they? > > NOARP does work. Does your network catch on fire if the interface sends > an ARP out? Does equipment start failing like dominos? well if your network sniffer started responding to arps, how would you feel? > > My point was don't make ARPs not work (the reason the "hack" is in > there is to make something work better) to preserve some fantasy of > "layering" that went out with the 8-track player. The check for > the NOARP flag is a better solution until the subsystem works the > way its supposed to work. > > Barney > > > From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 21:45:10 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26198106568C for ; Wed, 19 Aug 2009 21:45:10 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56407.mail.re3.yahoo.com (web56407.mail.re3.yahoo.com [216.252.111.86]) by mx1.freebsd.org (Postfix) with SMTP id C898C8FC59 for ; Wed, 19 Aug 2009 21:45:09 +0000 (UTC) Received: (qmail 51704 invoked by uid 60001); 19 Aug 2009 21:45:09 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250718308; bh=Nua4R20q0UxEM3PDI0NISXkM+5CDa6xlIoFP/sRTSfs=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=bXSGkahmqIG7wMO2uykSiSJwJafXDyc4QP4RYK7KYzVlkQd33q6/h87VTu4OuuglSV1H13b4Mgv3MDajEL2kwi57T67u2rgyktSIovYn7PVDUYWEZ1g7Rm3UDcZSVe58rbzvkWhgPh9qEFN/8ErPXB90/59a/53FquYrahcbS44= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=lbu9jQdT8SIvik078pJBuPkRW0ojLLcPe3lsJPcmS8NKNFh8S+XIk3bjyCvcctt0HzdHBnk8QRd5kgphW4BVTQrQoM1IAme/AuDMN5SLmo+gtEQZgPv5Da67sI6xRTi+0xkaZZhHd2sDWDE19MZ6VghoYVgg2c2uoAvsSvQnCU8=; Message-ID: <959379.51225.qm@web56407.mail.re3.yahoo.com> X-YMail-OSG: FY.gOZ4VM1nzxceTHyxPL1dxH1xcb_fDaaSI0ONWF4FYYC6tWFieSiQZTuYCFRiogY3Gg1Rvi_KbpZqqv6Sy66uoNPuUFyZ_qDa7.ESQn2yWVDNuKtmGufzEqvBlkVwAMY3KbXHXjUQwdcuwhCpMeHRoCnFe.Zsbq20Uh35DQP5L9jdpA1aw8f1zmC0Od2UGhh7.flFUv0rbxS8lz1RIa6Bu0Sr23GM0ugXmYT83LkwgxzQzn0FimjaDmEW.yfw3m4rYo_uSjahqYHMUJ_y8XLXserL31LmIjyklraf8ESe_CeiLxMLaiLEu8lgVWD15SfrkG9z21A-- Received: from [89.122.155.5] by web56407.mail.re3.yahoo.com via HTTP; Wed, 19 Aug 2009 14:45:08 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Wed, 19 Aug 2009 14:45:08 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: "H.Fazaeli" In-Reply-To: <4A8C15B7.5090404@sepehrs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 21:45:10 -0000 --- On Wed, 8/19/09, H.Fazaeli wrote: > From: H.Fazaeli > Subject: Re: em driver input errors > To: alexpalias-bsdnet@yahoo.com > Cc: freebsd-net@freebsd.org > Date: Wednesday, August 19, 2009, 6:09 PM > Have you tries fixed speed/duplex? Hello. Flow control is already disabled in the switch, and now I have configured fixed speed and duplex both on the switch and the network card. # ifconfig em0 em0: flags=8843 metric 0 mtu 1500 options=9b ether 00:xx:xx:xx:xx:xx media: Ethernet 1000baseTX status: active I will let you know if this fixes the problem; however, I seem to remember that speed/duplex problems usually result in lots of collisions and CRC errors, but I get very little of those (no collisions in fact): # netstat -nI em0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em0 1500 00:xx:xx:xx:xx:xx 10448059238 2746221 12382379424 0 0 Also: r# sysctl dev.em.0.stats=1 ; dmesg | tail -21 dev.em.0.stats: -1 -> -1 em0: Excessive collisions = 0 em0: Sequence errors = 0 em0: Defer count = 0 em0: Missed Packets = 2746217 em0: Receive No Buffers = 4996579 em0: Receive Length Errors = 0 em0: Receive errors = 2 em0: Crc errors = 2 em0: Alignment errors = 0 em0: Collision/Carrier extension errors = 0 em0: RX overruns = 1134 em0: watchdog timeouts = 0 em0: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK MSIX IRQ = 0 em0: XON Rcvd = 0 em0: XON Xmtd = 0 em0: XOFF Rcvd = 0 em0: XOFF Xmtd = 0 em0: Good Packets Rcvd = 10443504721 em0: Good Packets Xmtd = 12377389802 em0: TSO Contexts Xmtd = 0 em0: TSO Contexts Failed = 0 This seems to suggest that the number of errors seen in ifconfig is linked to the "missed packets" statistic that's produced by the sysctl. Note that there are only 2 CRC errors and 2 other errors; I can't see anything that corresponds to the "Receive no buffers" figure; are those packets dropped by the NIC itself? Alex From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 21:49:55 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22521106568D for ; Wed, 19 Aug 2009 21:49:55 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56406.mail.re3.yahoo.com (web56406.mail.re3.yahoo.com [216.252.111.85]) by mx1.freebsd.org (Postfix) with SMTP id C33D78FC51 for ; Wed, 19 Aug 2009 21:49:54 +0000 (UTC) Received: (qmail 54803 invoked by uid 60001); 19 Aug 2009 21:49:54 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250718594; bh=16KzcVRmGhrDKsdg1sGt+6gThSuXLwkJKE8Ue6oIICQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=NDLmwUsTG4wLg4Tkxr/DjurEFvZwcSioCxHNcLSbufGKsjjDdbJKSbiioQM5Q88m6cO+U0uxOu5IjYu8RUXeySC/DwVKi4RkH85AtDrz7GrdvJGoOsz5k+ddjI9xnuD9eTgUNdAqWpEfb6AC8lkGNhGHJBjBHVivnbSwFLebaW8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=T3hQaNJk4AadcPhV8Qm4ob1Ge2jI9VCbDZK7Q12YlWNsBCzE10hxlOqCo3qfIr0dLnwJB92VSxlyzPfYCH17wvkOJVNg1JN8LeaVOr+Cr8GXYSUH7rUkMsNxEZRdPGiw+wDsEe8e2UMNzWdqTSfHa84boeORUp9oJ2IwN6kOwGc=; Message-ID: <107062.54569.qm@web56406.mail.re3.yahoo.com> X-YMail-OSG: TArlVyEVM1nglfvVHBbshC3IaIo7qebP.gbqAewc6j1titqnx_bDz_xIXl38.Z3YNfiS0M1f9Pc4.deYBpKparB9SbDeet56Gi0AsRu2oQ8D7BhDWNscr_tnhQh4YwrJ_wYO92hpa88Ufs.sEmuKHjA_x3zDKhpIfCbOB1BmVWRA67.rQB.fem2cZW9mIkG171LquIIjszEVR02Q7.w6QKUCiSJKxIyJAMxZe8O_7rjn2AafGDVVcR0LbHRYUMSomLc.TAGiJKMTGXTybHc9BzTLyIfZI4ZeQ67kmQeDFm7PEUbCCJF.F8w3bdr49epOcVg1mMVGfibLJLrGaBI2Qme4lTzt0MO7k3PCdA-- Received: from [89.122.155.5] by web56406.mail.re3.yahoo.com via HTTP; Wed, 19 Aug 2009 14:49:53 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Wed, 19 Aug 2009 14:49:53 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: "H.Fazaeli" In-Reply-To: <959379.51225.qm@web56407.mail.re3.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 21:49:55 -0000 --- On Thu, 8/20/09, alexpalias-bsdnet@yahoo.com wrote: > From: alexpalias-bsdnet@yahoo.com > Subject: Re: em driver input errors > To: "H.Fazaeli" > Cc: freebsd-net@freebsd.org > Date: Thursday, August 20, 2009, 12:45 AM > > I will let you know if this fixes the problem; however, I > seem to remember that speed/duplex problems usually result > in lots of collisions and CRC errors, but I get very little > of those (no collisions in fact): Nope; still seeing errors (a burst of 2500 errors in 10 seconds recently). Thank you for the suggestion though Alex From owner-freebsd-net@FreeBSD.ORG Wed Aug 19 22:35:07 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59F53106568C for ; Wed, 19 Aug 2009 22:35:07 +0000 (UTC) (envelope-from fazaeli@sepehrs.com) Received: from sepehrs.com (sepehrs.com [213.217.59.98]) by mx1.freebsd.org (Postfix) with ESMTP id 51D8B8FC6B for ; Wed, 19 Aug 2009 22:35:05 +0000 (UTC) Received: from [192.168.4.180] ([192.168.3.1]) by mail (8.14.3/8.14.3) with ESMTP id n7JF60QF062753; Wed, 19 Aug 2009 19:36:00 +0430 (IRDT) Message-ID: <4A8C15B7.5090404@sepehrs.com> Date: Wed, 19 Aug 2009 19:39:43 +0430 From: "H.Fazaeli" User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: alexpalias-bsdnet@yahoo.com References: <11420.28890.qm@web56404.mail.re3.yahoo.com> In-Reply-To: <11420.28890.qm@web56404.mail.re3.yahoo.com> Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 22:35:07 -0000 Have you tries fixed speed/duplex? alexpalias-bsdnet@yahoo.com wrote: > Good day > > I'm running a FreeBSD 7.2 router and I am seeing a lot of input errors on one of the em interfaces (em0), coupled with (at approximately the same times) much fewer errors on em1 and em2. Monitoring is done with SNMP from another machine, and the CPU load as reported via SNMP is mostly below 30%, with a couple of spikes up to 35%. > > Software description: > > - FreeBSD 7.2-RELEASE-p2, amd64 > - bsnmpd with modules: hostres and (from ports) snmp_ucd > - quagga 0.99.12 (running only zebra and bgpd) > - netgraph (ng_ether and ng_netflow) > > Hardware description: > > - Dell machine, dual Xeon 3.20 GHz, 4 GB RAM > - 2 x built-in gigabit interfaces (em0, em1) > - 1 x dual-port gigabit interface, PCI-X (em2, em3) [see pciconf near the end] > > > The machine receives the global routing table ("netstat -nr | wc -l" gives 289115 currently). > > All of the em interfaces are just configured "up", with various vlan interfaces on them. Note that I use "kpps" to mean "thousands of packets per second", sorry if that's the wrong shorthand. > > - em0 sees a traffic of 10...22 kpps in, and 15...35 kpps out. In bits, it's 30...120Mbits/s in, and 100...210Mbits/s out. Vlans configured are vlan100 and vlan200, and most of the traffic is on vlan100 (vlan200 sees 4kpps in / 0.5kpps out maximum, with the average at about one third of this). em0 is the external interface, and its traffic corresponds to the sum of traffic through em1 and em2 > > - em1 has 5 vlans, and sees about 22kpps in / 11kpps out (maximum) > > - em2 has a single VLAN, and sees about 4...13kpps both in and out (almost equal in/out during most of the day) > > - em3 is a backup interface, with 2 VLANS, and is the only one which has seen no errors. > > Only the vlans on em0 are analyzed by ng_netflow, and the errors I'm seeing have started appearing days before netgraph was even loaded in the kernel. > > Tuning done: > > /boot/loader.conf: > hw.em.rxd=4096 > hw.em.txd=4096 > > Witout the above we were seeing way more errors, now they are reduced, but still come in bursts of over 1000 errors on em0. > > /etc/sysctl.conf: > net.inet.ip.fastforwarding=1 > dev.em.0.rx_processing_limit=300 > dev.em.1.rx_processing_limit=300 > dev.em.2.rx_processing_limit=300 > dev.em.3.rx_processing_limit=300 > > Still seeing errros, after some searching the mailing lists we also added: > > # the four lines below are repeated for em1, em2, em3 > dev.em.0.rx_int_delay=0 > dev.em.0.rx_abs_int_delay=0 > dev.em.0.tx_int_delay=0 > dev.em.0.tx_abs_int_delay=0 > > Still getting errors, so I also added: > > net.inet.ip.intr_queue_maxlen=4096 > net.route.netisr_maxqlen=1024 > > and > > kern.ipc.nmbclusters=655360 > > > Also tried with rx_processing_limit set to -1 on all em interfaces, still getting errors. > > Looking at the shape of the error and packet graphs, there seems to be a correlation between the number of packets per second on em0 and the height of the error "spikes" on the error graph. These spikes are spread throughout the day, with spaces (zones with no errors) of various lengths (10 minutes ... 2 hours spaces within the last 24 hours), but sometimes there are errors even in the lowest kpps times of the day. > > em0 and em1 error times are correlated, with all errors on the graph for em0 having a smaller corresponding error spike on em1 at the same time, and sometimes an error spike on em2. > > The old router was seeing about the same traffic, and had em0, em1, re0 and re1 network cards, and was only seeing errors on the em cards. It was running 7.2-PRERELEASE/i386 > > > Any suggestions would be greatly appreciated. Please note that this is a live router, and I can't reboot it (unless absolutely necessary). Tuning that can be applied without rebooting will be tried first. > > Here are some more details: > > Trimmed output of netstat -ni (sorry if there are line breaks): > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll > em0 1500 00:14:22:xx:xx:xx 19744458839 15494721 24284439443 0 0 > em1 1500 00:14:22:xx:xx:xx 12832245469 123181 10105031790 0 0 > em2 1500 00:04:23:xx:xx:xx 12082552403 10964 10339416865 0 0 > em3 1500 00:04:23:xx:xx:xx 79912337 0 48178737 0 0 > > Relevant part of pciconf -vl: > > em0@pci0:6:7:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 > vendor = 'Intel Corporation' > device = '82541EI Gigabit Ethernet Controller' > class = network > subclass = ethernet > em1@pci0:7:8:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 > vendor = 'Intel Corporation' > device = '82541EI Gigabit Ethernet Controller' > class = network > subclass = ethernet > em2@pci0:9:4:0: class=0x020000 card=0x10128086 chip=0x10108086 rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = '82546EB Dual Port Gigabit Ethernet Controller (Copper)' > class = network > subclass = ethernet > em3@pci0:9:4:1: class=0x020000 card=0x10128086 chip=0x10108086 rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = '82546EB Dual Port Gigabit Ethernet Controller (Copper)' > class = network > subclass = ethernet > > Kernel messages after sysctl dev.em.0.stats=1: > (note that I've removed the lines which only showed zeros in the second and third outputs) > > em0: Excessive collisions = 0 > em0: Sequence errors = 0 > em0: Defer count = 0 > em0: Missed Packets = 15435312 > em0: Receive No Buffers = 16446113 > em0: Receive Length Errors = 0 > em0: Receive errors = 1 > em0: Crc errors = 2 > em0: Alignment errors = 0 > em0: Collision/Carrier extension errors = 0 > em0: RX overruns = 96826 > em0: watchdog timeouts = 0 > em0: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK MSIX IRQ = 0 > em0: XON Rcvd = 0 > em0: XON Xmtd = 0 > em0: XOFF Rcvd = 0 > em0: XOFF Xmtd = 0 > em0: Good Packets Rcvd = 19002068797 > em0: Good Packets Xmtd = 23168462599 > em0: TSO Contexts Xmtd = 0 > em0: TSO Contexts Failed = 0 > > [later] > em0: Excessive collisions = 0 > em0: Missed Packets = 15459111 > em0: Receive No Buffers = 16447082 > em0: Receive errors = 1 > em0: Crc errors = 2 > em0: RX overruns = 96835 > em0: Good Packets Rcvd = 19165047284 > em0: Good Packets Xmtd = 23386976960 > > [later] > em0: Excessive collisions = 0 > em0: Missed Packets = 15470583 > em0: Receive No Buffers = 16447686 > em0: Receive errors = 1 > em0: Crc errors = 2 > em0: RX overruns = 96840 > em0: Good Packets Rcvd = 19255466068 > em0: Good Packets Xmtd = 23519004546 > > > Thank you for your time. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > -- Best regards. Hooman Fazaeli From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 03:10:03 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AC27106568C for ; Thu, 20 Aug 2009 03:10:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 213318FC5B for ; Thu, 20 Aug 2009 03:10:03 +0000 (UTC) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n7K3A24x028357 for ; Thu, 20 Aug 2009 03:10:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n7K3A2wC028356; Thu, 20 Aug 2009 03:10:02 GMT (envelope-from gnats) Date: Thu, 20 Aug 2009 03:10:02 GMT Message-Id: <200908200310.n7K3A2wC028356@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Ryan Steinmetz Cc: Subject: Re: kern/109733: [bge] bge link state issues [regression] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ryan Steinmetz List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 03:10:03 -0000 The following reply was made to PR kern/109733; it has been noted by GNATS. From: Ryan Steinmetz To: bug-followup@FreeBSD.org, donbrearley@hibbing.edu Cc: Subject: Re: kern/109733: [bge] bge link state issues [regression] Date: Wed, 19 Aug 2009 23:03:16 -0400 This issue appears to still exist in 7.2-RELEASE. The 3c996-sx doesn't function properly, nor does the remote side ever go into a link-up state. -r -- Ryan Steinmetz Lead Security/Systems Administrator Infrastructure Engineering Rochester Institute of Technology 585.475.5663 PGP: EF36 D45A 5CA9 28B1 A550 18CD A43C D111 7AD7 FAF2 From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 04:46:15 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F1BA106568C for ; Thu, 20 Aug 2009 04:46:15 +0000 (UTC) (envelope-from stef-list@memberwebs.com) Received: from mx.npubs.com (mail.npubs.com [94.75.203.100]) by mx1.freebsd.org (Postfix) with ESMTP id 193F58FC45 for ; Thu, 20 Aug 2009 04:46:14 +0000 (UTC) Received: from mx.npubs.com (avhost [94.75.203.103]) by mx.npubs.com (Postfix) with ESMTP id 9D8DC303982D for ; Thu, 20 Aug 2009 04:20:17 +0000 (UTC) Received: from sqlserver1 (unknown [74.82.45.12]) by mx.npubs.com (Postfix) with ESMTP id C8F913039754 for ; Thu, 20 Aug 2009 04:20:16 +0000 (UTC) From: Stef Walter User-Agent: Thunderbird 2.0.0.22 (X11/20090608) MIME-Version: 1.0 To: "freebsd-net@FreeBSD.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Message-Id: <20090820042016.C8F913039754@mx.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Thu, 20 Aug 2009 04:20:17 +0000 (UTC) Subject: ath0: ath_rx_proc: no mbuf! X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stef@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 04:46:15 -0000 I'm having a problem on an old FreeBSD 6.0 box, that's a wireless router, been running steadily for years. A short while ago (perhaps due to a change in traffic), every few hours, the wireless interface becomes unresponsive, and I started seeing thousands of lines like this in: ath0: ath_rx_proc: no mbuf! ath0: ath_rx_proc: no mbuf! ath0: ath_rx_proc: no mbuf! ath0: ath_rx_proc: no mbuf! ath0: ath_rx_proc: no mbuf! Has anyone seen this problem? An mbuf leak? Or perhaps fixed something like it in a later version of FreeBSD? Would upgrading the FreeBSD version fix this? It's in a really remote location, high on a tower, and tough to upgrade :( netstat -m shows: # netstat -m 104/19096/19200 mbufs in use (current/cache/total) 101/2971/3072/3072 mbuf clusters in use (current/cache/total/max) 0/3/1024 sfbufs in use (current/peak/max) 228K/10716K/10944K bytes allocated to network (current/cache/total) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines The mbufs are in fact all used up. I allocate more via kern.ipc.nmbclusters, and see the same behavior. 6.0-RELEASE-p5 FreeBSD 6.0-RELEASE-p5 #6: Thu May 18 18:02:20 UTC 2006 Thanks in advance, Stef From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 05:28:34 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A5D3106568B for ; Thu, 20 Aug 2009 05:28:34 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from ftp.translate.ru (ftp.translate.ru [77.221.156.50]) by mx1.freebsd.org (Postfix) with ESMTP id DDFC48FC3F for ; Thu, 20 Aug 2009 05:28:33 +0000 (UTC) Received: from desktop.home.serebryakov.spb.ru (85-142-52-164.well-com.net [85.142.52.164]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 39E7113DF46; Thu, 20 Aug 2009 09:12:36 +0400 (MSD) Date: Thu, 20 Aug 2009 09:12:30 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1038963609.20090820091230@serebryakov.spb.ru> To: Stef Walter In-Reply-To: <20090820042016.C8F913039754@mx.npubs.com> References: <20090820042016.C8F913039754@mx.npubs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@FreeBSD.org" , stef@memberwebs.com Subject: Re: ath0: ath_rx_proc: no mbuf! X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 05:28:34 -0000 Hello, Stef. You wrote 20 =E0=E2=E3=F3=F1=F2=E0 2009 =E3., 08:20:17: > I'm having a problem on an old FreeBSD 6.0 box, that's a wireless > router, been running steadily for years. > A short while ago (perhaps due to a change in traffic), every few hours, > the wireless interface becomes unresponsive, and I started seeing > thousands of lines like this in: > ath0: ath_rx_proc: no mbuf! > The mbufs are in fact all used up. I allocate more via > kern.ipc.nmbclusters, and see the same behavior. Same problem here on 7.2-STABLE, but incresaing kern.ipc.nmbclusters to 65536 helps. It seems, that when traffic is reauuly huge, system with ath need a lot of mbufs. At night, when traffic is almost zero, netstat -m shows a lot of free mbufs and clusters, so it seems, that there is no mbuf leaks. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 07:41:12 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E4F3106568B for ; Thu, 20 Aug 2009 07:41:12 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 100218FC62 for ; Thu, 20 Aug 2009 07:41:08 +0000 (UTC) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 399655C06F for ; Thu, 20 Aug 2009 15:41:06 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 2FC3C55CDCA3; Thu, 20 Aug 2009 15:41:05 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id NyiQX8++BVKw; Thu, 20 Aug 2009 15:39:59 +0800 (CST) Received: from charlie.delphij.net (c-67-188-2-183.hsd1.ca.comcast.net [67.188.2.183]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 549E355CDC86; Thu, 20 Aug 2009 15:39:45 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type; b=uK4SEhHG1OKhG/PDRmVXeAJJpgy1Woebwup1TqY25xVe07VPRhDZxWzaCTNYPIwQr cKthyayyk2dAhK7psskMg== Message-ID: <4A8CFDAF.1000309@delphij.net> Date: Thu, 20 Aug 2009 00:39:27 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.22 (X11/20090803) MIME-Version: 1.0 To: "freebsd-net@freebsd.org" X-Enigmail-Version: 0.96.0 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------000402020406030203090005" Subject: (just for fun) port of OpenBSD pf's sloppy mode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 07:41:12 -0000 This is a multi-part message in MIME format. --------------000402020406030203090005 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Since there is effort undergoing to port a newer pf version to FreeBSD, I think this work would not be useful for inclusion in -CURRENT. However, I'd like to share it here as someone may find it useful before the new pf code hits the tree. The patch can also be downloaded from my website: http://www.delphij.net/pf-sloppy.diff About this patch: When pf(4) is operating in a manner that not all packet would went through it, specifically, when being used in a DSR ("Direct Server Return") network, the strict TCP state tracking would prevent some packets from being able to pass through. This can exhibit as, when you upload files, the connection would stall at ~60KB (may differ if you have special TCP setting), or stalled connections. With this change, pf.conf would support a new syntax, i.e. "(sloppy)" as state flag, e.g.: pass in quick on em0 route-to { (em1 $server1), (em1 $server2) } round-robin proto tcp from any to $ext_ip port 80 keep state (sloppy) When enabled, the "sloppy" TCP FSM would be activated, which loosens the state check. When using this option, the backend server has to use its own mechanism to prevent ICMP teardown attack and/or insertion attacks, so please use caution and limit the use in cases where pf(4) won't see some packets in the connection. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkqM/a8ACgkQi+vbBBjt66BRSACfQaOY3gHdEhjhGO5bz1zYhdud NFMAmgLaVnzbBdA4ofj5helYkDtdqTds =N0nJ -----END PGP SIGNATURE----- --------------000402020406030203090005 Content-Type: text/plain; name="pf-sloppy.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="pf-sloppy.diff" Index: sys/contrib/pf/net/if_pfsync.c =================================================================== --- sys/contrib/pf/net/if_pfsync.c (revision 196398) +++ sys/contrib/pf/net/if_pfsync.c (working copy) @@ -465,7 +465,7 @@ pfsync_insert_net_state(struct pfsync_state *sp, u st->direction = sp->direction; st->log = sp->log; st->timeout = sp->timeout; - st->allow_opts = sp->allow_opts; + st->state_flags = sp->state_flags; bcopy(sp->id, &st->id, sizeof(st->id)); st->creatorid = sp->creatorid; @@ -1578,7 +1578,7 @@ pfsync_pack_state(u_int8_t action, struct pf_state sp->proto = st->proto; sp->direction = st->direction; sp->log = st->log; - sp->allow_opts = st->allow_opts; + sp->state_flags = st->state_flags; sp->timeout = st->timeout; if (flags & PFSYNC_FLAG_STALE) Index: sys/contrib/pf/net/pfvar.h =================================================================== --- sys/contrib/pf/net/pfvar.h (revision 196398) +++ sys/contrib/pf/net/pfvar.h (working copy) @@ -700,6 +700,7 @@ struct pf_rule { /* rule flags again */ #define PFRULE_IFBOUND 0x00010000 /* if-bound */ +#define PFRULE_STATESLOPPY 0x00020000 /* sloppy state tracking */ #define PFSTATE_HIWAT 10000 /* default state table size */ #define PFSTATE_ADAPT_START 6000 /* default adaptive timeout start */ @@ -800,7 +801,9 @@ struct pf_state { u_int8_t pad; #endif u_int8_t log; - u_int8_t allow_opts; + u_int8_t state_flags; +#define PFSTATE_ALLOWOPTS 0x01 +#define PFSTATE_SLOPPY 0x02 u_int8_t timeout; u_int8_t sync_flags; #define PFSTATE_NOSYNC 0x01 Index: sys/contrib/pf/net/if_pfsync.h =================================================================== --- sys/contrib/pf/net/if_pfsync.h (revision 196398) +++ sys/contrib/pf/net/if_pfsync.h (working copy) @@ -80,7 +80,7 @@ struct pfsync_state { u_int8_t proto; u_int8_t direction; u_int8_t log; - u_int8_t allow_opts; + u_int8_t state_flags; u_int8_t timeout; u_int8_t sync_flags; u_int8_t updates; Index: sys/contrib/pf/net/pf.c =================================================================== --- sys/contrib/pf/net/pf.c (revision 196398) +++ sys/contrib/pf/net/pf.c (working copy) @@ -253,6 +253,13 @@ int pf_test_fragment(struct pf_rule **, int, struct pfi_kif *, struct mbuf *, void *, struct pf_pdesc *, struct pf_rule **, struct pf_ruleset **); +int pf_tcp_track_full(struct pf_state_peer *, + struct pf_state_peer *, struct pf_state **, + struct pfi_kif *, struct mbuf *, int, + struct pf_pdesc *, u_short *, int *); +int pf_tcp_track_sloppy(struct pf_state_peer *, + struct pf_state_peer *, struct pf_state **, + struct pf_pdesc *, u_short *); int pf_test_state_tcp(struct pf_state **, int, struct pfi_kif *, struct mbuf *, int, void *, struct pf_pdesc *, u_short *); @@ -3528,7 +3535,10 @@ cleanup: s->nat_rule.ptr = nr; s->anchor.ptr = a; STATE_INC_COUNTERS(s); - s->allow_opts = r->allow_opts; + if (r->allow_opts) + s->state_flags |= PFSTATE_ALLOWOPTS; + if (r->rule_flag & PFRULE_STATESLOPPY) + s->state_flags |= PFSTATE_SLOPPY; s->log = r->log & PF_LOG_ALL; if (nr != NULL) s->log |= nr->log & PF_LOG_ALL; @@ -3925,7 +3935,10 @@ cleanup: s->nat_rule.ptr = nr; s->anchor.ptr = a; STATE_INC_COUNTERS(s); - s->allow_opts = r->allow_opts; + if (r->allow_opts) + s->state_flags |= PFSTATE_ALLOWOPTS; + if (r->rule_flag & PFRULE_STATESLOPPY) + s->state_flags |= PFSTATE_SLOPPY; s->log = r->log & PF_LOG_ALL; if (nr != NULL) s->log |= nr->log & PF_LOG_ALL; @@ -4238,7 +4251,10 @@ cleanup: s->nat_rule.ptr = nr; s->anchor.ptr = a; STATE_INC_COUNTERS(s); - s->allow_opts = r->allow_opts; + if (r->allow_opts) + s->state_flags |= PFSTATE_ALLOWOPTS; + if (r->rule_flag & PFRULE_STATESLOPPY) + s->state_flags |= PFSTATE_SLOPPY; s->log = r->log & PF_LOG_ALL; if (nr != NULL) s->log |= nr->log & PF_LOG_ALL; @@ -4525,7 +4541,10 @@ cleanup: s->nat_rule.ptr = nr; s->anchor.ptr = a; STATE_INC_COUNTERS(s); - s->allow_opts = r->allow_opts; + if (r->allow_opts) + s->state_flags |= PFSTATE_ALLOWOPTS; + if (r->rule_flag & PFRULE_STATESLOPPY) + s->state_flags |= PFSTATE_SLOPPY; s->log = r->log & PF_LOG_ALL; if (nr != NULL) s->log |= nr->log & PF_LOG_ALL; @@ -4666,166 +4685,16 @@ pf_test_fragment(struct pf_rule **rm, int directio } int -pf_test_state_tcp(struct pf_state **state, int direction, struct pfi_kif *kif, - struct mbuf *m, int off, void *h, struct pf_pdesc *pd, - u_short *reason) +pf_tcp_track_full(struct pf_state_peer *src, struct pf_state_peer *dst, + struct pf_state **state, struct pfi_kif *kif, struct mbuf *m, int off, + struct pf_pdesc *pd, u_short *reason, int *copyback) { - struct pf_state_cmp key; - struct tcphdr *th = pd->hdr.tcp; - u_int16_t win = ntohs(th->th_win); - u_int32_t ack, end, seq, orig_seq; - u_int8_t sws, dws; - int ackskew; - int copyback = 0; - struct pf_state_peer *src, *dst; + struct tcphdr *th = pd->hdr.tcp; + u_int16_t win = ntohs(th->th_win); + u_int32_t ack, end, seq, orig_seq; + u_int8_t sws, dws; + int ackskew; - key.af = pd->af; - key.proto = IPPROTO_TCP; - if (direction == PF_IN) { - PF_ACPY(&key.ext.addr, pd->src, key.af); - PF_ACPY(&key.gwy.addr, pd->dst, key.af); - key.ext.port = th->th_sport; - key.gwy.port = th->th_dport; - } else { - PF_ACPY(&key.lan.addr, pd->src, key.af); - PF_ACPY(&key.ext.addr, pd->dst, key.af); - key.lan.port = th->th_sport; - key.ext.port = th->th_dport; - } - - STATE_LOOKUP(); - - if (direction == (*state)->direction) { - src = &(*state)->src; - dst = &(*state)->dst; - } else { - src = &(*state)->dst; - dst = &(*state)->src; - } - - if ((*state)->src.state == PF_TCPS_PROXY_SRC) { - if (direction != (*state)->direction) { - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_SYNPROXY_DROP); - } - if (th->th_flags & TH_SYN) { - if (ntohl(th->th_seq) != (*state)->src.seqlo) { - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_DROP); - } -#ifdef __FreeBSD__ - pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, pd->dst, -#else - pf_send_tcp((*state)->rule.ptr, pd->af, pd->dst, -#endif - pd->src, th->th_dport, th->th_sport, - (*state)->src.seqhi, ntohl(th->th_seq) + 1, - TH_SYN|TH_ACK, 0, (*state)->src.mss, 0, 1, - 0, NULL, NULL); - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_SYNPROXY_DROP); - } else if (!(th->th_flags & TH_ACK) || - (ntohl(th->th_ack) != (*state)->src.seqhi + 1) || - (ntohl(th->th_seq) != (*state)->src.seqlo + 1)) { - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_DROP); - } else if ((*state)->src_node != NULL && - pf_src_connlimit(state)) { - REASON_SET(reason, PFRES_SRCLIMIT); - return (PF_DROP); - } else - (*state)->src.state = PF_TCPS_PROXY_DST; - } - if ((*state)->src.state == PF_TCPS_PROXY_DST) { - struct pf_state_host *src, *dst; - - if (direction == PF_OUT) { - src = &(*state)->gwy; - dst = &(*state)->ext; - } else { - src = &(*state)->ext; - dst = &(*state)->lan; - } - if (direction == (*state)->direction) { - if (((th->th_flags & (TH_SYN|TH_ACK)) != TH_ACK) || - (ntohl(th->th_ack) != (*state)->src.seqhi + 1) || - (ntohl(th->th_seq) != (*state)->src.seqlo + 1)) { - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_DROP); - } - (*state)->src.max_win = MAX(ntohs(th->th_win), 1); - if ((*state)->dst.seqhi == 1) - (*state)->dst.seqhi = htonl(arc4random()); -#ifdef __FreeBSD__ - pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, - &src->addr, -#else - pf_send_tcp((*state)->rule.ptr, pd->af, &src->addr, -#endif - &dst->addr, src->port, dst->port, - (*state)->dst.seqhi, 0, TH_SYN, 0, - (*state)->src.mss, 0, 0, (*state)->tag, NULL, NULL); - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_SYNPROXY_DROP); - } else if (((th->th_flags & (TH_SYN|TH_ACK)) != - (TH_SYN|TH_ACK)) || - (ntohl(th->th_ack) != (*state)->dst.seqhi + 1)) { - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_DROP); - } else { - (*state)->dst.max_win = MAX(ntohs(th->th_win), 1); - (*state)->dst.seqlo = ntohl(th->th_seq); -#ifdef __FreeBSD__ - pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, pd->dst, -#else - pf_send_tcp((*state)->rule.ptr, pd->af, pd->dst, -#endif - pd->src, th->th_dport, th->th_sport, - ntohl(th->th_ack), ntohl(th->th_seq) + 1, - TH_ACK, (*state)->src.max_win, 0, 0, 0, - (*state)->tag, NULL, NULL); -#ifdef __FreeBSD__ - pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, - &src->addr, -#else - pf_send_tcp((*state)->rule.ptr, pd->af, &src->addr, -#endif - &dst->addr, src->port, dst->port, - (*state)->src.seqhi + 1, (*state)->src.seqlo + 1, - TH_ACK, (*state)->dst.max_win, 0, 0, 1, - 0, NULL, NULL); - (*state)->src.seqdiff = (*state)->dst.seqhi - - (*state)->src.seqlo; - (*state)->dst.seqdiff = (*state)->src.seqhi - - (*state)->dst.seqlo; - (*state)->src.seqhi = (*state)->src.seqlo + - (*state)->dst.max_win; - (*state)->dst.seqhi = (*state)->dst.seqlo + - (*state)->src.max_win; - (*state)->src.wscale = (*state)->dst.wscale = 0; - (*state)->src.state = (*state)->dst.state = - TCPS_ESTABLISHED; - REASON_SET(reason, PFRES_SYNPROXY); - return (PF_SYNPROXY_DROP); - } - } - - if (((th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN) && - dst->state >= TCPS_FIN_WAIT_2 && - src->state >= TCPS_FIN_WAIT_2) { - if (pf_status.debug >= PF_DEBUG_MISC) { - printf("pf: state reuse "); - pf_print_state(*state); - pf_print_flags(th->th_flags); - printf("\n"); - } - /* XXX make sure it's the same direction ?? */ - (*state)->src.state = (*state)->dst.state = TCPS_CLOSED; - pf_unlink_state(*state); - *state = NULL; - return (PF_DROP); - } - if (src->wscale && dst->wscale && !(th->th_flags & TH_SYN)) { sws = src->wscale & PF_WSCALE_MASK; dws = dst->wscale & PF_WSCALE_MASK; @@ -4863,7 +4732,7 @@ int pf_change_a(&th->th_seq, &th->th_sum, htonl(seq + src->seqdiff), 0); pf_change_a(&th->th_ack, &th->th_sum, htonl(ack), 0); - copyback = 1; + *copyback = 1; } else { ack = ntohl(th->th_ack); } @@ -4915,7 +4784,7 @@ int pf_change_a(&th->th_seq, &th->th_sum, htonl(seq + src->seqdiff), 0); pf_change_a(&th->th_ack, &th->th_sum, htonl(ack), 0); - copyback = 1; + *copyback = 1; } end = seq + pd->p_len; if (th->th_flags & TH_SYN) @@ -4961,7 +4830,7 @@ int */ if (dst->seqdiff && (th->th_off << 2) > sizeof(struct tcphdr)) { if (pf_modulate_sack(m, off, pd, th, dst)) - copyback = 1; + *copyback = 1; } @@ -4980,7 +4849,7 @@ int if (dst->scrub || src->scrub) { if (pf_normalize_tcp_stateful(m, off, pd, reason, th, - *state, src, dst, ©back)) + *state, src, dst, copyback)) return (PF_DROP); } @@ -5082,7 +4951,7 @@ int if (dst->scrub || src->scrub) { if (pf_normalize_tcp_stateful(m, off, pd, reason, th, - *state, src, dst, ©back)) + *state, src, dst, copyback)) return (PF_DROP); } @@ -5128,6 +4997,7 @@ int src->seqhi = 1; src->max_win = 1; } else if (pf_status.debug >= PF_DEBUG_MISC) { +#if 0 printf("pf: BAD state: "); pf_print_state(*state); pf_print_flags(th->th_flags); @@ -5140,8 +5010,8 @@ int #else (*state)->packets[0], (*state)->packets[1], #endif - direction == PF_IN ? "in" : "out", - direction == (*state)->direction ? "fwd" : "rev"); + pd->dir == PF_IN ? "in" : "out", + pd->dir == (*state)->direction ? "fwd" : "rev"); printf("pf: State failure on: %c %c %c %c | %c %c\n", SEQ_GEQ(src->seqhi, end) ? ' ' : '1', SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws)) ? @@ -5150,13 +5020,255 @@ int (ackskew <= (MAXACKWINDOW << sws)) ? ' ' : '4', SEQ_GEQ(src->seqhi + MAXACKWINDOW, end) ?' ' :'5', SEQ_GEQ(seq, src->seqlo - MAXACKWINDOW) ?' ' :'6'); +#endif } REASON_SET(reason, PFRES_BADSTATE); return (PF_DROP); } /* Any packets which have gotten here are to be passed */ + return (PF_PASS); +} +int +pf_tcp_track_sloppy(struct pf_state_peer *src, struct pf_state_peer *dst, + struct pf_state **state, struct pf_pdesc *pd, u_short *reason) +{ + struct tcphdr *th = pd->hdr.tcp; + + if (th->th_flags & TH_SYN) + if (src->state < TCPS_SYN_SENT) + src->state = TCPS_SYN_SENT; + if (th->th_flags & TH_FIN) + if (src->state < TCPS_CLOSING) + src->state = TCPS_CLOSING; + if (th->th_flags & TH_ACK) { + if (dst->state == TCPS_SYN_SENT) { + dst->state = TCPS_ESTABLISHED; + if (src->state == TCPS_ESTABLISHED && + (*state)->src_node != NULL && + pf_src_connlimit(state)) { + REASON_SET(reason, PFRES_SRCLIMIT); + return (PF_DROP); + } + } else if (dst->state == TCPS_CLOSING) { + dst->state = TCPS_FIN_WAIT_2; + } else if (src->state == TCPS_SYN_SENT && + dst->state < TCPS_SYN_SENT) { + /* + * Handle a special sloppy case where we only see one + * half of the connection. If there is a ACK after + * the initial SYN without ever seeing a packet from + * the destination, set the connection to established. + */ + dst->state = src->state = TCPS_ESTABLISHED; + if ((*state)->src_node != NULL && + pf_src_connlimit(state)) { + REASON_SET(reason, PFRES_SRCLIMIT); + return (PF_DROP); + } + } else if (src->state == TCPS_CLOSING && + dst->state == TCPS_ESTABLISHED && + dst->seqlo == 0) { + /* + * Handle the closing of half connections where we + * don't see the full bidirectional FIN/ACK+ACK + * handshake. + */ + dst->state = TCPS_CLOSING; + } + } + if (th->th_flags & TH_RST) + src->state = dst->state = TCPS_TIME_WAIT; + + /* update expire time */ + (*state)->expire = time_second; + if (src->state >= TCPS_FIN_WAIT_2 && + dst->state >= TCPS_FIN_WAIT_2) + (*state)->timeout = PFTM_TCP_CLOSED; + else if (src->state >= TCPS_CLOSING && + dst->state >= TCPS_CLOSING) + (*state)->timeout = PFTM_TCP_FIN_WAIT; + else if (src->state < TCPS_ESTABLISHED || + dst->state < TCPS_ESTABLISHED) + (*state)->timeout = PFTM_TCP_OPENING; + else if (src->state >= TCPS_CLOSING || + dst->state >= TCPS_CLOSING) + (*state)->timeout = PFTM_TCP_CLOSING; + else + (*state)->timeout = PFTM_TCP_ESTABLISHED; + + return (PF_PASS); +} + + +/* XXXXX */ +int +pf_test_state_tcp(struct pf_state **state, int direction, struct pfi_kif *kif, + struct mbuf *m, int off, void *h, struct pf_pdesc *pd, + u_short *reason) +{ + struct pf_state_cmp key; + struct tcphdr *th = pd->hdr.tcp; + int copyback = 0; + struct pf_state_peer *src, *dst; + + key.af = pd->af; + key.proto = IPPROTO_TCP; + if (direction == PF_IN) { + PF_ACPY(&key.ext.addr, pd->src, key.af); + PF_ACPY(&key.gwy.addr, pd->dst, key.af); + key.ext.port = th->th_sport; + key.gwy.port = th->th_dport; + } else { + PF_ACPY(&key.lan.addr, pd->src, key.af); + PF_ACPY(&key.ext.addr, pd->dst, key.af); + key.lan.port = th->th_sport; + key.ext.port = th->th_dport; + } + + STATE_LOOKUP(); + + if (direction == (*state)->direction) { + src = &(*state)->src; + dst = &(*state)->dst; + } else { + src = &(*state)->dst; + dst = &(*state)->src; + } + + if ((*state)->src.state == PF_TCPS_PROXY_SRC) { + if (direction != (*state)->direction) { + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_SYNPROXY_DROP); + } + if (th->th_flags & TH_SYN) { + if (ntohl(th->th_seq) != (*state)->src.seqlo) { + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_DROP); + } +#ifdef __FreeBSD__ + pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, pd->dst, +#else + pf_send_tcp((*state)->rule.ptr, pd->af, pd->dst, +#endif + pd->src, th->th_dport, th->th_sport, + (*state)->src.seqhi, ntohl(th->th_seq) + 1, + TH_SYN|TH_ACK, 0, (*state)->src.mss, 0, 1, + 0, NULL, NULL); + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_SYNPROXY_DROP); + } else if (!(th->th_flags & TH_ACK) || + (ntohl(th->th_ack) != (*state)->src.seqhi + 1) || + (ntohl(th->th_seq) != (*state)->src.seqlo + 1)) { + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_DROP); + } else if ((*state)->src_node != NULL && + pf_src_connlimit(state)) { + REASON_SET(reason, PFRES_SRCLIMIT); + return (PF_DROP); + } else + (*state)->src.state = PF_TCPS_PROXY_DST; + } + if ((*state)->src.state == PF_TCPS_PROXY_DST) { + struct pf_state_host *src, *dst; + + if (direction == PF_OUT) { + src = &(*state)->gwy; + dst = &(*state)->ext; + } else { + src = &(*state)->ext; + dst = &(*state)->lan; + } + if (direction == (*state)->direction) { + if (((th->th_flags & (TH_SYN|TH_ACK)) != TH_ACK) || + (ntohl(th->th_ack) != (*state)->src.seqhi + 1) || + (ntohl(th->th_seq) != (*state)->src.seqlo + 1)) { + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_DROP); + } + (*state)->src.max_win = MAX(ntohs(th->th_win), 1); + if ((*state)->dst.seqhi == 1) + (*state)->dst.seqhi = htonl(arc4random()); +#ifdef __FreeBSD__ + pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, + &src->addr, +#else + pf_send_tcp((*state)->rule.ptr, pd->af, &src->addr, +#endif + &dst->addr, src->port, dst->port, + (*state)->dst.seqhi, 0, TH_SYN, 0, + (*state)->src.mss, 0, 0, (*state)->tag, NULL, NULL); + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_SYNPROXY_DROP); + } else if (((th->th_flags & (TH_SYN|TH_ACK)) != + (TH_SYN|TH_ACK)) || + (ntohl(th->th_ack) != (*state)->dst.seqhi + 1)) { + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_DROP); + } else { + (*state)->dst.max_win = MAX(ntohs(th->th_win), 1); + (*state)->dst.seqlo = ntohl(th->th_seq); +#ifdef __FreeBSD__ + pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, pd->dst, +#else + pf_send_tcp((*state)->rule.ptr, pd->af, pd->dst, +#endif + pd->src, th->th_dport, th->th_sport, + ntohl(th->th_ack), ntohl(th->th_seq) + 1, + TH_ACK, (*state)->src.max_win, 0, 0, 0, + (*state)->tag, NULL, NULL); +#ifdef __FreeBSD__ + pf_send_tcp(NULL, (*state)->rule.ptr, pd->af, + &src->addr, +#else + pf_send_tcp((*state)->rule.ptr, pd->af, &src->addr, +#endif + &dst->addr, src->port, dst->port, + (*state)->src.seqhi + 1, (*state)->src.seqlo + 1, + TH_ACK, (*state)->dst.max_win, 0, 0, 1, + 0, NULL, NULL); + (*state)->src.seqdiff = (*state)->dst.seqhi - + (*state)->src.seqlo; + (*state)->dst.seqdiff = (*state)->src.seqhi - + (*state)->dst.seqlo; + (*state)->src.seqhi = (*state)->src.seqlo + + (*state)->dst.max_win; + (*state)->dst.seqhi = (*state)->dst.seqlo + + (*state)->src.max_win; + (*state)->src.wscale = (*state)->dst.wscale = 0; + (*state)->src.state = (*state)->dst.state = + TCPS_ESTABLISHED; + REASON_SET(reason, PFRES_SYNPROXY); + return (PF_SYNPROXY_DROP); + } + } + + if (((th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN) && + dst->state >= TCPS_FIN_WAIT_2 && + src->state >= TCPS_FIN_WAIT_2) { + if (pf_status.debug >= PF_DEBUG_MISC) { + printf("pf: state reuse "); + pf_print_state(*state); + pf_print_flags(th->th_flags); + printf("\n"); + } + /* XXX make sure it's the same direction ?? */ + (*state)->src.state = (*state)->dst.state = TCPS_CLOSED; + pf_unlink_state(*state); + *state = NULL; + return (PF_DROP); + } + + if ((*state)->state_flags & PFSTATE_SLOPPY) { + if (pf_tcp_track_sloppy(src, dst, state, pd, reason) == PF_DROP) + return (PF_DROP); + } else { + if (pf_tcp_track_full(src, dst, state, kif, m, off, pd, reason, + ©back) == PF_DROP) + return (PF_DROP); + } + /* translate source/destination address, if necessary */ if (STATE_TRANSLATE(*state)) { if (direction == PF_OUT) @@ -5533,8 +5645,9 @@ pf_test_state_icmp(struct pf_state **state, int di copyback = 1; } - if (!SEQ_GEQ(src->seqhi, seq) || - !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws))) { + if (!((*state)->state_flags & PFSTATE_SLOPPY) && + (!SEQ_GEQ(src->seqhi, seq) || + !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws)))) { if (pf_status.debug >= PF_DEBUG_MISC) { printf("pf: BAD ICMP %d:%d ", icmptype, pd->hdr.icmp->icmp_code); @@ -7052,7 +7165,7 @@ pf_test(int dir, struct ifnet *ifp, struct mbuf ** done: if (action == PF_PASS && h->ip_hl > 5 && - !((s && s->allow_opts) || r->allow_opts)) { + !((s && s->state_flags & PFSTATE_ALLOWOPTS) || r->allow_opts)) { action = PF_DROP; REASON_SET(&reason, PFRES_IPOPTIONS); log = 1; @@ -7513,7 +7626,7 @@ pf_test6(int dir, struct ifnet *ifp, struct mbuf * done: /* handle dangerous IPv6 extension headers. */ if (action == PF_PASS && rh_cnt && - !((s && s->allow_opts) || r->allow_opts)) { + !((s && s->state_flags & PFSTATE_ALLOWOPTS) || r->allow_opts)) { action = PF_DROP; REASON_SET(&reason, PFRES_IPOPTIONS); log = 1; Index: contrib/pf/pfctl/parse.y =================================================================== --- contrib/pf/pfctl/parse.y (revision 196387) +++ contrib/pf/pfctl/parse.y (working copy) @@ -128,7 +128,7 @@ enum { PF_STATE_OPT_MAX, PF_STATE_OPT_NOSYNC, PF_S PF_STATE_OPT_MAX_SRC_STATES, PF_STATE_OPT_MAX_SRC_CONN, PF_STATE_OPT_MAX_SRC_CONN_RATE, PF_STATE_OPT_MAX_SRC_NODES, PF_STATE_OPT_OVERLOAD, PF_STATE_OPT_STATELOCK, - PF_STATE_OPT_TIMEOUT }; + PF_STATE_OPT_TIMEOUT, PF_STATE_OPT_SLOPPY }; enum { PF_SRCTRACK_NONE, PF_SRCTRACK, PF_SRCTRACK_GLOBAL, PF_SRCTRACK_RULE }; @@ -423,7 +423,7 @@ typedef struct { %token QUEUE PRIORITY QLIMIT RTABLE %token LOAD RULESET_OPTIMIZATION %token STICKYADDRESS MAXSRCSTATES MAXSRCNODES SOURCETRACK GLOBAL RULE -%token MAXSRCCONN MAXSRCCONNRATE OVERLOAD FLUSH +%token MAXSRCCONN MAXSRCCONNRATE OVERLOAD FLUSH SLOPPY %token TAGGED TAG IFBOUND FLOATING STATEPOLICY ROUTE %token STRING %token PORTBINARY @@ -1891,6 +1891,14 @@ pfrule : action dir logquick interface route af p statelock = 1; r.rule_flag |= o->data.statelock; break; + case PF_STATE_OPT_SLOPPY: + if (r.rule_flag & PFRULE_STATESLOPPY) { + yyerror("state sloppy option: " + "multiple definitions"); + YYERROR; + } + r.rule_flag |= PFRULE_STATESLOPPY; + break; case PF_STATE_OPT_TIMEOUT: if (o->data.timeout.number == PFTM_ADAPTIVE_START || @@ -3216,6 +3224,14 @@ state_opt_item : MAXIMUM number { $$->next = NULL; $$->tail = $$; } + | SLOPPY { + $$ = calloc(1, sizeof(struct node_state_opt)); + if ($$ == NULL) + err(1, "state_opt_item: calloc"); + $$->type = PF_STATE_OPT_SLOPPY; + $$->next = NULL; + $$->tail = $$; + } | STRING number { int i; @@ -4101,6 +4117,13 @@ filter_consistent(struct pf_rule *r, int anchor_ca yyerror("keep state on block rules doesn't make sense"); problems++; } + if (r->rule_flag & PFRULE_STATESLOPPY && + (r->keep_state == PF_STATE_MODULATE || + r->keep_state == PF_STATE_SYNPROXY)) { + yyerror("sloppy state matching cannot be used with " + "synproxy state or modulate state"); + problems++; + } return (-problems); } @@ -4969,6 +4992,7 @@ lookup(char *s) { "scrub", SCRUB}, { "set", SET}, { "skip", SKIP}, + { "sloppy", SLOPPY}, { "source-hash", SOURCEHASH}, { "source-track", SOURCETRACK}, { "state", STATE}, Index: contrib/pf/pfctl/pf_print_state.c =================================================================== --- contrib/pf/pfctl/pf_print_state.c (revision 196387) +++ contrib/pf/pfctl/pf_print_state.c (working copy) @@ -294,6 +294,8 @@ print_state(struct pf_state *s, int opts) printf(", anchor %u", s->anchor.nr); if (s->rule.nr != -1) printf(", rule %u", s->rule.nr); + if (s->state_flags & PFSTATE_SLOPPY) + printf(", sloppy"); if (s->src_node != NULL) printf(", source-track"); if (s->nat_src_node != NULL) Index: contrib/pf/pfctl/pfctl_parser.c =================================================================== --- contrib/pf/pfctl/pfctl_parser.c (revision 196387) +++ contrib/pf/pfctl/pfctl_parser.c (working copy) @@ -873,6 +873,8 @@ print_rule(struct pf_rule *r, const char *anchor_c opts = 1; if (r->rule_flag & PFRULE_IFBOUND) opts = 1; + if (r->rule_flag & PFRULE_STATESLOPPY) + opts = 1; for (i = 0; !opts && i < PFTM_MAX; ++i) if (r->timeout[i]) opts = 1; @@ -939,6 +941,12 @@ print_rule(struct pf_rule *r, const char *anchor_c printf("if-bound"); opts = 0; } + if (r->rule_flag & PFRULE_STATESLOPPY) { + if (!opts) + printf(", "); + printf("sloppy"); + opts = 0; + } for (i = 0; i < PFTM_MAX; ++i) if (r->timeout[i]) { int j; --------------000402020406030203090005-- From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 09:08:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78FBB106568C for ; Thu, 20 Aug 2009 09:08:41 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.9]) by mx1.freebsd.org (Postfix) with ESMTP id 0EAB18FC57 for ; Thu, 20 Aug 2009 09:08:41 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-032-025.pools.arcor-ip.net [88.66.32.25]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0MKsym-1Me3dP3csh-000lLb; Thu, 20 Aug 2009 11:08:39 +0200 Received: (qmail 16825 invoked from network); 20 Aug 2009 09:08:39 -0000 Received: from kvm.laiers.local (HELO kvm.localnet) (192.168.4.200) by mx.laiers.local with SMTP; 20 Aug 2009 09:08:39 -0000 From: Max Laier Organization: FreeBSD To: freebsd-net@freebsd.org, d@delphij.net Date: Thu, 20 Aug 2009 11:08:38 +0200 User-Agent: KMail/1.12.0 (Linux/2.6.30-ARCH; KDE/4.3.0; x86_64; ; ) References: <4A8CFDAF.1000309@delphij.net> In-Reply-To: <4A8CFDAF.1000309@delphij.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <200908201108.39177.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+rMCYVYDLFe7knC20llxS1JNwALQuTDKUQTb9 mZNx0FdjtbRAmA5aJb0i/+EzY5N09Kp5voaBVfWIYDktrBuhdb QstOpTUZVdC+u5QaXuw/g== Cc: freebsd-pf@freebsd.org Subject: Re: (just for fun) port of OpenBSD pf's sloppy mode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 09:08:41 -0000 Nice Work! Thanks a lot! On Thursday 20 August 2009 09:39:27 Xin LI wrote: > Since there is effort undergoing to port a newer pf version to FreeBSD, > I think this work would not be useful for inclusion in -CURRENT. > However, I'd like to share it here as someone may find it useful before > the new pf code hits the tree. The patch can also be downloaded from my I disagree about the usefulness of this. As your patch doesn't affect ABI this could make it into 8.1 (which the all new pf won't). With SVN it is also much simpler to manage the vendor branch differences, now. > website: > > http://www.delphij.net/pf-sloppy.diff freebsd-pf@ test and provide feedback - I know people have asked about this in the past. > About this patch: > > When pf(4) is operating in a manner that not all packet would went > through it, specifically, when being used in a DSR ("Direct Server > Return") network, the strict TCP state tracking would prevent some > packets from being able to pass through. This can exhibit as, when you > upload files, the connection would stall at ~60KB (may differ if you > have special TCP setting), or stalled connections. > > With this change, pf.conf would support a new syntax, i.e. "(sloppy)" as > state flag, e.g.: > > pass in quick on em0 route-to { (em1 $server1), (em1 $server2) } > round-robin proto tcp from any to $ext_ip port 80 keep state (sloppy) > > When enabled, the "sloppy" TCP FSM would be activated, which loosens the > state check. When using this option, the backend server has to use its > own mechanism to prevent ICMP teardown attack and/or insertion attacks, > so please use caution and limit the use in cases where pf(4) won't see > some packets in the connection. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 09:42:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5769E106568D for ; Thu, 20 Aug 2009 09:42:41 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56407.mail.re3.yahoo.com (web56407.mail.re3.yahoo.com [216.252.111.86]) by mx1.freebsd.org (Postfix) with SMTP id F13CF8FC66 for ; Thu, 20 Aug 2009 09:42:40 +0000 (UTC) Received: (qmail 65150 invoked by uid 60001); 20 Aug 2009 09:42:40 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250761360; bh=8dV90FBevGoXkqghlLqzNbnOcEIHqxfPkWvLHuYlCLI=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=xwdE7hkXfBw86pm/YOhOfSRVXOhG9D124puVJvgegLFtyCV+r5JOTwNuNB2D/Y90hlCvn8OGsbwh/CTKWhtQsqSS07WjDyPCBp3KxduWAmWw4Foh+0vV0LkNNSv+GVMrahhM8lAzwdWdrMVllZlY6VH/APe0JIwQdWRE8RAsRYs= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=XZj2I7+oZGVroUnQVnEZWfhXXl6N9kx/Zws8wj/+/9ZhQ6LpE9mGR07y7nWRCAJJgjuQss1GsvYqS6eGw7XhvIR2ZFezkpSdKBI0JO4vY9cF0W+BxQr3a2GkfbdOML4Mt5qJ5d4ftxGUCYjteOsmHAaDYVpjsqzfbezq5mo3QS4=; Message-ID: <206677.64959.qm@web56407.mail.re3.yahoo.com> X-YMail-OSG: TQG_2n0VM1l_rToUADNb.KwHwVkRPlJ1Df8EZEfmBl0rGzkVmna4TlZ2jl3TCi8hGvsX6JkCBU_5TjcDgFGDJyLEcris4vPk01wNtj4H.DG8MZapLZFgqJ1sb_lxKibuKH9k2fJJXzYS65crbOWbqPWy05y2KeRtSUi3pYwP167CvzXUH4lzBGCIimIuRyOXUb0.ddGs.P7FCl8xYR4.tpOLOt1Iw7xZ4fsC8NufBxWqtDHf.to0zrSKJqaBxa9RoKSHoPQxRGzMJpXfyCXpvUwjeh_vbIvMcF_KCUW9XWvWetC2GqSuYfmxrZsGx9EFjP06z.8FCA-- Received: from [91.200.96.83] by web56407.mail.re3.yahoo.com via HTTP; Thu, 20 Aug 2009 02:42:40 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Thu, 20 Aug 2009 02:42:40 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: "H.Fazaeli" In-Reply-To: <4A8D15C9.6020002@sepehrs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 09:42:41 -0000 --- On Thu, 8/20/09, H.Fazaeli wrote: > From: H.Fazaeli > Subject: Re: em driver input errors > To: alexpalias-bsdnet@yahoo.com > Cc: freebsd-net@freebsd.org > Date: Thursday, August 20, 2009, 12:22 PM > > > can you provide sysctl dev.em.0.debug=1 output? em0: Adapter hardware address = 0xfffffffe80230320 em0: CTRL = 0x40f00241 RCTL = 0x8002 em0: Packet buffer = Tx=16k Rx=48k em0: Flow control watermarks high = 47104 low = 45604 em0: tx_int_delay = 0, tx_abs_int_delay = 0 em0: rx_int_delay = 0, rx_abs_int_delay = 0 em0: fifo workaround = 0, fifo_reset_count = 0 em0: hw tdh = 259, hw tdt = 259 em0: hw rdh = 603, hw rdt = 602 em0: Num Tx descriptors avail = 4095 em0: Tx Descriptors not avail1 = 0 em0: Tx Descriptors not avail2 = 0 em0: Std mbuf failed = 0 em0: Std mbuf cluster failed = 0 em0: Driver dropped packets = 0 em0: Driver tx dma failure in encap = 0 > There are posts that show the following workarounds have > fixed problems similar to yours on other interface types: > > - disable tso (ifconfig em0 -tso) There's no TSO flag active currently, but I did try it... I guess it's a no-op. Also, the machine is a router, meaning most of the packets are just being forwarded. As far as I understand, TSO is concerned with TCP connections with one local endpoint. Anyway, I did this (after looking in the README for the new intel driver): sysctl net.inet.tcp.tso=0 > - disable msi/msix (hw.em.enable_msi=0 in > /boot/loader.conf. There are also hw.pci.enable_msix and > hw.pci.enable_msi but I don't know their exact > effect). This, and the next one (new driver), will be much harder to try in a production environment... I'll try to find some test machines to torture. > You may also try upgrading to the latest intel em driver > (6.9.12). > http://downloadcenter.intel.com/Filter_Results.aspx?strOSs=52&strTypes=all&ProductID=1067&OSFullName=FreeBSD*&lang=eng > Thanks Alex From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 10:07:55 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3CD4106568C for ; Thu, 20 Aug 2009 10:07:55 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56404.mail.re3.yahoo.com (web56404.mail.re3.yahoo.com [216.252.111.83]) by mx1.freebsd.org (Postfix) with SMTP id 5B19D8FC3D for ; Thu, 20 Aug 2009 10:07:54 +0000 (UTC) Received: (qmail 64464 invoked by uid 60001); 20 Aug 2009 10:07:54 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250762874; bh=6q2FRV9CVjnKPrwnuu7Wf4k0iraXAQiHgIL+g5wb8uU=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=C6IKpro0IEVS77es41ar6OdBO2gHq+gc0Rf/fZRJxng4Q01jE34CZZOotrXitA2l5N+ihZu50KxjvXneWv2taYK9GisRZOJKMgEgmfMnonWNsUJX+9DUwTJodYiLw0cnIKvC6iGMwGNQaOZXi13o/2ZUR20stH5s3YjLpjDRQc0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=vRTq2cgr3+fH68T0vhbN2LWcQMLtoeaa99rr3HfEl1jZcnaQ83izOqrmKF1LdNeuQZEcBtf2KQKjJ0UD+jz4rrRstJcLLo9f8rPvmUB8IpG9A91JbnZajLNdRFEzy+cMyrOo2GVdu8M+zztmZsWzn3JXJhGUO5EFviK8JSXtnhA=; Message-ID: <550146.64358.qm@web56404.mail.re3.yahoo.com> X-YMail-OSG: AcgEfSsVM1kRMYHHSBqFdtA4sSGpPhEMSbJ1makL166IWuO166sm7exVY.rpTtaL7i4ATbTMxgktbKB6UdTBLXUbi6GIKDRhNcbDiCI2C_IJOWZ8Xtff6qgVNUKAnMTX9pQ9wc1ftqY8XfltAJ3ca3PLwVv6fHRu.6mgMKFFnPBYdZl7oCV_FKHgUOByuqkZ3fQZZyXlN5icqVRoKh3WzdT0TDfSL_ukAFGKEcnj_9Nn3a8J4KdaLHbjr4EzjzXn_tR87DeXtWnv3WLVQ_UXRPffK_lUc0.mxqpqloUWRXgZaKVfC0z3yoZXi_ryCW5tkTqUMciASH9CuWp0xwv1euZEI6qd3Q2uIl7XgnQS_P9vc29N2HOs Received: from [91.200.96.83] by web56404.mail.re3.yahoo.com via HTTP; Thu, 20 Aug 2009 03:07:54 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Thu, 20 Aug 2009 03:07:54 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: =?utf-8?B?0JTQvNC40YLRgNC40Lkg0JfQsNC80YPRgNCw0LXQsg==?= In-Reply-To: <000e01ca20e9$e19caa10$1e010a0a@in72.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 10:07:55 -0000 Hello.=0A=0A--- On Wed, 8/19/09, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9= =D0=97=D0=B0=D0=BC=D1=83=D1=80=D0=B0=D0=B5=D0=B2 = wrote:=0A=0A> From: =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=97=D0=B0= =D0=BC=D1=83=D1=80=D0=B0=D0=B5=D0=B2 =0A> Subject: = Re: em driver input errors=0A> To: alexpalias-bsdnet@yahoo.com=0A> Cc: free= bsd-net@freebsd.org=0A> Date: Wednesday, August 19, 2009, 7:27 PM=0A> Hello= Alex.=0A> =0A> What sheduler are you using? ULE or 4BSD=0A> Have you NIC I= RQ sharing with other hardware?=0A> What HZ value? 1000?=0A=0ASCHED_ULE, HZ= =3D1000:=0A=0Ahost# sysctl kern.sched.name kern.hz=0Akern.sched.name: ULE= =0Akern.hz: 1000=0Ahost#=0A=0A> > Thanks for the suggestion.=0A> > From a "= clean" box:=0A> > dev.em.0.rx_int_delay: 0=0A> > dev.em.0.tx_int_delay: 66= =0A> > dev.em.0.rx_abs_int_delay: 66=0A> > dev.em.0.tx_abs_int_delay: 66=0A= > > I reset all the values (errors still appearing), then=0A> tried your su= ggestion (rx_int_delay=3D600,=0A> rx_abs_int_delay=3D1000).=C2=A0 This has = reduced the number of=0A> >interrupts for em0 (from about 7200/sec to aroun= d=0A> 6500/sec).=C2=A0 After some time, I started getting errors=0A> again.= =0A> mmm, try the maximum value 67108, what hapens...=0A=0AI will try this = today when there's enough traffic to see errors.=0A=0A> > But that has made= me try this also:=0A> > dev.em.0.tx_int_delay=3D600=0A> > dev.em.0.tx_abs_= int_delay=3D1000=0A> I think it's a bad idea, but don't know because:=0A> >= Meaning using your suggested values for tx too.=C2=A0=0A> Now em0 is seein= g about 1800 interrupts/second, which is way=0A> better, but after some tim= e I saw errors >again...=0A> =0A> > From the output of "netstat -nI em0 -w = 5":=0A> maybe mistake, did you meen "netstat -w5 em0" ?=0A=0ANope, exactly = as in my mail, "netstat -nI em0 -w 5". It does take 5 seconds to produce m= eaningful output.=0A=0A> I have PPPoE concenrator based on S3000AHV motherb= oard with=0A> Core2Quad 6600 and four (to load all cores in CPU) Intel=0A> = PCI-E x1 and PCI-E x4 NIC's=0A> My load:=0A=0APretty impressive figures. A= nd "netstat -ni" shows 0 errors on all cards?=0A=0A> And i have't any probl= ems. I think i select the good=0A> hardware.=0A> =0A> > Interrupts total (a= s reported by systat):=C2=A0 around=0A> 13500/second.=C2=A0 I would estimat= e the old IRQ load at=0A> around 30000-35000/second, which doesn't seem too= >much=0A> to me, for a dual xeon machine.=0A=0A> I think it depends by mot= herbord, what full hardware=0A> specification are you using? with chips nam= es=0A=0AThe machine is a Dell PowerEdge 2850. According to its specs, the = chipset is Intel E7520. Two 64-bit Xeon processors at 3.20GHz, 4 GB RAM.= =0A =0A> > Speaking of which, I did compile the kernel with=0A> "options DE= VICE_POLLING", but enabling polling only made the=0A> errors appear more of= ten, and in greater >numbers.=0A> I don't use polling on FBSD 7.x, it's usa= ble on FBSD older=0A> versions=0A=0AI tried as many possibilities as I coul= d.=0A =0A> > - 1 x dual-port gigabit interface, PCI-X=0A> Maybe I have this= card. And it works unstable, i don't=0A> remember what happens, but i seen= by tcpdump "truncated IP,=0A> missing XX bytes"=0A=0ACurrently most errors= are on the motherboard-embedded em0 interface. Second is embedded em1. L= ast are em2 and em3 which are on the dual-port card (em2 under 170k errors = as opposed to 2.6M for em0, and em3 0 errors)=0A> Good luck. =0A=0AThanks! From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 11:58:28 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 340F7106568D for ; Thu, 20 Aug 2009 11:58:28 +0000 (UTC) (envelope-from gigabyte.tmn@gmail.com) Received: from mail-bw0-f206.google.com (mail-bw0-f206.google.com [209.85.218.206]) by mx1.freebsd.org (Postfix) with ESMTP id A9EBE8FC57 for ; Thu, 20 Aug 2009 11:58:27 +0000 (UTC) Received: by bwz2 with SMTP id 2so3977087bwz.43 for ; Thu, 20 Aug 2009 04:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:reply-to:from:to :cc:references:subject:date:organization:mime-version:content-type :content-transfer-encoding:x-priority:x-msmail-priority:x-mailer :x-mimeole; bh=J1N/WNsqgcSezylBCCCQax4pdATLDMWrqHmkv2rywiw=; b=xBr3i++IJ6SWZJJkE5m5cXnxpz52aM4/3X4z7/Mqf74QV+ztQUmt2JTLaKprKexOll E+KuwkrOLeyQbmLobbbX0HEQp3aVN4TNfmpGDFrLAKy/K6Fn3WmkuTay7lcwI0Y6vwbY 0eHLuH1CA6DRRbqdOp1tmbH1Ksfydqj438IrQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:reply-to:from:to:cc:references:subject:date:organization :mime-version:content-type:content-transfer-encoding:x-priority :x-msmail-priority:x-mailer:x-mimeole; b=jprBmedm7zzkcYHNjXY6lhS4MIOCMeGrBFcU+U9fLA7I16Fbn+icLp6IeAyvmIJLrZ pQJZZOrKWw+sgy9Of2Q29+VDYqfovB1eYTg4/rWm0bUc3OWryoObSejPxcTqQv1Ruf5U AaqfybdXiJQmfN7WuTozxVAwxwGbhwhQkcq4I= Received: by 10.204.160.66 with SMTP id m2mr6005544bkx.156.1250769506695; Thu, 20 Aug 2009 04:58:26 -0700 (PDT) Received: from dm (7.dynamic-n193.r72.info [91.211.193.7]) by mx.google.com with ESMTPS id d13sm1485871fka.2.2009.08.20.04.58.25 (version=SSLv3 cipher=RC4-MD5); Thu, 20 Aug 2009 04:58:26 -0700 (PDT) Message-ID: <010901ca218d$878276a0$1e010a0a@in72.ru> From: "Dmitriy Zamuraev" To: References: <550146.64358.qm@web56404.mail.re3.yahoo.com> Date: Thu, 20 Aug 2009 17:58:26 +0600 Organization: Netline NSP MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dmitriy Zamuraev List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 11:58:28 -0000 Hello Alex, > SCHED_ULE, HZ=1000: I use this too >> > From the output of "netstat -nI em0 -w 5": >> maybe mistake, did you meen "netstat -w5 em0" ? > Nope, exactly as in my mail, "netstat -nI em0 -w 5". It does take 5 > seconds to produce meaningful output. hmm, just comments: -n Show network addresses and ports as numbers. -l Shows listen sockets -nl with -w is wrong parameters. > I have PPPoE concenrator based on S3000AHV motherboard with > Core2Quad 6600 and four (to load all cores in CPU) Intel > PCI-E x1 and PCI-E x4 NIC's > My load: > ... > Pretty impressive figures. And "netstat -ni" shows 0 errors on all cards? Not exactly zero, but for uptime 155 days it seems to be ok. bras1 [/usr/home/dm]# netstat -i|grep em em0 1500 00:15:17:71:f8:52 2457503820 20175 2096211799 0 0 em1 1500 00:15:17:71:f8:52 1084492221 11188 909418060 0 0 em2 1500 00:15:17:71:f8:52 4212941427 29566 3500442287 0 0 em3 1500 00:15:17:71:f8:52 2143321197 0 1878792786 0 0 This counters was made by UDP flood, when dummynet can't process all packets and swi:net loads one core up to 100%. Yes, the dummynet on this machine, its bad idea but it's working stable. (After this incident the switches now control the flood attack) NOTE: the MAC is equal because i use if_lagg(4) for this interfaces for load all cores in CPU >> I think it depends by motherbord, what full hardware specification are >> you using? with chips names > The machine is a Dell PowerEdge 2850. According to its specs, the chipset > is Intel E7520. > Two 64-bit Xeon processors at 3.20GHz, 4 GB RAM. For you bandwidth this server must work fine. Check the UDP/ICMP or other flood on em0 when errors appear. What kind of device at the end of em0 copper cable? If this a manageable switch, and supports tools - try to investigate what happens when errors appear. From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 12:34:39 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5FBE3106568B for ; Thu, 20 Aug 2009 12:34:39 +0000 (UTC) (envelope-from alexpalias-bsdnet@yahoo.com) Received: from web56402.mail.re3.yahoo.com (web56402.mail.re3.yahoo.com [216.252.111.81]) by mx1.freebsd.org (Postfix) with SMTP id 081EE8FC3F for ; Thu, 20 Aug 2009 12:34:38 +0000 (UTC) Received: (qmail 76120 invoked by uid 60001); 20 Aug 2009 12:34:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250771678; bh=6iZBls7arz4MvA1RvZmEU7n6CUXNuLYMmMhS308HEfQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=T5C9acq9Gf0PWlJi5j4JM/vL3ikd4o/hgSjo8eA+t60bDMCyP6IZD/KLLl1yrkPEPONOU00F6xB3QUb0qxntk8p8NxwKz3OQreifd3CGnf6K83RHIZP6WXPZL9TliLpsc4OorV9z3S6TsTKKveUF2FNGC9pvZQTyV1TOh2tzWhk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kxsykg6lc9lwa6lJ20ikiMGRIP4j9To/LhFV6BJzE1oVTxZh0gmBkLzAOY4N2K5X3KU2tmx6hOg3/VGlOu5AlAapDWEfb655VXHQR2EyJexFgk1dEO3pDQ2KFTGva3rmSyoWtDMK7UtdsHrRiOMfSSLYWOMjF1HtBNaIH1Q3jw4=; Message-ID: <487959.75728.qm@web56402.mail.re3.yahoo.com> X-YMail-OSG: lNBOXW8VM1l4FCHsNuoNJ0ZYANsEeqoQB.RccUsPWxWxn8.1oyfKzFuH9KzpnfmW71N6EdxKHGg3addQUo9lNm9CGW.TOLH9MGgJxKmfW1glr6Gq8By_BSU_q1JYF4cMXrCI6ygJocPJnOQKbrWiQ_1r1oNuN_1diDnMf8Xa6h3U9WAYiSoEZWsmS9VjPHzQwr73Tj2b3wFmz_e.A__.k.7.w1kQyB94HwFUmSPJJAYmrgRbBk_5NHMx3DKceX9bniX62an3PrctWRcxVXKAeEM4hgkyS7Jjjabl_4xHpb0gFotnPyKbDivmDQ2kHARfpqhH9hK4iJd9yYreaTebx3sYOLVrWHNPB3w- Received: from [91.200.96.83] by web56402.mail.re3.yahoo.com via HTTP; Thu, 20 Aug 2009 05:34:38 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Thu, 20 Aug 2009 05:34:38 -0700 (PDT) From: alexpalias-bsdnet@yahoo.com To: Dmitriy Zamuraev In-Reply-To: <010901ca218d$878276a0$1e010a0a@in72.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexpalias-bsdnet@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 12:34:39 -0000 =0A=0A--- On Thu, 8/20/09, Dmitriy Zamuraev wrote:= =0A=0A> From: Dmitriy Zamuraev =0A> Subject: Re: em= driver input errors=0A> To: alexpalias-bsdnet@yahoo.com=0A> Cc: freebsd-ne= t@freebsd.org=0A> Date: Thursday, August 20, 2009, 2:58 PM=0A> Hello Alex,= =0A> =0A> > SCHED_ULE, HZ=3D1000:=0A> I use this too=0A> =0A=0A> > And "ne= tstat -ni"=0A> shows 0 errors on all cards?=0A> Not exactly zero, but for u= ptime 155 days it seems to be=0A> ok.=0A> bras1 [/usr/home/dm]# netstat -i|= grep em=0A> em0=A0 =A0 1500 =A0 =A0 =A000:15:17:71:f8:52 2457503820= 20175 2096211799 0=A0=A00=0A> em1=A0 =A0 1500 =A0 =A0 =A000:15:17:= 71:f8:52 1084492221 11188 909418060=A0=A0=A0=A00 0=0A> em2=A0 =A0 1500 =A0 =A0 =A000:15:17:71:f8:52 4212941427 29566 3500442287 0=A0=A0=A0=A00= =0A> em3=A0 =A0 1500 =A0 =A0 =A000:15:17:71:f8:52 2143321197=A0 =A0= =A0=A00 1878792786 0=A0 =A0=A0=A00=0A=0ADefinitely way better than my numbe= rs.=0A=0AName Mtu Network Address Ipkts Ierrs Opkts Oer= rs Coll=0Aem0 1500 blah1 11166666892 2776856 131945601= 32 0 0=0Aem1 1500 blah2 7420822703 462462 61850= 54971 0 0=0Aem2 1500 blah3 6070962640 167193 53= 41606634 0 0=0Aem3 1500 blah4 15183452 0 17= 424318 0 0=0A=0A# uptime=0A 3:29PM up 8 days, 11:19, 2 users, load= averages: 0.69, 0.78, 0.83=0A=0A=0A> This counters was made by UDP flood, = when dummynet can't=0A> process all packets and=0A> swi:net loads one core = up to 100%. Yes, the dummynet on=0A> this machine, its bad idea but it's=0A= > working stable. (After this incident the switches now=0A> control the flo= od attack)=0A> NOTE: the MAC is equal because i use if_lagg(4) for this=0A>= interfaces for load all cores in CPU=0A=0AInteresting. But I'm seeing thi= s during normal traffic. I can't remember if it was this machine, or the o= lder machine (also with em driver) when I did get a flood of over 90.000 pa= ckets/second, and the error rate jumped to around 50.000/second...=0A=0A> >= > I think it depends by motherbord, what full=0A> hardware specification ar= e you using? with chips names=0A> > The machine is a Dell PowerEdge 2850.= =A0 According=0A> to its specs, the chipset is Intel E7520.=0A> > Two 64-bi= t Xeon processors at 3.20GHz, 4 GB RAM.=0A> For you bandwidth this server m= ust work fine.=0A> Check the UDP/ICMP or other flood on em0 when errors=0A>= appear.=0A> What kind of device at the end of em0 copper cable?=0A> If thi= s a manageable switch, and supports tools - try to=0A> investigate what hap= pens when errors appear.=0A=0AIt's a ZyXEL managed switch, unfortunately SN= MP support doesn't seem to work very well on it. But even before I added t= his switch (when there was a Cisco Catalyst 3560 at the end of the cable) I= was seeing the error bursts.=0A=0ABut I will try and see if I detect anyth= ing on the switch end.=0A=0AThanks=0AAlex From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 12:39:40 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10347106568C for ; Thu, 20 Aug 2009 12:39:40 +0000 (UTC) (envelope-from sepron@gmail.com) Received: from mail-fx0-f210.google.com (mail-fx0-f210.google.com [209.85.220.210]) by mx1.freebsd.org (Postfix) with ESMTP id 9A0CB8FC51 for ; Thu, 20 Aug 2009 12:39:39 +0000 (UTC) Received: by fxm6 with SMTP id 6so3963871fxm.43 for ; Thu, 20 Aug 2009 05:39:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=m/yjOkAuCnTVsiT/lFgjkK7jth6E+HMQN/UojBlYHMo=; b=OV7Pb68EjwzmRURk2aNpm0FevCvwNFvfuj1WAmjhZxd9aH8JnHYSDx4hxhlNQS5kmz TppvdWYT1dsTLaXs/lxjbh7ZiT1mlmZJ6agptDu6rxXrz7pW+vfR9AX6KK3d6bHT/Voq FW99bHF4r4jpH2k26S4OLIhcSyF/FwWGx6aX8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=pqdGISZYhCoyWx28hEzC5/ucujxQ37doENXBSiTXMIlocAAT2FKZtWqORs+Fw9BDwD pk2azXA1E2Wec9t95gWURv1Pc3XOI9smaIfOFM3zKG9cohg/8TNjac/kALW0UPIUkJyl Svwd3KKVOXZlSMm/bcND6xrF1qFWtx29em/5c= MIME-Version: 1.0 Received: by 10.103.76.34 with SMTP id d34mr3031460mul.31.1250770248877; Thu, 20 Aug 2009 05:10:48 -0700 (PDT) Date: Thu, 20 Aug 2009 16:10:48 +0400 Message-ID: From: Sergey Pronin To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: em driver input errors X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 12:39:40 -0000 Try to set sysctl net.isr.direct=1 swi:net will not use 100% of your CPU. From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 12:49:39 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90928106568B for ; Thu, 20 Aug 2009 12:49:39 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n6b.bullet.mail.ac4.yahoo.com (n6b.bullet.mail.ac4.yahoo.com [76.13.13.76]) by mx1.freebsd.org (Postfix) with SMTP id 33E758FC51 for ; Thu, 20 Aug 2009 12:49:38 +0000 (UTC) Received: from [76.13.13.26] by n6.bullet.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 12:49:38 -0000 Received: from [76.13.10.161] by t3.bullet.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 12:49:38 -0000 Received: from [127.0.0.1] by omp102.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 12:49:38 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 521477.18747.bm@omp102.mail.ac4.yahoo.com Received: (qmail 25164 invoked by uid 60001); 20 Aug 2009 12:49:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250772578; bh=heZ5f0nY5RBnap70Sq6WfBTs5luW8Ob4/GIhnWOAIWo=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=godl/g8bU3b2FjWQOhVWX1Snnd37FXTA0HR7DqBsSCpp8E6OCzOg2ooZT0ouMALpM84X84mznc2OWBkm988fpSEObveXvFwSGkbdwNlOuhSVZluXLUfSi+BDFZHbvmOPNag6qVA3F6iwshyHWrwzbon98n7w/g1rpCWt579vYpI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=4+GAxJOiXU5X1P1umKqod51/9nVHThCFzvrl/06JSB1EzSi0+qGwp1oAPCzslVf7tW4nEeNovKbYgMGt6ntmpFRYDKhjP15sRIPoZsxldCHK8z5l3bjbk0SeyFAAuT0UW6TQGKY5fnTumbAPQZfpXMTOZLnx0bKpOZbzJzbJ9pg=; Message-ID: <435336.24858.qm@web63908.mail.re1.yahoo.com> X-YMail-OSG: oYusjDUVM1lB0K8.bk21VpKGJeg835yK9alMcQFrbNuCrW9U46._dp8AYrCHoZwo2L7pTHcu0VNhJQq7T1iZBDpxUfGXmml73oJqsrZGuQIL3OJxVJPjunTwE5Wu_a4TZB6o2vYoD91L8N8c3h2ZrgR5hMZ2kU13.wldTVKwL_i2mtY.rhCIRjFOR6qs2Y8EsL4rGg3cbHXsTpvAXHYmwGfFwoZrZQFaNguKJAJf90Cd09Lia2J0CPQ9OX6HVSENRqUWCmQRuKJLiF8ozyqFUpXH Received: from [66.176.162.245] by web63908.mail.re1.yahoo.com via HTTP; Thu, 20 Aug 2009 05:49:38 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Thu, 20 Aug 2009 05:49:38 -0700 (PDT) From: Barney Cordoba To: Manish Vachharajani In-Reply-To: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 12:49:39 -0000 =0A=0A--- On Wed, 8/19/09, Manish Vachharajani wrote:=0A=0A> From: Manish Vachharajani =0A>= Subject: Re: Dropped vs. missed packets in the ixgbe driver=0A> To: "Barne= y Cordoba" =0A> Cc: freebsd-net@freebsd.org=0A> D= ate: Wednesday, August 19, 2009, 2:46 PM=0A> Agreed, the errors are reporte= d but=0A> missed packets are not.=A0 The=0A> question is, is the correct fi= x to just add stats.mpc[0] to=0A> if_ierrors=0A> in that line or to add it = to if_iqdrops.=A0 The fix is=0A> easy once we=0A> agree on what the correct= behavior is.=0A> =0A> Manish=0A> =0A> > Barney wrote:=0A> >=0A> > if you l= ook in ixgbe_update_stats_counters at the=0A> bottom:=0A> >=0A> > =A0 =A0 = =A0 =A0ifp->if_ierrors =3D missed_rx +=0A> adapter->stats.crcerrs +=0A> > = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec;=0A> >=0A> > the errors = are added in.=0A> >=0A> > BC=0A=0AHuh? missed_rx are the missed packets. So= they are counted.=0A=0ABC=0A=0A=0A From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 13:09:52 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6734C1065691 for ; Thu, 20 Aug 2009 13:09:52 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n2a.bullet.mail.ac4.yahoo.com (n2a.bullet.mail.ac4.yahoo.com [76.13.13.65]) by mx1.freebsd.org (Postfix) with SMTP id 188DC8FC6C for ; Thu, 20 Aug 2009 13:09:51 +0000 (UTC) Received: from [76.13.13.26] by n2.bullet.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 13:09:51 -0000 Received: from [76.13.10.178] by t3.bullet.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 13:09:51 -0000 Received: from [127.0.0.1] by omp119.mail.ac4.yahoo.com with NNFMP; 20 Aug 2009 13:09:51 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 387296.63071.bm@omp119.mail.ac4.yahoo.com Received: (qmail 86629 invoked by uid 60001); 20 Aug 2009 13:09:51 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250773791; bh=PqFD6SJ6QYTMdBIJyNEyiUVPoFfFniO9yIlDJ8a7mzI=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=IyHrEpPosJrSCI+5O5qlSfNiHO4xzeJcuCUT9Jqp1ir5gMmljpzF32Y1rBwC/0LJxckhque+PxHwHyKgGdAENgrUh99Z4/DL745howwwr5PGLffZGqpjCc2Ug7G82/Knn4trFHPUZbUlY1vERm84PgncM7LEimUKA6dVAgWzH5Q= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=T5pxhMh/n1XdDTeoYp8VAFypWiiSCfW3gIQjWw5APOVRNpzr+UkSXggHf8wrD2gEvMU/Wnf10zQ8FxpWRvC1b95ILw74q//sxT8XL4saF+EJxBEZibaTJH+ai6IqKUOQ5YyKjTxoSMJnT8PRxjS0OHO5W2/ng8LcR22B2Ts7tmY=; Message-ID: <233049.86158.qm@web63904.mail.re1.yahoo.com> X-YMail-OSG: Em5l1iEVM1nrpwYuXxpTIgaOg5T5i2cdXK8Lk26JzbW4jWEMoPP_UAyADBYsoCAH_yw1KcNbCvu8.j3rhueNGaLJRxir6PrceMV85THxewbAC6SRn8SN7..W26uJNyw.TieO1v808RRxBEKyfekwTKqWNmmd045Tyss4u71dprDetOk9bmagxjF06caVORR_4wUnLb.WCc86SkTWaxXIE1bmy1LYbYjJKsz1P.0uk03fpPzEfbm1APXmq4BoHpaW_j60LDG81Rw152eVWwRoi7RWmLa7KajW2KDvZGCJ34jMnpDuB0iR4Ur.jlxJmw-- Received: from [66.176.162.245] by web63904.mail.re1.yahoo.com via HTTP; Thu, 20 Aug 2009 06:09:51 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.1 Date: Thu, 20 Aug 2009 06:09:51 -0700 (PDT) From: Barney Cordoba To: Julian Elischer In-Reply-To: <4A8C53AD.1040102@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: pyunyh@gmail.com, David Christensen , d@delphij.net, "freebsd-net@freebsd.org" , Jack Vogel , Jack F Vogel , yongari@freebsd.org Subject: Re: [PATCH] Fix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of NOARP flag] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 13:09:52 -0000 =0A=0A--- On Wed, 8/19/09, Julian Elischer wrote:=0A= =0A> From: Julian Elischer =0A> Subject: Re: [PATCH] F= ix for e1000 (em/igb) NOARP issue [Was Re: em(4): sending ARP regardless of= NOARP flag]=0A> To: "Barney Cordoba" =0A> Cc: d@= delphij.net, pyunyh@gmail.com, "David Christensen" , = "freebsd-net@freebsd.org" , "Jack Vogel" , "Jack F Vogel" , yongari@freebsd.org=0A> Date:= Wednesday, August 19, 2009, 3:34 PM=0A> Barney Cordoba wrote:=0A> > =0A> >= --- On Tue, 8/18/09, Julian Elischer =0A> wrote:=0A> = > =0A> >> From: Julian Elischer =0A> >> Subject: Re: [= PATCH] Fix for e1000 (em/igb) NOARP=0A> issue [Was Re: em(4): sending ARP r= egardless of NOARP flag]=0A> >> To: d@delphij.net=0A> >> Cc: pyunyh@gmail.c= om,=0A> "Barney Cordoba" ,=0A> "David Christensen= " ,=0A> "freebsd-net@freebsd.org"=0A> ,=0A> "Jack Vogel" ,=0A> "Jack F Vogel" ,=0A> yongari@freebsd.org=0A> >> Date: Tuesday, August 18, 2009,= 6:55 PM=0A> >> Xin LI wrote:=0A> >>> -----BEGIN PGP SIGNED MESSAGE-----=0A= > >>> Hash: SHA1=0A> >>> =0A> >>> Barney Cordoba wrote:=0A> >>>> --- On Tue= , 8/18/09, Pyun YongHyeon =0A> >> wrote:=0A> >>>>> From: = Pyun YongHyeon =0A> >>>>> Subject: Re: [PATCH] Fix for e1= 000=0A> (em/igb)=0A> >> NOARP issue [Was Re: em(4): sending ARP regardless= =0A> of NOARP=0A> >> flag]=0A> >>>>> To: "Xin LI" =0A>= >>>>> Cc: "Barney Cordoba" ,=0A> >> "David Chris= tensen" ,=0A> >> "d@delphij..net"=0A> >> ,=0A> >> "freebsd-net@freebsd.org"=0A> >> ,=0A>= >> "Jack Vogel" ,=0A> >> "Jack F Vogel" ,=0A> >> yongari@freebsd.org,=0A> >> "Julian Elischer" =0A> >>>>> Date: Tuesday, August 18, 2009, 5:49=0A> PM=0A> >>>>> On Tue,= Aug 18, 2009 at 02:03:37PM=0A> >>>>> -0700, Xin LI wrote:=0A> >>>>>> -----= BEGIN PGP SIGNED=0A> MESSAGE-----=0A> >>>>>> Hash: SHA1=0A> >>>>>> =0A> >>>= >>> Hi, Jack,=0A> >>>>>> =0A> >>>>>> I have looked into the code=0A> histor= y and=0A> >> found that=0A> >>>>> sys/dev/em/if_em.c,v=0A> >>>>>> 1.119 has= introduced the=0A> arp_ifinit() call=0A> >> in order to=0A> >>>>> fix the = problem=0A> >>>>>> that if_em won't send ARP when IP=0A> address=0A> >> is = changed.=0A> >>>>>> I think we can further improve it=0A> as=0A> >> attache= d, say,=0A> >>>>> only do it when=0A> >>>>>> IFF_NOARP is not set.=A0 This= =0A> should=0A> >> have no effect=0A> >>>>> for usual=0A> >>>>>> configurat= ion but fix the problem=0A> when=0A> >> NOARP is the=0A> >>>>> desired beha= vior.=0A> >>>>> That change was introduced by me. I=0A> guess the=0A> >> ro= ot cause of=0A> >>>>> the=0A> >>>>> problem was long initialization time=0A= > of=0A> >> hardware which in=0A> >>>>> turn=0A> >>>>> resulted in unbearab= le boot time when=0A> >> multiple-alias=0A> >>>>> addresses are=0A> >>>>> a= ssigned to em(4). I don't remember=0A> >> details,though.=0A> >>>>> Since w= e're in the release cycle, the=0A> change=0A> >> you suggested=0A> >>>>> wo= uld be=0A> >>>>> quick fix for 8.0. I think=0A> em(4)/igb(4) should=0A> >> = remove=0A> >>>>> SIOCSIFADDR=0A> >>>>> handling in driver which is layering= =0A> >> violation.=0A> >>>> There are 2 kinds of programmers; those=0A> who= do=0A> >> things "correctly',=0A> >>>> and those that do things that work.= =0A> >>>> 99.99999% of the people will be using=0A> ARPs, so=0A> >> don't = be silly and=0A> >>>> break the driver to solve a case that=0A> almost=0A> = >> no-one cares about please.=0A> >> Cisco.Ironport=A0 runs 50% (2 out of 4= ) of=0A> their em=0A> >> interfaces in noarp mode.=0A> > =0A> > =0A> > Ah, = are they running Jack's drivers unmodified? Seems=0A> unlikely.=0A> =0A> we= ll they will be when they go to 8.=0A> They stay a revision or two back for= stability reasons of=0A> course.=0A> why wouldn't they?=0A=0ABecause the g= eneric drivers generally suck rocks? And from all of=0Athe complaints, it c= ertainly doesn't seem that they are stable.=0A=0A> =0A> =0A> > =0A> > NOARP= does work. Does your network catch on fire if=0A> the interface sends=0A> = > an ARP out? Does equipment start failing like dominos?=0A> =0A> =0A> well= if your network sniffer started responding to arps,=0A> how would you feel= ?=0A=0AThe problem here is that an ARP is sent on address initialization. S= o=0Awhat does that have to do with responding to ARPs? Do you usually=0Agiv= e a network sniffer an IP address on the monitor port? You seem=0Ato be arg= uing the case that NOARP isn't completely useless, which=0Ahas no relevance= to this discussion.=0A=0AOf course my solution would be to simply comment = out the arp init code,=0Abecause trying to convince 34 bearded gurus of the= correct way to do=0Ait is way too much work. :)=0A=0ABC=0A=0A=0A> > =0A> >= My point was don't make ARPs not work (the reason the=0A> "hack" is in=0A>= > there is to make something work better) to preserve=0A> some fantasy of= =0A> > "layering" that went out with the 8-track player. The=0A> check for = the NOARP flag is a better solution until the=0A> subsystem works the way i= ts supposed to work.=0A> > =0A> > Barney=0A> > =0A> > =0A> >=A0 =A0 =A0=A0= =A0=0A> =0A> =0A=0A=0A From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 16:17:04 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD6F1106568E for ; Thu, 20 Aug 2009 16:17:04 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outZ.internet-mail-service.net (outz.internet-mail-service.net [216.240.47.249]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD8E8FC5B for ; Thu, 20 Aug 2009 16:17:04 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 86386A1EA7; Thu, 20 Aug 2009 09:17:04 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 5974E2D6011; Thu, 20 Aug 2009 09:17:03 -0700 (PDT) Message-ID: <4A8D76FE.7040302@elischer.org> Date: Thu, 20 Aug 2009 09:17:02 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: Max Laier References: <4A8CFDAF.1000309@delphij.net> <200908201108.39177.max@love2party.net> In-Reply-To: <200908201108.39177.max@love2party.net> Content-Type: multipart/mixed; boundary="------------010104040109010004090600" Cc: freebsd-net@freebsd.org, d@delphij.net, freebsd-pf@freebsd.org Subject: pf and vimage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 16:17:04 -0000 This is a multi-part message in MIME format. --------------010104040109010004090600 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit there were some people looking at adding vnet support to pf. Since we discussed it last, the rules of the game have significantly changed for the better. With the addition of some new facilitiesin FreeBSD, the work needed to virtualize a module has significantly decreased. The following doc gives the new rules.. --------------010104040109010004090600 Content-Type: text/plain; name="porting_to_vimage.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="porting_to_vimage.txt" August 17 2009 Julian Elischer =================== Vimage: what is it? =================== Vimage is a framework in the BSD kernel which allows a co-operating module to operate on multiple independent instances of its state so that it can participate in a virtual machine / virtual environment scenario. It refers to a part of the Jail infrastructure in FreeBSD. For historical reasons "Virtual network stack enabled jails"(1) are also known as "vimage enabled jails"(2) or "vnet enabled jails"(3). The currently correct term is the latter, which is a contraction of the first. In the future other parts of the system may be virtualized using the same technology and the term to cover all such components would be VIMAGE enhanced modules. The implementation approach taken by the vimage framework is a redefinition of selected global state variables to evaluate to constructs that allow for the virtualized state to be stored and resolved in appropriate instances of 'jail' specific container storage regions. The code operating on virtualized state has to conform to a set of rules described further below. Among other things in order to allow for all the changes to be conditionally compilable. i.e. permitting the virtualized code to fall back to operation on global state. The rest of this document will discuss NETWORK virtualization though the concepts may be true in the future for other parts of the system. The most visible change throughout the existing code is typically replacement of direct references to global variables with macros; foo_bar thus becomes V_foo_bar. V_foo_bar macros will resolve back to the foo_bar global in default kernel builds, and alternatively to the logical equivalent of some_base_pointer->_foo_bar for "options VIMAGE" kernel configs. Prepending of "V_" prefixes to variable references helps in visual discrimination between global and virtualized state. It is also possible to use an alternative syntax, of VNET(foo_bar) to achieve the same thing. The developers felt that V_foo_bar was less visually distracting while still providing enough clues to the reader that the variable is virtualized. In fact the V_foo_bar macro is locally defined near the definition of foo_bar to be an alias for VNET(foo_bar) so the two are not only equivalent, they are the same. The framework also extends the sysctl infrastructure to support access to virtualized state through introduction of the SYSCTL_VNET family of macros; those also automatically fall back to their standard SYSCTL counterparts in default kernel builds. Transparent libkvm(3) lookups are provided to virtualized variables which permits userland binaries such as netstat to operate unmodified on "options VIMAGE" kernels, though this may have some security implications. Vnets are associated with jails. In 8.0, every process is associated with a jail, usually the default (null) jail, and jails currently hang off of a processes ucred. This relationship defines a process's administrative affinity to a vnet and thus indirectly to all of its state. All network interfaces and sockets hold pointers back to their associated vnets. This relationship is obviously entirely independent from proc->ucred->jail bindings. Hence, when a process opens a socket, the socket will get bound to a vnet instance hanging off of proc->ucred->jail->vnet, but once such a socket->vnet binding gets established, it cannot be changed for the entire socket lifetime. The mapping of a from a thread to a vnet should always be done via the TD_TO_VNET macro as the path may change in the future as we get more experience with using the system. Certain classes of network interfaces (Ethernet in particular) can be reassigned from one vnet to another at any time. By definition all vnets are independent and can communicate only if they are explicitly provided with communication paths. Currently mainly netgraph is used to establish inter-vnet datapaths, though other paths are being explored such as the 'epair' back-to-back virtual interface pair, in which the different sides may exist in different jails. In network traffic processing the vnet affinity is defined either by the inbound interface or by the socket / pcb -> vnet binding. However, there are many functions in the network stack that cannot implicitly fetch the vnet context from their standard arguments. Instead of explicitly extending argument lists of such functions with a struct vnet *, the concept of a "current vnet", a per-thread variable was introduced, which can be fetched efficiently via the curvnet macro. The correct network context has to be set on entry to the network stack (socket operations, packet reception, or timer-driven functions) and cleared on exit. This must be done via provided CURVNET_SET() / CURVNET_RESTORE() family of macros, which allow for "stacking" of curvnet context setting and provide additional debugging info in INVARIANTS kernel configs. In most cases however a developer writing virtualized code will not have to set / restore the curvnet context unless the code would include timer-driven events, given that those are inherently vnet-contextless on entry. The current rule is that when not in networking code, the result of the 'curvnet' macro will return NULL and evaluating a V_xxx (or VNET(xxx)) macro will result in an kernel page-fault error. While this is not strictly necessary, it aids in debugging and assurance of program correctness. Note this does NOT mean that TD_TO_VNET(curthread) is invalid. A thread is always associated with a vnet, but just the efficient "curvnet" access method is disabled along with the ability to resolve virtualized symbols. Converting / virtualizing existing code ======================================= There are several steps need in virtualisation. 1/ Decide whether the module needs to be virtualised. If the module is a driver for specific hardware, it makes sense that there be only one instance of the driver as there is only one piece of physical hardware. There are changes in the networking code to allow physical (or virtual) interfaces to be moved between vnets. This generally requires NO changes to the network drivers of the classes covered (e.g. ethernet). Currently if your module is does not have any networking facet, the answer is "no" by default. 2/ If the module is to be virtualised, decide which attributes of the module should be virtualised. For example, It may make sense that there be a single central pool of "struct foo" and a single uma zone for them to come from, with a single lock guarding it. It might also make sense if the "foo_debug" sysctl controls all the instances at once, while on the other hand, the "foo_mode" sysctl might make better sense if it were controllable on a virtual system by virtual system basis. 3/ Work out what global variables and structures are to be virtualised to achieve the behaviour required for part #2. 4/ Work out for all the code paths through the module, how the thread entering the module can divine which virtual environment it is on. Some examples: * Since interfaces are all assigned to one vnet or another, an incoming packet has a pointer to the receive interface, which in turn has a pointer back to the vnet. Often "curvnet" will already have been set by the time your code is called anyhow. * Similarly, on any request from outside the kernel, (direct or indirect) the current thread has a way to get to the current virtual environment instance via TD_TO_VNET(curthread). For existing sockets the vnet context must be used via so->so_vnet since the thread's vnet might change after socket creation. * Timer initiated actions usually have a (void *) argument which points to some private structure for the module. It should be possible to add a pointer to the appropriate module instance into whatever structure that points to. * Sometimes an action (timer trigerred or trigerred by module load or unload simply has to check all the vimage or module instances. There are macro (pairs) for this which will iterate through all the VNET or instances. (see sample code below). This covers most of the cases, however in some cases it may still be required for the module to stash away the virtual environment instance somewhere, and make associated changes in the code. 5/ Decide which parts of the initialization and teardown are per jail and which parts are global, and separate out the code accordingly. Global initialization is done using the SYSINIT facility. Per jail initialization is done using VNET_SYSINIT(). Per jail teardown is doen using VNET_SYSUNINIT(). Global teardown is done using SYSUNIT(). In addition, the modevent handler is called with various event types before any of these are called. The modevent handler may veto load or teardown. On Shutdown, only the modevent handler is called so it may have to simulate the calling of the other handlers if clean shutdown is a requirement of your module. (see sample code below). Don't forget to unregister event handlers, and destroy locks and condition variables. 6/ Add the code described below to the files that make up the module. Details: (VNET implementation details) Firstly the file must be included. Depending on what code you use you may find you also need one or more of: , and . These requirements may change slightly as the ABI settles. Having decided which variables need to be virtualized, the definition of thosvariables needs to be modified to use the VNET_DEFINE() macro. For example: static int foo = 3; struct bar thebar = { 1,2,3 }; would become: static VNET_DEFINE(int, foo) = 3; VNET_DEFINE(struct bar, thebar) = { 1,2,3 }; extern int foo; in an include file might become: VNET_DECLARE(int foo); Normal rules regarding 'static/extern' apply. The initial values that you give in this way will be stored and used as the initial values for EACH NEW INSTANCE of these variables as new jails/vnets are created. As mentioned above, accesses to virtualized symbols are achieved via macros, which generally are of the same name as the original symbol but with a "V_" prepended, thus the head of the interface list, called 'ifnet' is replaced whereever used with "V_ifnet". We do this, by adding the following lines after the definitions above: #define V_foo VNET(foo) #define V_thebar VNET(thebar) --- side-note --- In SCTP, because the code is shared with other OS's they are replaced with a macro MODULE_GLOBAL(modulename, symbol). (this may simplify in light of recent changes). -------------- In addition, should any of your values need to be changed or viewed via sysctl, the following SYSCTL definitions would be needed: SYSCTL_VNET_PROC(_net_inet, OID_AUTO, thebar, CTLTYPE_?? | CTLFLAG_RW | CTLFLAG_SECURE3, &VNET_NAME(thebar), 0, thebar, "?", "the bar is open"); {[XXX] robert fix this is possible ^^^} SYSCTL_VNET_INT(_net_inet, OID_AUTO, foo, CTLFLAG_RW, &VNET_NAME(foo), 0, "size of foo"); In the current version of vimage, when VIMAGE is not compiled into the kernel, the macros evaluate to a direct reference to the one and only symbol/variable, so that there is no speed penalty for those not using vnets. When VIMAGE is compiled in, the macro will evaluate to an access to an offset into a data structure that is accessed on a per-vet basis. The vnet used for this is always curvnet. For this reason an attempt to access such a variable while curvnet is not valid, will result in an exception. To ensure that curvnet has a valid value when needed one needs to add the following code on all entry code paths into the networking code: int my_func(int arg) { CURVNET_SET(TD_TO_VNET(curthread)); do_my_network_stuff(arg); CURVNET_RESTORE(); return (0); } The initial value is usually something like "TD_TO_VNET(curthread) which in turn is a macro that derives the vnet affinity from the current thread. It could also be (m->m_ifp->if_vnet) if we were receiving an mbuf, or so->so_vnet if we had a socket involved. Usually, when a packet enters the system it is carried through the processing path via a single thread, and that thread will set its virtual environment reference to that indicated by the packet on picking up that new packet. This means that in the normal inbound processing path as well as the outgoing process path the current thread can be used to indicate the current virtual environment and curvet will always be valid once most user supplied code is reached. In timer events, it is sometimes necessary to add an "outer loop" to iterate through all the possible vnets if there is just one timer for all instances. When a new loadable module is virtualised the module definitions and intializers need to be examined. The following example illustrates what is needed in the case that you are not loading a new protocol, or domain. (for that see later) ============= sample skeleton code ========== /* init on boot or module load */ static int mymod_init(void) { return (error); } /**************** * Stuff that must be initialized for every instance * (including the first of course). */ static int mymod_vnet_init(const void *unused) { return (0); } /********************** * Called for the removal of the last instance only on module unload. */ static void mymod_uninit(void) { } /*********************** * Called for the removal of each instance. */ static int mymod_vnet_uninit(const void *unused) { return (0) } mymod_modevent(module_t mod, int type, void *unused) { int err = 0; switch (type) { case MOD_LOAD: /* check that loading is ok */ break; case MOD_UNLOAD: /* check that unloading is ok */ break; case MOD_QUIESCE: /* warning: try stop processing */ /* maybe sleep 1 mSec or something to let threads get out */ break; case MOD_SHUTDOWN: /* * this is called once but you may want to shut down * things in each jail, or something global. * In that case it's up to us to simulate the SYSUNINIT() * or the VNET_SYSUNINIT() */ { VNET_ITERATOR_DECL(vnet_iter); VNET_LIST_RLOCK(); VNET_FOREACH(vnet_iter) { CURVNET_SET(vnet_iter); mymod_vnet_uninit(NULL); CURVNET_RESTORE(); } VNET_LIST_RUNLOCK(); } /* you may need to shutdown something global. */ mymod_uninit(); break; default: err = EOPNOTSUPP; break; } return err; } static moduledata_t mymodmod = { "mymod", mymod_modevent, 0 }; /* define execution order using constants from /sys/sys/kernel.h */ #define MYMOD_MAJOR_ORDER SI_SUB_PROTO_BEGIN /* for example */ #define MYMOD_MODULE_ORDER (SI_ORDER_ANY + 64) /* not fussy */ #define MYMOD_SYSINIT_ORDER (MYMOD_MODULE_ORDER + 1) /* a bit later */ #define MYMOD_VNET_ORDER (MYMOD_MODULE_ORDER + 2) /* later still */ DECLARE_MODULE(mymod, mymodmod, MYMOD_MAJOR_ORDER, MYMOD_MODULE_ORDER); MODULE_DEPEND(mymod, ipfw, 2, 2, 2); /* depend on ipfw version (exactly) 2 */ MODULE_VERSION(mymod, 1); SYSINIT(mymod_init, MYMOD_MAJOR_ORDER, MYMOD_SYSINIT_ORDER, mymod_init, NULL); SYSUNINIT(mymod_uninit, MYMOD_MAJOR_ORDER, MYMOD_SYSINIT_ORDER, mymod_uninit, NULL); VNET_SYSINIT(mymod_vnet_init, MYMOD_MAJOR_ORDER, MYMOD_VNET_ORDER, mymod_vnet_init, NULL); VNET_SYSUNINIT(mymod_vnet_uninit, MYMOD_MAJOR_ORDER, MYMOD_VNET_ORDER, mymod_vnet_uninit, NULL); ========== end sample code ======= On BOOT, the order of evaluation will be: In a NON-VIMAGE kernel where the module is compiled: MODEVENT, SYSINIT and VNET_SYSINIT both runm with order defined by their order declarations. {good foot shooting material if you get it wrong!} In a VIMAGE kernel where the module is compiled in: MODEVNET, SYSINIT and VNET_SYSINIT all run with order defined by their order declarations. AND in addition, the VNET_SYSINIT is repeated once for every existing or new jail/vnet. On loading a vnet enabled kernel module after boot: MODEVENT("event = load"); SYSINIT() VNET_SYSINIT() for every existing jail AND in addition, VNET_SYSINIT being called for each new jail created. On unloading of module: MODEVENT("event = MOD_QUIESCE") MODEVENT("event = MOD_UNLOAD") VNET_SYSUNINIT called for every jail/vnet SYSUNINIT On system shutdown: MODEVENT(shutdown) NOTICE that while the order of the SYSINIT and VNET_SYSINIT is reversed from that of SYSUNINIT and VNET_SYSUNINIT, MODEVENTS do not follow this rule and thus it is dangerous to initialise and uninitialise things which are order dependent using MODEVENTs. Or, put another way, Since MODEVENT is called first during module load, it would, by the assumption that everything is reversed, be easy to assume that MODEVENT is called AFTER the SYSINITS during unload. This is in fact not the case. (and I have the scars to prove it). It might be make some sense if the "QUIESCE" was called before the SYSINIT/SYSUNINIT and the UNLOAD called after.. with a millisecond sleep between them, but this is not the case either. Since initial values are copied into the virtualized variables on each new instantiatin, it is quite possible to have modules for which some of the above methods are not needed, and they may be left out. (but not the modevent). Sometimes there is a need to iterate through the vnets. See the modevent shutdown handler (above) for an example of how to do this. Don't forget the locks. In the case where you are loading a new protocol, or domain (protocol family) there are some "shortcuts" that are in place to allow you to maintain a bit more source compatibility with older revisions of FreeBSD. It must be added that the sample code above works just fine for protocols, however protcols also have an aditional initialization vector which is via the prtocol structure, which has a pr_init() entry. When a protocol is registered using pf_proto_register(), the pr_init() for the protocol is called once for every existing vnet. in addition, it will be called for each new vnet. The pr_destroy() method will be called as well on vnet teardown. The pf_proto_register() funcion can be called either from a modevent handler of from the SYSINIT() if you have one, and the pf_proto_unregister() called from the SYSUNINIT or the unload modevent handler. If you are adding a whole new protocol domain, (protocol family) then you should add the VNET_DOMAIN_SET(domainname) (e,g, inet, inet6) macro. These use VNET_SYSINIT internally to indirectly call the dom_init() and pr_init() functions for each vnet, (and the equivalent for teardown.) In this case one needs to be absolutely sure that both your domain and protocol initializers can be called multiple times, once for each vnet. One can still add SYSINITs for once only initialization, or use the modevent handler. I prefer to do as much explicitly in the SYSINITS and VNET_SYSINITS as then you have no surprises. finally: The command to make a new jail with a new vnet: jail -c host.hostname=test path=/ vnet command=/bin/tcsh jail -c host.hostname=test path=/ children.max=4 vnet command=/bin/tcsh (children.max allows hierarchical jail creation). Note that the command must come last. --------------010104040109010004090600-- From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 16:53:59 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89DF7106564A for ; Thu, 20 Aug 2009 16:53:59 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 4D3808FC62 for ; Thu, 20 Aug 2009 16:53:58 +0000 (UTC) Received: by vws10 with SMTP id 10so40149vws.7 for ; Thu, 20 Aug 2009 09:53:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.16.146 with SMTP id o18mr3171vca.70.1250787238201; Thu, 20 Aug 2009 09:53:58 -0700 (PDT) In-Reply-To: <435336.24858.qm@web63908.mail.re1.yahoo.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> Date: Thu, 20 Aug 2009 10:53:58 -0600 Message-ID: <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> From: Manish Vachharajani To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 16:53:59 -0000 Oh whoops, sorry didn't see that. So the plot thickens. Why don't these errors show up in the netstat output I forwarded originally? Ierrs was 0 but the dmesg output clearly shows missed packets. Any thoughts on what is going on? Looking at the code, missed_rx should certainly get counted in the ierrors field as you said. Is the Ierrs in the netstat output some other counter? If so, how do I get the if_ierrors variable from the command line? Manish On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba w= rote: > > > --- On Wed, 8/19/09, Manish Vachharajani wr= ote: > >> From: Manish Vachharajani >> Subject: Re: Dropped vs. missed packets in the ixgbe driver >> To: "Barney Cordoba" >> Cc: freebsd-net@freebsd.org >> Date: Wednesday, August 19, 2009, 2:46 PM >> Agreed, the errors are reported but >> missed packets are not.=A0 The >> question is, is the correct fix to just add stats.mpc[0] to >> if_ierrors >> in that line or to add it to if_iqdrops.=A0 The fix is >> easy once we >> agree on what the correct behavior is. >> >> Manish >> >> > Barney wrote: >> > >> > if you look in ixgbe_update_stats_counters at the >> bottom: >> > >> > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + >> adapter->stats.crcerrs + >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >> > >> > the errors are added in. >> > >> > BC > > Huh? missed_rx are the missed packets. So they are counted. > > BC > > > > > --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:08:10 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FD4F106568F for ; Thu, 20 Aug 2009 17:08:10 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-yx0-f181.google.com (mail-yx0-f181.google.com [209.85.210.181]) by mx1.freebsd.org (Postfix) with ESMTP id B78D28FC55 for ; Thu, 20 Aug 2009 17:08:09 +0000 (UTC) Received: by yxe11 with SMTP id 11so52319yxe.3 for ; Thu, 20 Aug 2009 10:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=xnstFKBzqVtXbvrpEOYAFAAvJIhTx/B6RL8NIYo2J1w=; b=G+MeHOepfLmJHrzgt+zpzbT5JvsY2Pl1ac1D16amTJCIWDaza8O9QD7oXYy7XoQSAz +cL0U5JJnkj8seh/ul5eldHPmdQu7UPXl4SfyprVN4so+n56OS+9tmyDZi2GzR2wTA+u iIHNBmHDVSDjRwOrygwKngXNKOgvBaaXFFihs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=vGuBC/erAxlrvMEFxUi8Y6iYvUo5HFp5CMSwz4pkKrgLDkxkUx6erA6jwZeFyounox k8vG4ydBDLXDOKFo3K8eE7oTqgCcINsSG3V9Gy+11ennd9W3PEum4EBG1vw6dusXSdbv mhKjeplOEsvugCNXTWEbeCxyenYqJahbLV634= MIME-Version: 1.0 Received: by 10.100.76.8 with SMTP id y8mr61863ana.34.1250788089117; Thu, 20 Aug 2009 10:08:09 -0700 (PDT) In-Reply-To: <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> Date: Thu, 20 Aug 2009 10:08:09 -0700 Message-ID: <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> From: Jack Vogel To: Manish Vachharajani Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:08:10 -0000 I've been looking at the code due to another problem of bogus flow control numbers, and there are some changes needed, things that should be 82599 vs 82598 and were not seperated properly. Will be forthcoming. Not sure if it has any relevance to this, but its possible. Jack On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani < manishv@lineratesystems.com> wrote: > Oh whoops, sorry didn't see that. So the plot thickens. Why don't > these errors show up in the netstat output I forwarded originally? > Ierrs was 0 but the dmesg output clearly shows missed packets. Any > thoughts on what is going on? Looking at the code, missed_rx should > certainly get counted in the ierrors field as you said. > > Is the Ierrs in the netstat output some other counter? If so, how do > I get the if_ierrors variable from the command line? > > Manish > > On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba > wrote: > > > > > > --- On Wed, 8/19/09, Manish Vachharajani > wrote: > > > >> From: Manish Vachharajani > >> Subject: Re: Dropped vs. missed packets in the ixgbe driver > >> To: "Barney Cordoba" > >> Cc: freebsd-net@freebsd.org > >> Date: Wednesday, August 19, 2009, 2:46 PM > >> Agreed, the errors are reported but > >> missed packets are not. The > >> question is, is the correct fix to just add stats.mpc[0] to > >> if_ierrors > >> in that line or to add it to if_iqdrops. The fix is > >> easy once we > >> agree on what the correct behavior is. > >> > >> Manish > >> > >> > Barney wrote: > >> > > >> > if you look in ixgbe_update_stats_counters at the > >> bottom: > >> > > >> > ifp->if_ierrors = missed_rx + > >> adapter->stats.crcerrs + > >> > adapter->stats.rlec; > >> > > >> > the errors are added in. > >> > > >> > BC > > > > Huh? missed_rx are the missed packets. So they are counted. > > > > BC > > > > > > > > > > > > > > -- > Manish Vachharajani > Founder > LineRate Systems > manishv@lineratesystems.com > (609)635-9531 M > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:23:16 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 250D6106568F for ; Thu, 20 Aug 2009 17:23:16 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id D4D8E8FC3D for ; Thu, 20 Aug 2009 17:23:15 +0000 (UTC) Received: by vws10 with SMTP id 10so63804vws.7 for ; Thu, 20 Aug 2009 10:23:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.12.148 with SMTP id x20mr43430vcx.86.1250788994198; Thu, 20 Aug 2009 10:23:14 -0700 (PDT) In-Reply-To: <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> Date: Thu, 20 Aug 2009 11:23:14 -0600 Message-ID: <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> From: Manish Vachharajani To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:23:16 -0000 I noticed the bogus XON, XOFF numbers. I'm glad to see it will be fixed so I can cross it off my todo list. :) I don't think the issue is related though, but you never know. Barney pointed out that missed_rx in the ixgbe_update_stats_counters function accumulates the missed packet registers into the missed_rx field. At the end of the function, these are summed into ifp->if_ierrors which should be reported under Ierrs by netstat -idh. However that count is zero as reported via netstat. The stats printfs activated via sysctl dev.ix.#.stats=3D1 prints stats.mpc[0]. This number is large (around 400k on the machine I've been using to find the problem). If you look at the code, missed_rx and mpc[i] are updated with the same variables. Given that missed_rx is unsigned, I don't understand how the number fails to make it into ifp->if_ierrors, but I have yet to dive into debugging this seriously. Initially, I thought the fix was simple since I didn't see the missed_rx getting added to ierrors. But now, I am not sure where the problem is. Hopefully, it will be more obvious to you than it is to me. I'm sure it is something simple that I am missing. Manish On Thu, Aug 20, 2009 at 11:08 AM, Jack Vogel wrote: > I've been looking at the code due to another problem of bogus flow contro= l > numbers, and there are some changes needed, things that should be 82599 v= s > 82598 and were not seperated properly. Will be forthcoming. Not sure if i= t > has any relevance to this, but its possible. > > Jack > > > On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani > wrote: >> >> Oh whoops, sorry didn't see that. =A0So the plot thickens. =A0Why don't >> these errors show up in the netstat output I forwarded originally? >> Ierrs was 0 but the dmesg output clearly shows missed packets. =A0Any >> thoughts on what is going on? =A0Looking at the code, missed_rx should >> certainly get counted in the ierrors field as you said. >> >> Is the Ierrs in the netstat output some other counter? =A0If so, how do >> I get the if_ierrors variable from the command line? >> >> Manish >> >> On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba >> wrote: >> > >> > >> > --- On Wed, 8/19/09, Manish Vachharajani >> > wrote: >> > >> >> From: Manish Vachharajani >> >> Subject: Re: Dropped vs. missed packets in the ixgbe driver >> >> To: "Barney Cordoba" >> >> Cc: freebsd-net@freebsd.org >> >> Date: Wednesday, August 19, 2009, 2:46 PM >> >> Agreed, the errors are reported but >> >> missed packets are not.=A0 The >> >> question is, is the correct fix to just add stats.mpc[0] to >> >> if_ierrors >> >> in that line or to add it to if_iqdrops.=A0 The fix is >> >> easy once we >> >> agree on what the correct behavior is. >> >> >> >> Manish >> >> >> >> > Barney wrote: >> >> > >> >> > if you look in ixgbe_update_stats_counters at the >> >> bottom: >> >> > >> >> > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + >> >> adapter->stats.crcerrs + >> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >> >> > >> >> > the errors are added in. >> >> > >> >> > BC >> > >> > Huh? missed_rx are the missed packets. So they are counted. >> > >> > BC >> > >> > >> > >> > >> > >> >> >> >> -- >> Manish Vachharajani >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > --=20 Manish Vachharajani From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:32:30 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F7C9106568B for ; Thu, 20 Aug 2009 17:32:30 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 3C4838FC3D for ; Thu, 20 Aug 2009 17:32:29 +0000 (UTC) Received: by vws10 with SMTP id 10so71115vws.7 for ; Thu, 20 Aug 2009 10:32:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.106.4 with SMTP id v4mr101652vco.44.1250789549349; Thu, 20 Aug 2009 10:32:29 -0700 (PDT) In-Reply-To: <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> Date: Thu, 20 Aug 2009 11:32:29 -0600 Message-ID: <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> From: Manish Vachharajani To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:32:30 -0000 My co-founder, John, just pointed out the problem. The MPC register on ixgbe is clear on read. stats.mpc[i] correctly accumulates the misses, but missed_rx gets set to 0 on any interval where there are no misses and subsequently, if_errors gets set to 0 (assuming no crcerrs or rlecs.) I believe the correct fix is in the patch below: diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c index f1fa728..9cbaec6 100644 --- a/fbsd/ixgbe-1.7.4/ixgbe.c +++ b/fbsd/ixgbe-1.7.4/ixgbe.c @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) { struct ifnet *ifp =3D adapter->ifp;; struct ixgbe_hw *hw =3D &adapter->hw; - u32 missed_rx =3D 0, bprc, lxon, lxoff, total; + u32 missed_rx =3D 0, missed_rx_cum =3D 0, bprc, lxon, lxoff, total= ; adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CRCERRS); @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) mp =3D IXGBE_READ_REG(hw, IXGBE_MPC(i)); missed_rx +=3D mp; adapter->stats.mpc[i] +=3D mp; + missed_rx_cum +=3D adapter->stats.mpc[i]; adapter->stats.rnbc[i] +=3D IXGBE_READ_REG(hw, IXGBE_RNBC(i= )); } (Note that it may not apply since I've pulled the driver out into a separate directory structure) We need the missed_rx_cum that is the total number of rx misses seen because missed_rx is decremented from gprc to work around a hardware issue and so needs to be the misses seen on this call to the function. Manish On Thu, Aug 20, 2009 at 11:23 AM, Manish Vachharajani wrote: > I noticed the bogus XON, XOFF numbers. =A0I'm glad to see it will be > fixed so I can cross it off my todo list. =A0:) =A0I don't think the issu= e > is related though, but you never know. > > Barney pointed out that missed_rx in the ixgbe_update_stats_counters > function accumulates the missed packet registers into the missed_rx > field. =A0At the end of the function, these are summed into > ifp->if_ierrors which should be reported under Ierrs by netstat -idh. > However that count is zero as reported via netstat. =A0The stats printfs > activated via sysctl dev.ix.#.stats=3D1 prints stats.mpc[0]. =A0This > number is large (around 400k on the machine I've been using to find > the problem). =A0If you look at the code, missed_rx and mpc[i] are > updated with the same variables. =A0Given that missed_rx is unsigned, I > don't understand how the number fails to make it into ifp->if_ierrors, > but I have yet to dive into debugging this seriously. > > Initially, I thought the fix was simple since I didn't see the > missed_rx getting added to ierrors. =A0But now, I am not sure where the > problem is. =A0Hopefully, it will be more obvious to you than it is to > me. =A0I'm sure it is something simple that I am missing. > > Manish > > On Thu, Aug 20, 2009 at 11:08 AM, Jack Vogel wrote: >> I've been looking at the code due to another problem of bogus flow contr= ol >> numbers, and there are some changes needed, things that should be 82599 = vs >> 82598 and were not seperated properly. Will be forthcoming. Not sure if = it >> has any relevance to this, but its possible. >> >> Jack >> >> >> On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani >> wrote: >>> >>> Oh whoops, sorry didn't see that. =A0So the plot thickens. =A0Why don't >>> these errors show up in the netstat output I forwarded originally? >>> Ierrs was 0 but the dmesg output clearly shows missed packets. =A0Any >>> thoughts on what is going on? =A0Looking at the code, missed_rx should >>> certainly get counted in the ierrors field as you said. >>> >>> Is the Ierrs in the netstat output some other counter? =A0If so, how do >>> I get the if_ierrors variable from the command line? >>> >>> Manish >>> >>> On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba >>> wrote: >>> > >>> > >>> > --- On Wed, 8/19/09, Manish Vachharajani >>> > wrote: >>> > >>> >> From: Manish Vachharajani >>> >> Subject: Re: Dropped vs. missed packets in the ixgbe driver >>> >> To: "Barney Cordoba" >>> >> Cc: freebsd-net@freebsd.org >>> >> Date: Wednesday, August 19, 2009, 2:46 PM >>> >> Agreed, the errors are reported but >>> >> missed packets are not.=A0 The >>> >> question is, is the correct fix to just add stats.mpc[0] to >>> >> if_ierrors >>> >> in that line or to add it to if_iqdrops.=A0 The fix is >>> >> easy once we >>> >> agree on what the correct behavior is. >>> >> >>> >> Manish >>> >> >>> >> > Barney wrote: >>> >> > >>> >> > if you look in ixgbe_update_stats_counters at the >>> >> bottom: >>> >> > >>> >> > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + >>> >> adapter->stats.crcerrs + >>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >>> >> > >>> >> > the errors are added in. >>> >> > >>> >> > BC >>> > >>> > Huh? missed_rx are the missed packets. So they are counted. >>> > >>> > BC >>> > >>> > >>> > >>> > >>> > >>> >>> >>> >>> -- >>> Manish Vachharajani > >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> > > -- > Manish Vachharajani > From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:34:13 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CD391065691 for ; Thu, 20 Aug 2009 17:34:13 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 297818FC7E for ; Thu, 20 Aug 2009 17:34:13 +0000 (UTC) Received: by vws10 with SMTP id 10so72409vws.7 for ; Thu, 20 Aug 2009 10:34:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.12.10 with SMTP id v10mr51985vcv.97.1250789652485; Thu, 20 Aug 2009 10:34:12 -0700 (PDT) In-Reply-To: <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> Date: Thu, 20 Aug 2009 11:34:12 -0600 Message-ID: <5bc218350908201034u553df7feiaead037432279360@mail.gmail.com> From: Manish Vachharajani To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:34:13 -0000 Whoops, the correct fix is below. Forgot to use missed_rx_cum when summing= : diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c index f1fa728..262d64d 100644 --- a/fbsd/ixgbe-1.7.4/ixgbe.c +++ b/fbsd/ixgbe-1.7.4/ixgbe.c @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) { struct ifnet *ifp =3D adapter->ifp;; struct ixgbe_hw *hw =3D &adapter->hw; - u32 missed_rx =3D 0, bprc, lxon, lxoff, total; + u32 missed_rx =3D 0, missed_rx_cum =3D 0, bprc, lxon, lxoff, total= ; adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CRCERRS); @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) mp =3D IXGBE_READ_REG(hw, IXGBE_MPC(i)); missed_rx +=3D mp; adapter->stats.mpc[i] +=3D mp; + missed_rx_cum +=3D adapter->stats.mpc[i]; adapter->stats.rnbc[i] +=3D IXGBE_READ_REG(hw, IXGBE_RNBC(i= )); } @@ -4370,7 +4371,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) ifp->if_collisions =3D 0; /* Rx Errors */ - ifp->if_ierrors =3D missed_rx + adapter->stats.crcerrs + + ifp->if_ierrors =3D missed_rx_cum + adapter->stats.crcerrs + adapter->stats.rlec; } On Thu, Aug 20, 2009 at 11:32 AM, Manish Vachharajani wrote: > My co-founder, John, just pointed out the problem. > > The MPC register on ixgbe is clear on read. =A0stats.mpc[i] correctly > accumulates the misses, but missed_rx gets set to 0 on any interval > where there are no misses and subsequently, if_errors gets set to 0 > (assuming no crcerrs or rlecs.) =A0I believe the correct fix is in the > patch below: > > diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c > index f1fa728..9cbaec6 100644 > --- a/fbsd/ixgbe-1.7.4/ixgbe.c > +++ b/fbsd/ixgbe-1.7.4/ixgbe.c > @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter *adapter= ) > =A0{ > =A0 =A0 =A0 =A0struct ifnet =A0 *ifp =3D adapter->ifp;; > =A0 =A0 =A0 =A0struct ixgbe_hw *hw =3D &adapter->hw; > - =A0 =A0 =A0 u32 =A0missed_rx =3D 0, bprc, lxon, lxoff, total; > + =A0 =A0 =A0 u32 =A0missed_rx =3D 0, missed_rx_cum =3D 0, bprc, lxon, lx= off, total; > > =A0 =A0 =A0 =A0adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CRCER= RS); > > @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter *adapter= ) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mp =3D IXGBE_READ_REG(hw, IXGBE_MPC(i)); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0missed_rx +=3D mp; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.mpc[i] +=3D mp; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 missed_rx_cum +=3D adapter->stats.mpc[i]; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rnbc[i] +=3D IXGBE_READ_REG= (hw, IXGBE_RNBC(i)); > =A0 =A0 =A0 =A0} > > (Note that it may not apply since I've pulled the driver out into a > separate directory structure) > > We need the missed_rx_cum that is the total number of rx misses seen > because missed_rx is decremented from gprc to work around a hardware > issue and so needs to be the misses seen on this call to the function. > > Manish > > > On Thu, Aug 20, 2009 at 11:23 AM, Manish > Vachharajani wrote: >> I noticed the bogus XON, XOFF numbers. =A0I'm glad to see it will be >> fixed so I can cross it off my todo list. =A0:) =A0I don't think the iss= ue >> is related though, but you never know. >> >> Barney pointed out that missed_rx in the ixgbe_update_stats_counters >> function accumulates the missed packet registers into the missed_rx >> field. =A0At the end of the function, these are summed into >> ifp->if_ierrors which should be reported under Ierrs by netstat -idh. >> However that count is zero as reported via netstat. =A0The stats printfs >> activated via sysctl dev.ix.#.stats=3D1 prints stats.mpc[0]. =A0This >> number is large (around 400k on the machine I've been using to find >> the problem). =A0If you look at the code, missed_rx and mpc[i] are >> updated with the same variables. =A0Given that missed_rx is unsigned, I >> don't understand how the number fails to make it into ifp->if_ierrors, >> but I have yet to dive into debugging this seriously. >> >> Initially, I thought the fix was simple since I didn't see the >> missed_rx getting added to ierrors. =A0But now, I am not sure where the >> problem is. =A0Hopefully, it will be more obvious to you than it is to >> me. =A0I'm sure it is something simple that I am missing. >> >> Manish >> >> On Thu, Aug 20, 2009 at 11:08 AM, Jack Vogel wrote: >>> I've been looking at the code due to another problem of bogus flow cont= rol >>> numbers, and there are some changes needed, things that should be 82599= vs >>> 82598 and were not seperated properly. Will be forthcoming. Not sure if= it >>> has any relevance to this, but its possible. >>> >>> Jack >>> >>> >>> On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani >>> wrote: >>>> >>>> Oh whoops, sorry didn't see that. =A0So the plot thickens. =A0Why don'= t >>>> these errors show up in the netstat output I forwarded originally? >>>> Ierrs was 0 but the dmesg output clearly shows missed packets. =A0Any >>>> thoughts on what is going on? =A0Looking at the code, missed_rx should >>>> certainly get counted in the ierrors field as you said. >>>> >>>> Is the Ierrs in the netstat output some other counter? =A0If so, how d= o >>>> I get the if_ierrors variable from the command line? >>>> >>>> Manish >>>> >>>> On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba >>>> wrote: >>>> > >>>> > >>>> > --- On Wed, 8/19/09, Manish Vachharajani >>>> > wrote: >>>> > >>>> >> From: Manish Vachharajani >>>> >> Subject: Re: Dropped vs. missed packets in the ixgbe driver >>>> >> To: "Barney Cordoba" >>>> >> Cc: freebsd-net@freebsd.org >>>> >> Date: Wednesday, August 19, 2009, 2:46 PM >>>> >> Agreed, the errors are reported but >>>> >> missed packets are not.=A0 The >>>> >> question is, is the correct fix to just add stats.mpc[0] to >>>> >> if_ierrors >>>> >> in that line or to add it to if_iqdrops.=A0 The fix is >>>> >> easy once we >>>> >> agree on what the correct behavior is. >>>> >> >>>> >> Manish >>>> >> >>>> >> > Barney wrote: >>>> >> > >>>> >> > if you look in ixgbe_update_stats_counters at the >>>> >> bottom: >>>> >> > >>>> >> > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + >>>> >> adapter->stats.crcerrs + >>>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >>>> >> > >>>> >> > the errors are added in. >>>> >> > >>>> >> > BC >>>> > >>>> > Huh? missed_rx are the missed packets. So they are counted. >>>> > >>>> > BC >>>> > >>>> > >>>> > >>>> > >>>> > >>>> >>>> >>>> >>>> -- >>>> Manish Vachharajani >> >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >> >> -- >> Manish Vachharajani >> > --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:37:24 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FA42106568D for ; Thu, 20 Aug 2009 17:37:24 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.243]) by mx1.freebsd.org (Postfix) with ESMTP id D47118FC51 for ; Thu, 20 Aug 2009 17:37:23 +0000 (UTC) Received: by an-out-0708.google.com with SMTP id d14so20745and.13 for ; Thu, 20 Aug 2009 10:37:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=66N4EH8hE/DXJBGwX5A0lOC5GoDMYQ0CNFHLCxBIemk=; b=nquJ8vC1N+NjLCw8tXQwj8OWsM2zBxMvNH4xkWCkbGuqVLSoySygFcB6PYuhzwbzIN n34//usneoEolK14Ny7PvJo9PhlMB0zgSx2Fv9EN6Ujj7DlcdM/nLHDAEs1zkDScMYo+ DdtHlE6VWcD4mvzDbHR5Vyf4UDkoOxrW8Ue9A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=QbLqDlfJiUfaIWjBuDBL+fseMi7CFrRiRmxiqe2cGADyKglaioraK2KDhnV0hrBcCu uudLX3jNMVxi3GKQ85e7xwsYXodBHjlULmiu8ckN1Wjm9Tnas2q2W8kfnu33r9wTDoAm DKaMObABVlFHWd4edrhHtIbdK7I9xlNMfPhF0= MIME-Version: 1.0 Received: by 10.101.128.27 with SMTP id f27mr36758ann.182.1250789843172; Thu, 20 Aug 2009 10:37:23 -0700 (PDT) In-Reply-To: <5bc218350908201034u553df7feiaead037432279360@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> <5bc218350908201034u553df7feiaead037432279360@mail.gmail.com> Date: Thu, 20 Aug 2009 10:37:23 -0700 Message-ID: <2a41acea0908201037n10505b04le924f29efd5398a7@mail.gmail.com> From: Jack Vogel To: Manish Vachharajani Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:37:24 -0000 Thanks Manish, I will keep this diff around and work it into my final changes in the code, you confirmed this solved the problem you were seeing I assume? Cheers, Jack On Thu, Aug 20, 2009 at 10:34 AM, Manish Vachharajani < manishv@lineratesystems.com> wrote: > Whoops, the correct fix is below. Forgot to use missed_rx_cum when > summing: > > diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c > index f1fa728..262d64d 100644 > --- a/fbsd/ixgbe-1.7.4/ixgbe.c > +++ b/fbsd/ixgbe-1.7.4/ixgbe.c > @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) > { > struct ifnet *ifp = adapter->ifp;; > struct ixgbe_hw *hw = &adapter->hw; > - u32 missed_rx = 0, bprc, lxon, lxoff, total; > + u32 missed_rx = 0, missed_rx_cum = 0, bprc, lxon, lxoff, total; > > adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); > > @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) > mp = IXGBE_READ_REG(hw, IXGBE_MPC(i)); > missed_rx += mp; > adapter->stats.mpc[i] += mp; > + missed_rx_cum += adapter->stats.mpc[i]; > adapter->stats.rnbc[i] += IXGBE_READ_REG(hw, IXGBE_RNBC(i)); > } > > @@ -4370,7 +4371,7 @@ ixgbe_update_stats_counters(struct adapter *adapter) > ifp->if_collisions = 0; > > /* Rx Errors */ > - ifp->if_ierrors = missed_rx + adapter->stats.crcerrs + > + ifp->if_ierrors = missed_rx_cum + adapter->stats.crcerrs + > adapter->stats.rlec; > } > > > > On Thu, Aug 20, 2009 at 11:32 AM, Manish > Vachharajani wrote: > > My co-founder, John, just pointed out the problem. > > > > The MPC register on ixgbe is clear on read. stats.mpc[i] correctly > > accumulates the misses, but missed_rx gets set to 0 on any interval > > where there are no misses and subsequently, if_errors gets set to 0 > > (assuming no crcerrs or rlecs.) I believe the correct fix is in the > > patch below: > > > > diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c > > index f1fa728..9cbaec6 100644 > > --- a/fbsd/ixgbe-1.7.4/ixgbe.c > > +++ b/fbsd/ixgbe-1.7.4/ixgbe.c > > @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter > *adapter) > > { > > struct ifnet *ifp = adapter->ifp;; > > struct ixgbe_hw *hw = &adapter->hw; > > - u32 missed_rx = 0, bprc, lxon, lxoff, total; > > + u32 missed_rx = 0, missed_rx_cum = 0, bprc, lxon, lxoff, total; > > > > adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); > > > > @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter > *adapter) > > mp = IXGBE_READ_REG(hw, IXGBE_MPC(i)); > > missed_rx += mp; > > adapter->stats.mpc[i] += mp; > > + missed_rx_cum += adapter->stats.mpc[i]; > > adapter->stats.rnbc[i] += IXGBE_READ_REG(hw, > IXGBE_RNBC(i)); > > } > > > > (Note that it may not apply since I've pulled the driver out into a > > separate directory structure) > > > > We need the missed_rx_cum that is the total number of rx misses seen > > because missed_rx is decremented from gprc to work around a hardware > > issue and so needs to be the misses seen on this call to the function. > > > > Manish > > > > > > On Thu, Aug 20, 2009 at 11:23 AM, Manish > > Vachharajani wrote: > >> I noticed the bogus XON, XOFF numbers. I'm glad to see it will be > >> fixed so I can cross it off my todo list. :) I don't think the issue > >> is related though, but you never know. > >> > >> Barney pointed out that missed_rx in the ixgbe_update_stats_counters > >> function accumulates the missed packet registers into the missed_rx > >> field. At the end of the function, these are summed into > >> ifp->if_ierrors which should be reported under Ierrs by netstat -idh. > >> However that count is zero as reported via netstat. The stats printfs > >> activated via sysctl dev.ix.#.stats=1 prints stats.mpc[0]. This > >> number is large (around 400k on the machine I've been using to find > >> the problem). If you look at the code, missed_rx and mpc[i] are > >> updated with the same variables. Given that missed_rx is unsigned, I > >> don't understand how the number fails to make it into ifp->if_ierrors, > >> but I have yet to dive into debugging this seriously. > >> > >> Initially, I thought the fix was simple since I didn't see the > >> missed_rx getting added to ierrors. But now, I am not sure where the > >> problem is. Hopefully, it will be more obvious to you than it is to > >> me. I'm sure it is something simple that I am missing. > >> > >> Manish > >> > >> On Thu, Aug 20, 2009 at 11:08 AM, Jack Vogel wrote: > >>> I've been looking at the code due to another problem of bogus flow > control > >>> numbers, and there are some changes needed, things that should be 82599 > vs > >>> 82598 and were not seperated properly. Will be forthcoming. Not sure if > it > >>> has any relevance to this, but its possible. > >>> > >>> Jack > >>> > >>> > >>> On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani > >>> wrote: > >>>> > >>>> Oh whoops, sorry didn't see that. So the plot thickens. Why don't > >>>> these errors show up in the netstat output I forwarded originally? > >>>> Ierrs was 0 but the dmesg output clearly shows missed packets. Any > >>>> thoughts on what is going on? Looking at the code, missed_rx should > >>>> certainly get counted in the ierrors field as you said. > >>>> > >>>> Is the Ierrs in the netstat output some other counter? If so, how do > >>>> I get the if_ierrors variable from the command line? > >>>> > >>>> Manish > >>>> > >>>> On Thu, Aug 20, 2009 at 6:49 AM, Barney Cordoba< > barney_cordoba@yahoo.com> > >>>> wrote: > >>>> > > >>>> > > >>>> > --- On Wed, 8/19/09, Manish Vachharajani < > manishv@lineratesystems.com> > >>>> > wrote: > >>>> > > >>>> >> From: Manish Vachharajani > >>>> >> Subject: Re: Dropped vs. missed packets in the ixgbe driver > >>>> >> To: "Barney Cordoba" > >>>> >> Cc: freebsd-net@freebsd.org > >>>> >> Date: Wednesday, August 19, 2009, 2:46 PM > >>>> >> Agreed, the errors are reported but > >>>> >> missed packets are not. The > >>>> >> question is, is the correct fix to just add stats.mpc[0] to > >>>> >> if_ierrors > >>>> >> in that line or to add it to if_iqdrops. The fix is > >>>> >> easy once we > >>>> >> agree on what the correct behavior is. > >>>> >> > >>>> >> Manish > >>>> >> > >>>> >> > Barney wrote: > >>>> >> > > >>>> >> > if you look in ixgbe_update_stats_counters at the > >>>> >> bottom: > >>>> >> > > >>>> >> > ifp->if_ierrors = missed_rx + > >>>> >> adapter->stats.crcerrs + > >>>> >> > adapter->stats.rlec; > >>>> >> > > >>>> >> > the errors are added in. > >>>> >> > > >>>> >> > BC > >>>> > > >>>> > Huh? missed_rx are the missed packets. So they are counted. > >>>> > > >>>> > BC > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > >>>> > >>>> > >>>> -- > >>>> Manish Vachharajani > >> > >>>> _______________________________________________ > >>>> freebsd-net@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > >>> > >>> > >> > >> -- > >> Manish Vachharajani > >> > > > > > > -- > Manish Vachharajani > Founder > LineRate Systems > manishv@lineratesystems.com > (609)635-9531 M > From owner-freebsd-net@FreeBSD.ORG Thu Aug 20 17:39:01 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65EDE106568D for ; Thu, 20 Aug 2009 17:39:01 +0000 (UTC) (envelope-from manishv@lineratesystems.com) Received: from mail-vw0-f180.google.com (mail-vw0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id 227518FC64 for ; Thu, 20 Aug 2009 17:39:00 +0000 (UTC) Received: by vws10 with SMTP id 10so76053vws.7 for ; Thu, 20 Aug 2009 10:39:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.13.224 with SMTP id d32mr130828vca.26.1250789940395; Thu, 20 Aug 2009 10:39:00 -0700 (PDT) In-Reply-To: <2a41acea0908201037n10505b04le924f29efd5398a7@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> <5bc218350908201034u553df7feiaead037432279360@mail.gmail.com> <2a41acea0908201037n10505b04le924f29efd5398a7@mail.gmail.com> Date: Thu, 20 Aug 2009 11:39:00 -0600 Message-ID: <5bc218350908201039q574f92e3mabe73d01c35f662c@mail.gmail.com> From: Manish Vachharajani To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2009 17:39:01 -0000 Yes, in our latest tests, netstat correctly outputs the misses as Ierrs. Manish On Thu, Aug 20, 2009 at 11:37 AM, Jack Vogel wrote: > Thanks Manish, I will keep this diff around and work it into my final > changes in the > code, you confirmed this solved the problem you were seeing I assume? > > Cheers, > > Jack > > > On Thu, Aug 20, 2009 at 10:34 AM, Manish Vachharajani > wrote: >> >> Whoops, the correct fix is below. =A0Forgot to use missed_rx_cum when >> summing: >> >> diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c >> index f1fa728..262d64d 100644 >> --- a/fbsd/ixgbe-1.7.4/ixgbe.c >> +++ b/fbsd/ixgbe-1.7.4/ixgbe.c >> @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter *adapte= r) >> =A0{ >> =A0 =A0 =A0 =A0struct ifnet =A0 *ifp =3D adapter->ifp;; >> =A0 =A0 =A0 =A0struct ixgbe_hw *hw =3D &adapter->hw; >> - =A0 =A0 =A0 u32 =A0missed_rx =3D 0, bprc, lxon, lxoff, total; >> + =A0 =A0 =A0 u32 =A0missed_rx =3D 0, missed_rx_cum =3D 0, bprc, lxon, l= xoff, total; >> >> =A0 =A0 =A0 =A0adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CRCE= RRS); >> >> @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter *adapte= r) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mp =3D IXGBE_READ_REG(hw, IXGBE_MPC(i)); >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0missed_rx +=3D mp; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.mpc[i] +=3D mp; >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 missed_rx_cum +=3D adapter->stats.mpc[i]; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rnbc[i] +=3D IXGBE_READ_RE= G(hw, >> IXGBE_RNBC(i)); >> =A0 =A0 =A0 =A0} >> >> @@ -4370,7 +4371,7 @@ ixgbe_update_stats_counters(struct adapter *adapte= r) >> =A0 =A0 =A0 =A0ifp->if_collisions =3D 0; >> >> =A0 =A0 =A0 =A0/* Rx Errors */ >> - =A0 =A0 =A0 ifp->if_ierrors =3D missed_rx + adapter->stats.crcerrs + >> + =A0 =A0 =A0 ifp->if_ierrors =3D missed_rx_cum + adapter->stats.crcerrs= + >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >> =A0} >> >> >> >> On Thu, Aug 20, 2009 at 11:32 AM, Manish >> Vachharajani wrote: >> > My co-founder, John, just pointed out the problem. >> > >> > The MPC register on ixgbe is clear on read. =A0stats.mpc[i] correctly >> > accumulates the misses, but missed_rx gets set to 0 on any interval >> > where there are no misses and subsequently, if_errors gets set to 0 >> > (assuming no crcerrs or rlecs.) =A0I believe the correct fix is in the >> > patch below: >> > >> > diff --git a/fbsd/ixgbe-1.7.4/ixgbe.c b/fbsd/ixgbe-1.7.4/ixgbe.c >> > index f1fa728..9cbaec6 100644 >> > --- a/fbsd/ixgbe-1.7.4/ixgbe.c >> > +++ b/fbsd/ixgbe-1.7.4/ixgbe.c >> > @@ -4294,7 +4294,7 @@ ixgbe_update_stats_counters(struct adapter >> > *adapter) >> > =A0{ >> > =A0 =A0 =A0 =A0struct ifnet =A0 *ifp =3D adapter->ifp;; >> > =A0 =A0 =A0 =A0struct ixgbe_hw *hw =3D &adapter->hw; >> > - =A0 =A0 =A0 u32 =A0missed_rx =3D 0, bprc, lxon, lxoff, total; >> > + =A0 =A0 =A0 u32 =A0missed_rx =3D 0, missed_rx_cum =3D 0, bprc, lxon,= lxoff, total; >> > >> > =A0 =A0 =A0 =A0adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CR= CERRS); >> > >> > @@ -4303,6 +4303,7 @@ ixgbe_update_stats_counters(struct adapter >> > *adapter) >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mp =3D IXGBE_READ_REG(hw, IXGBE_MPC(i))= ; >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0missed_rx +=3D mp; >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.mpc[i] +=3D mp; >> > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 missed_rx_cum +=3D adapter->stats.mpc[i]= ; >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rnbc[i] +=3D IXGBE_READ_= REG(hw, >> > IXGBE_RNBC(i)); >> > =A0 =A0 =A0 =A0} >> > >> > (Note that it may not apply since I've pulled the driver out into a >> > separate directory structure) >> > >> > We need the missed_rx_cum that is the total number of rx misses seen >> > because missed_rx is decremented from gprc to work around a hardware >> > issue and so needs to be the misses seen on this call to the function. >> > >> > Manish >> > >> > >> > On Thu, Aug 20, 2009 at 11:23 AM, Manish >> > Vachharajani wrote: >> >> I noticed the bogus XON, XOFF numbers. =A0I'm glad to see it will be >> >> fixed so I can cross it off my todo list. =A0:) =A0I don't think the = issue >> >> is related though, but you never know. >> >> >> >> Barney pointed out that missed_rx in the ixgbe_update_stats_counters >> >> function accumulates the missed packet registers into the missed_rx >> >> field. =A0At the end of the function, these are summed into >> >> ifp->if_ierrors which should be reported under Ierrs by netstat -idh. >> >> However that count is zero as reported via netstat. =A0The stats prin= tfs >> >> activated via sysctl dev.ix.#.stats=3D1 prints stats.mpc[0]. =A0This >> >> number is large (around 400k on the machine I've been using to find >> >> the problem). =A0If you look at the code, missed_rx and mpc[i] are >> >> updated with the same variables. =A0Given that missed_rx is unsigned,= I >> >> don't understand how the number fails to make it into ifp->if_ierrors= , >> >> but I have yet to dive into debugging this seriously. >> >> >> >> Initially, I thought the fix was simple since I didn't see the >> >> missed_rx getting added to ierrors. =A0But now, I am not sure where t= he >> >> problem is. =A0Hopefully, it will be more obvious to you than it is t= o >> >> me. =A0I'm sure it is something simple that I am missing. >> >> >> >> Manish >> >> >> >> On Thu, Aug 20, 2009 at 11:08 AM, Jack Vogel wrote= : >> >>> I've been looking at the code due to another problem of bogus flow >> >>> control >> >>> numbers, and there are some changes needed, things that should be >> >>> 82599 vs >> >>> 82598 and were not seperated properly. Will be forthcoming. Not sure >> >>> if it >> >>> has any relevance to this, but its possible. >> >>> >> >>> Jack >> >>> >> >>> >> >>> On Thu, Aug 20, 2009 at 9:53 AM, Manish Vachharajani >> >>> wrote: >> >>>> >> >>>> Oh whoops, sorry didn't see that. =A0So the plot thickens. =A0Why d= on't >> >>>> these errors show up in the netstat output I forwarded originally? >> >>>> Ierrs was 0 but the dmesg output clearly shows missed packets. =A0A= ny >> >>>> thoughts on what is going on? =A0Looking at the code, missed_rx sho= uld >> >>>> certainly get counted in the ierrors field as you said. >> >>>> >> >>>> Is the Ierrs in the netstat output some other counter? =A0If so, ho= w do >> >>>> I get the if_ierrors variable from the command line? >> >>>> >> >>>> Manish >> >>>> >> >>>> On Thu, Aug 20, 2009 at 6:49 AM, Barney >> >>>> Cordoba >> >>>> wrote: >> >>>> > >> >>>> > >> >>>> > --- On Wed, 8/19/09, Manish Vachharajani >> >>>> > >> >>>> > wrote: >> >>>> > >> >>>> >> From: Manish Vachharajani >> >>>> >> Subject: Re: Dropped vs. missed packets in the ixgbe driver >> >>>> >> To: "Barney Cordoba" >> >>>> >> Cc: freebsd-net@freebsd.org >> >>>> >> Date: Wednesday, August 19, 2009, 2:46 PM >> >>>> >> Agreed, the errors are reported but >> >>>> >> missed packets are not.=A0 The >> >>>> >> question is, is the correct fix to just add stats.mpc[0] to >> >>>> >> if_ierrors >> >>>> >> in that line or to add it to if_iqdrops.=A0 The fix is >> >>>> >> easy once we >> >>>> >> agree on what the correct behavior is. >> >>>> >> >> >>>> >> Manish >> >>>> >> >> >>>> >> > Barney wrote: >> >>>> >> > >> >>>> >> > if you look in ixgbe_update_stats_counters at the >> >>>> >> bottom: >> >>>> >> > >> >>>> >> > =A0 =A0 =A0 =A0ifp->if_ierrors =3D missed_rx + >> >>>> >> adapter->stats.crcerrs + >> >>>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0adapter->stats.rlec; >> >>>> >> > >> >>>> >> > the errors are added in. >> >>>> >> > >> >>>> >> > BC >> >>>> > >> >>>> > Huh? missed_rx are the missed packets. So they are counted. >> >>>> > >> >>>> > BC >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Manish Vachharajani >> >> >> >>>> _______________________________________________ >> >>>> freebsd-net@freebsd.org mailing list >> >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >>>> To unsubscribe, send any mail to >> >>>> "freebsd-net-unsubscribe@freebsd.org" >> >>> >> >>> >> >> >> >> -- >> >> Manish Vachharajani >> >> >> > >> >> >> >> -- >> Manish Vachharajani >> Founder >> LineRate Systems >> manishv@lineratesystems.com >> (609)635-9531 M > > --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 00:10:17 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77E421065693 for ; Fri, 21 Aug 2009 00:10:17 +0000 (UTC) (envelope-from stef-list@memberwebs.com) Received: from mx.npubs.com (mail.npubs.com [94.75.203.100]) by mx1.freebsd.org (Postfix) with ESMTP id 3ECA88FC52 for ; Fri, 21 Aug 2009 00:10:17 +0000 (UTC) Received: from mx.npubs.com (avhost [94.75.203.103]) by mx.npubs.com (Postfix) with ESMTP id 2DD783039807; Fri, 21 Aug 2009 00:10:16 +0000 (UTC) Received: from sqlserver1 (unknown [74.82.45.12]) by mx.npubs.com (Postfix) with ESMTP id B45D5303970C; Fri, 21 Aug 2009 00:10:13 +0000 (UTC) From: Stef Walter User-Agent: Thunderbird 2.0.0.22 (X11/20090608) MIME-Version: 1.0 To: Lev Serebryakov References: <20090820042016.C8F913039754@mx.npubs.com> <1038963609.20090820091230@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 7bit Message-Id: <20090821001014.B45D5303970C@mx.npubs.com> X-Virus-Scanned: ClamAV using ClamSMTP Date: Fri, 21 Aug 2009 00:10:16 +0000 (UTC) Cc: "freebsd-net@FreeBSD.org" Subject: Re: ath0: ath_rx_proc: no mbuf! X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stef@memberwebs.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 00:10:17 -0000 Lev Serebryakov wrote: > Hello, Stef. > >> ath0: ath_rx_proc: no mbuf! > >> The mbufs are in fact all used up. I allocate more via >> kern.ipc.nmbclusters, and see the same behavior. > Same problem here on 7.2-STABLE, but incresaing kern.ipc.nmbclusters > to 65536 helps. It seems, that when traffic is reauuly huge, system > with ath need a lot of mbufs. At night, when traffic is almost zero, > netstat -m shows a lot of free mbufs and clusters, so it seems, that > there is no mbuf leaks. Thanks for responding. I'll try tweaking some more. Sadly this is an 'embedded' style box that has 64 MB of RAM. According to tuning(7) Having 65536 mbuf clusters would use up 128 MB of RAM :( Cheers, Stef From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 00:21:36 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1638D1065672 for ; Fri, 21 Aug 2009 00:21:36 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id BDA398FC16 for ; Fri, 21 Aug 2009 00:21:35 +0000 (UTC) Received: by an-out-0708.google.com with SMTP id d14so111825and.13 for ; Thu, 20 Aug 2009 17:21:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=seNcSCXRLAGnsuQAGzQPfsX9AKkmCkHwOTOYP0f1R+U=; b=ei2aZhUVnP5N6OhchcUSQvXAH/tXOPYH0nSkP38k3NI3gOowcjyAoRQ+dj6M9nD4VQ 1qQ8ERbB7BPicMdgHwqXbpLXtcS1uztkqOuPQ8moBSGMq+2hCRYWhaBat8dSmZ2NBhgv tIOnWLeg5xubuFFFstJyQPWLNTGa1fRVavn3U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=dSsiZ6PAD+6hx1WFL4ir0lXSrHTJSNLQ10S0wYSp2dTkz/gWguFyHRevgJvQDCn8Ro e0sDjgXWmLy14in1s69QAX4WHDzaIFOgQh+Xp0i+E48NEY4RZQnKvmUWG23X9WZRZY1P DAvAC7b38YFrbMF7cdLBinZwTlD10/Za2ODyw= MIME-Version: 1.0 Received: by 10.100.233.19 with SMTP id f19mr541233anh.72.1250814095148; Thu, 20 Aug 2009 17:21:35 -0700 (PDT) In-Reply-To: <5bc218350908201039q574f92e3mabe73d01c35f662c@mail.gmail.com> References: <5bc218350908191146j2a22f8dcrdecb0b67eedce5c2@mail.gmail.com> <435336.24858.qm@web63908.mail.re1.yahoo.com> <5bc218350908200953p630d99c6u1538999b308c55f9@mail.gmail.com> <2a41acea0908201008y6e8f160dx27b406db7d3081b7@mail.gmail.com> <5bc218350908201023q14c51cer6effadd49cc4c604@mail.gmail.com> <5bc218350908201032l44859117obc3203ad91fc5706@mail.gmail.com> <5bc218350908201034u553df7feiaead037432279360@mail.gmail.com> <2a41acea0908201037n10505b04le924f29efd5398a7@mail.gmail.com> <5bc218350908201039q574f92e3mabe73d01c35f662c@mail.gmail.com> Date: Thu, 20 Aug 2009 17:21:35 -0700 Message-ID: <2a41acea0908201721o33372c89q25e33b8cde8edf06@mail.gmail.com> From: Jack Vogel To: Manish Vachharajani Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Barney Cordoba , freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 00:21:36 -0000 Manish, This is a diff on my changes, note some differences: first, I don't know what vintage your code was, but you do NOT want to read RNBC(i) into stats.mpc. also that is an 82598-only thing. This means that missed_rx is going to accumulate just as it should, except it should be 64 bit. This diff also contains a fix to the flow control stats on 82598, there were differences between that and 82599 that somehow got stripped from my code along the way, this corrects that. Try these changes and let me know if it works for you. Jack Index: ixgbe.c =================================================================== --- ixgbe.c (revision 195857) +++ ixgbe.c (working copy) @@ -4456,7 +4456,8 @@ { struct ifnet *ifp = adapter->ifp;; struct ixgbe_hw *hw = &adapter->hw; - u32 missed_rx = 0, bprc, lxon, lxoff, total; + u32 bprc, lxon, lxoff, total; + u64 missed_rx = 0; adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); @@ -4465,16 +4466,31 @@ mp = IXGBE_READ_REG(hw, IXGBE_MPC(i)); missed_rx += mp; adapter->stats.mpc[i] += mp; - adapter->stats.rnbc[i] += IXGBE_READ_REG(hw, IXGBE_RNBC(i)); + if (hw->mac.type == ixgbe_mac_82598EB) + adapter->stats.rnbc[i] += IXGBE_READ_REG(hw, IXGBE_RNBC(i)); } /* Hardware workaround, gprc counts missed packets */ adapter->stats.gprc += IXGBE_READ_REG(hw, IXGBE_GPRC); adapter->stats.gprc -= missed_rx; - adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GORCH); - adapter->stats.gotc += IXGBE_READ_REG(hw, IXGBE_GOTCH); - adapter->stats.tor += IXGBE_READ_REG(hw, IXGBE_TORH); + if (hw->mac.type == ixgbe_mac_82599EB) { + adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GORCL); + IXGBE_READ_REG(hw, IXGBE_GORCH); /* clears register */ + adapter->stats.gotc += IXGBE_READ_REG(hw, IXGBE_GOTCL); + IXGBE_READ_REG(hw, IXGBE_GOTCH); /* clears register */ + adapter->stats.tor += IXGBE_READ_REG(hw, IXGBE_TORL); + IXGBE_READ_REG(hw, IXGBE_TORH); /* clears register */ + adapter->stats.lxonrxc += IXGBE_READ_REG(hw, IXGBE_LXONRXCNT); + adapter->stats.lxoffrxc += IXGBE_READ_REG(hw, IXGBE_LXOFFRXCNT); + } else { + adapter->stats.lxonrxc += IXGBE_READ_REG(hw, IXGBE_LXONRXC); + adapter->stats.lxoffrxc += IXGBE_READ_REG(hw, IXGBE_LXOFFRXC); + /* 82598 only has a counter in the high register */ + adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GORCH); + adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GOTCH); + adapter->stats.tor += IXGBE_READ_REG(hw, IXGBE_TORH); + } /* * Workaround: mprc hardware is incorrectly counting @@ -4494,9 +4510,6 @@ adapter->stats.prc1522 += IXGBE_READ_REG(hw, IXGBE_PRC1522); adapter->stats.rlec += IXGBE_READ_REG(hw, IXGBE_RLEC); - adapter->stats.lxonrxc += IXGBE_READ_REG(hw, IXGBE_LXONRXCNT); - adapter->stats.lxoffrxc += IXGBE_READ_REG(hw, IXGBE_LXOFFRXCNT); - lxon = IXGBE_READ_REG(hw, IXGBE_LXONTXC); adapter->stats.lxontxc += lxon; lxoff = IXGBE_READ_REG(hw, IXGBE_LXOFFTXC); From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 12:16:32 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60097106568B for ; Fri, 21 Aug 2009 12:16:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outQ.internet-mail-service.net (outq.internet-mail-service.net [216.240.47.240]) by mx1.freebsd.org (Postfix) with ESMTP id 499BE8FC62 for ; Fri, 21 Aug 2009 12:16:31 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 5903B94C63; Fri, 21 Aug 2009 05:16:32 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 88AC12D600F; Fri, 21 Aug 2009 05:16:31 -0700 (PDT) Message-ID: <4A8E901F.6080903@elischer.org> Date: Fri, 21 Aug 2009 05:16:31 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: fabient@freebsd.org References: <4A8CFDAF.1000309@delphij.net> <200908201108.39177.max@love2party.net> <4A8D76FE.7040302@elischer.org> <1321ED43-81C5-4507-AFC0-4B2DEE71BB78@netasq.com> In-Reply-To: <1321ED43-81C5-4507-AFC0-4B2DEE71BB78@netasq.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: pf and vimage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 12:16:32 -0000 Fabien Thomas wrote: > Thanks very useful! > Do you have an "official" page to look for update. you can always find teh latest here: http://p4db.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/vimage named "porting_to_vimage.txt" > What do you think of putting it on the FreeBSD Wiki? > > Fabien > I plan to write a man page some time. From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 12:42:04 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A65106568B for ; Fri, 21 Aug 2009 12:42:04 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from netasq.netasq.com (netasq.netasq.com [213.30.137.178]) by mx1.freebsd.org (Postfix) with ESMTP id 24B978FC64 for ; Fri, 21 Aug 2009 12:42:02 +0000 (UTC) Received: from [10.2.1.5] (unknown [10.0.0.126]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by netasq.netasq.com (Postfix) with ESMTP id 4B4C81BB98; Fri, 21 Aug 2009 14:11:08 +0200 (CEST) From: Fabien Thomas To: Julian Elischer In-Reply-To: <4A8D76FE.7040302@elischer.org> References: <4A8CFDAF.1000309@delphij.net> <200908201108.39177.max@love2party.net> <4A8D76FE.7040302@elischer.org> Message-Id: <1321ED43-81C5-4507-AFC0-4B2DEE71BB78@netasq.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v936) Date: Fri, 21 Aug 2009 14:10:35 +0200 X-Mailer: Apple Mail (2.936) Cc: FreeBSD Net Subject: Re: pf and vimage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fabient@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 12:42:04 -0000 Thanks very useful! Do you have an "official" page to look for update. What do you think of putting it on the FreeBSD Wiki? Fabien Le 20 ao=FBt 09 =E0 18:17, Julian Elischer a =E9crit : > there were some people looking at adding vnet support to pf. > Since we discussed it last, the rules of the game have > significantly changed for the better. With the addition > of some new facilitiesin FreeBSD, the work needed to virtualize > a module has significantly decreased. > > > The following doc gives the new rules.. > > > August 17 2009 > Julian Elischer > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Vimage: what is it? > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Vimage is a framework in the BSD kernel which allows a co-operating =20= > module > to operate on multiple independent instances of its state so that it =20= > can > participate in a virtual machine / virtual environment scenario. It =20= > refers > to a part of the Jail infrastructure in FreeBSD. For historical =20 > reasons > "Virtual network stack enabled jails"(1) are also known as "vimage =20 > enabled > jails"(2) or "vnet enabled jails"(3). The currently correct term is =20= > the > latter, which is a contraction of the first. In the future other =20 > parts of > the system may be virtualized using the same technology and the term =20= > to > cover all such components would be VIMAGE enhanced modules. > > The implementation approach taken by the vimage framework is a =20 > redefinition > of selected global state variables to evaluate to constructs that =20 > allow for > the virtualized state to be stored and resolved in appropriate =20 > instances of > 'jail' specific container storage regions. The code operating on =20 > virtualized > state has to conform to a set of rules described further below. =20 > Among other > things in order to allow for all the changes to be conditionally =20 > compilable. > i.e. permitting the virtualized code to fall back to operation on =20 > global state. > > The rest of this document will discuss NETWORK virtualization > though the concepts may be true in the future for other parts of the > system. > > The most visible change throughout the existing code is typically =20 > replacement > of direct references to global variables with macros; foo_bar thus =20 > becomes > V_foo_bar. V_foo_bar macros will resolve back to the foo_bar global =20= > in > default kernel builds, and alternatively to the logical equivalent of > some_base_pointer->_foo_bar for "options VIMAGE" kernel configs. > > Prepending of "V_" prefixes to variable references helps in > visual discrimination between global and virtualized state. > It is also possible to use an alternative syntax, of VNET(foo_bar) to > achieve the same thing. The developers felt that V_foo_bar was less > visually distracting while still providing enough clues to the reader > that the variable is virtualized. In fact the V_foo_bar macro is > locally defined near the definition of foo_bar to be an alias for > VNET(foo_bar) so the two are not only equivalent, they are the same. > > The framework also extends the sysctl infrastructure to support =20 > access to > virtualized state through introduction of the SYSCTL_VNET family of =20= > macros; > those also automatically fall back to their standard SYSCTL =20 > counterparts > in default kernel builds. > > Transparent libkvm(3) lookups are provided to virtualized variables > which permits userland binaries such as netstat to operate unmodified > on "options VIMAGE" kernels, though this may have some security =20 > implications. > > Vnets are associated with jails. In 8.0, every process is =20 > associated with > a jail, usually the default (null) jail, and jails currently hang =20 > off of > a processes ucred. This relationship defines a process's =20 > administrative > affinity to a vnet and thus indirectly to all of its state. All =20 > network > interfaces and sockets hold pointers back to their associated vnets. > This relationship is obviously entirely independent from proc->ucred-=20= > >jail > bindings. Hence, when a process opens a socket, the socket will get =20= > bound > to a vnet instance hanging off of proc->ucred->jail->vnet, but once =20= > such a > socket->vnet binding gets established, it cannot be changed for the =20= > entire > socket lifetime. > > The mapping of a from a thread to a vnet should always be done via the > TD_TO_VNET macro as the path may change in the future as we get more > experience with using the system. > > Certain classes of network interfaces (Ethernet in particular) can be > reassigned from one vnet to another at any time. By definition all =20= > vnets > are independent and can communicate only if they are explicitly > provided with communication paths. Currently mainly netgraph is used =20= > to > establish inter-vnet datapaths, though other paths are being explored > such as the 'epair' back-to-back virtual interface pair, in which > the different sides may exist in different jails. > > In network traffic processing the vnet affinity is defined either by =20= > the > inbound interface or by the socket / pcb -> vnet binding. However, =20= > there > are many functions in the network stack that cannot implicitly fetch > the vnet context from their standard arguments. Instead of explicitly > extending argument lists of such functions with a struct vnet *, > the concept of a "current vnet", a per-thread variable was introduced, > which can be fetched efficiently via the curvnet macro. The correct > network context has to be set on entry to the network stack (socket > operations, packet reception, or timer-driven functions) and cleared =20= > on exit. > This must be done via provided CURVNET_SET() / CURVNET_RESTORE() =20 > family of > macros, which allow for "stacking" of curvnet context setting and =20 > provide > additional debugging info in INVARIANTS kernel configs. In most cases > however a developer writing virtualized code will not have to set / > restore the curvnet context unless the code would include timer-driven > events, given that those are inherently vnet-contextless on entry. > > The current rule is that when not in networking code, the result of > the 'curvnet' macro will return NULL and evaluating a V_xxx (or =20 > VNET(xxx)) > macro will result in an kernel page-fault error. While this is not =20 > strictly > necessary, it aids in debugging and assurance of program correctness. > Note this does NOT mean that TD_TO_VNET(curthread) is invalid. > A thread is always associated with a vnet, but just the efficient > "curvnet" access method is disabled along with the ability to resolve > virtualized symbols. > > > Converting / virtualizing existing code > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > There are several steps need in virtualisation. > > 1/ Decide whether the module needs to be virtualised. > > If the module is a driver for specific hardware, it makes sense that > there be only one instance of the driver as there is only one =20 > piece of > physical hardware. There are changes in the networking code to =20 > allow > physical (or virtual) interfaces to be moved between vnets. This > generally requires NO changes to the network drivers of the classes > covered (e.g. ethernet). Currently if your module is does not have =20= > any > networking facet, the answer is "no" by default. > > 2/ If the module is to be virtualised, decide which attributes of the > module should be virtualised. > > For example, It may make sense that there be a single central pool > of "struct foo" and a single uma zone for them to come from, with =20= > a single > lock guarding it. It might also make sense if the "foo_debug" sysctl > controls all the instances at once, while on the other hand, the > "foo_mode" sysctl might make better sense if it were controllable > on a virtual system by virtual system basis. > > 3/ Work out what global variables and structures are to be =20 > virtualised to > achieve the behaviour required for part #2. > > 4/ Work out for all the code paths through the module, how the =20 > thread entering > the module can divine which virtual environment it is on. > > Some examples: > * Since interfaces are all assigned to one vnet or another, an =20 > incoming > packet has a pointer to the receive interface, which in turn has a > pointer back to the vnet. Often "curvnet" will already have been =20= > set > by the time your code is called anyhow. > * Similarly, on any request from outside the kernel, (direct or =20 > indirect) > the current thread has a way to get to the current virtual =20 > environment > instance via TD_TO_VNET(curthread). For existing sockets the vnet > context must be used via so->so_vnet since the thread's vnet might > change after socket creation. > * Timer initiated actions usually have a (void *) argument which =20 > points to > some private structure for the module. It should be possible to =20= > add > a pointer to the appropriate module instance into whatever =20 > structure > that points to. > * Sometimes an action (timer trigerred or trigerred by module load =20= > or > unload simply has to check all the vimage or module instances. > There are macro (pairs) for this which will iterate through all =20= > the > VNET or instances. (see sample code below). > > This covers most of the cases, however in some cases it may still be > required for the module to stash away the virtual environment =20 > instance > somewhere, and make associated changes in the code. > > 5/ Decide which parts of the initialization and teardown are per =20 > jail and > which parts are global, and separate out the code accordingly. > Global initialization is done using the SYSINIT facility. > Per jail initialization is done using VNET_SYSINIT(). > Per jail teardown is doen using VNET_SYSUNINIT(). > Global teardown is done using SYSUNIT(). > In addition, the modevent handler is called with various event =20 > types before > any of these are called. The modevent handler may veto load or =20 > teardown. > On Shutdown, only the modevent handler is called so it may have to =20= > simulate > the calling of the other handlers if clean shutdown is a requirement > of your module. (see sample code below). Don't forget to unregister > event handlers, and destroy locks and condition variables. > > 6/ Add the code described below to the files that make up the module. > > Details: (VNET implementation details) > > Firstly the file must be included. Depending on what > code you use you may find you also need one or more of: , > and . These requirements may change slightly > as the ABI settles. > > Having decided which variables need to be virtualized, the definition > of thosvariables needs to be modified to use the VNET_DEFINE() macro. > For example: > > static int foo =3D 3; > struct bar thebar =3D { 1,2,3 }; > > would become: > > static VNET_DEFINE(int, foo) =3D 3; > VNET_DEFINE(struct bar, thebar) =3D { 1,2,3 }; > > extern int foo; > in an include file might become: > VNET_DECLARE(int foo); > > Normal rules regarding 'static/extern' apply. The initial values =20 > that you > give in this way will be stored and used as the initial values for > EACH NEW INSTANCE of these variables as new jails/vnets are created. > > As mentioned above, accesses to virtualized symbols are achieved via =20= > macros, > which generally are of the same name as the original symbol but with =20= > a "V_" > prepended, thus the head of the interface list, called 'ifnet' is =20 > replaced > whereever used with "V_ifnet". We do this, by adding the following > lines after the definitions above: > > #define V_foo VNET(foo) > #define V_thebar VNET(thebar) > > --- side-note --- > In SCTP, because the code is shared with > other OS's they are replaced with a macro MODULE_GLOBAL(modulename, =20= > symbol). > (this may simplify in light of recent changes). > -------------- > > In addition, should any of your values need to be changed or viewed > via sysctl, the following SYSCTL definitions would be needed: > > SYSCTL_VNET_PROC(_net_inet, OID_AUTO, thebar, > CTLTYPE_?? | CTLFLAG_RW | CTLFLAG_SECURE3, &VNET_NAME(thebar), 0, > thebar, "?", "the bar is open"); > {[XXX] robert fix this is possible ^^^} > SYSCTL_VNET_INT(_net_inet, OID_AUTO, foo, > CTLFLAG_RW, &VNET_NAME(foo), 0, "size of foo"); > > > In the current version of vimage, when VIMAGE is not compiled into > the kernel, the macros evaluate to a direct reference to the one and =20= > only > symbol/variable, so that there is no speed penalty for those not =20 > using vnets. > > When VIMAGE is compiled in, the macro will evaluate to an access to =20= > an offset > into a data structure that is accessed on a per-vet basis. The vnet > used for this is always curvnet. For this reason an attempt to access > such a variable while curvnet is not valid, will result in an =20 > exception. > > To ensure that curvnet has a valid value when needed one needs to > add the following code on all entry code paths into the networking =20 > code: > int > my_func(int arg) > { > CURVNET_SET(TD_TO_VNET(curthread)); > do_my_network_stuff(arg); > CURVNET_RESTORE(); > return (0); > } > > The initial value is usually something like "TD_TO_VNET(curthread) > which in turn is a macro that derives the vnet affinity from the =20 > current > thread. It could also be (m->m_ifp->if_vnet) if we were receiving =20 > an mbuf, > or so->so_vnet if we had a socket involved. > > Usually, when a packet enters the system it is carried through the =20 > processing > path via a single thread, and that thread will set its virtual =20 > environment > reference to that indicated by the packet on picking up that new =20 > packet. > This means that in the normal inbound processing path as well as the > outgoing process path the current thread can be used to indicate the > current virtual environment and curvet will always be valid once most > user supplied code is reached. In timer events, it is sometimes > necessary to add an "outer loop" to iterate through all the possible =20= > vnets > if there is just one timer for all instances. > > When a new loadable module is virtualised the module definitions > and intializers need to be examined. The following example illustrates > what is needed in the case that you are not loading a new protocol, =20= > or domain. > (for that see later) > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D sample skeleton code = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > /* init on boot or module load */ > static int > mymod_init(void) > { > return (error); > } > > /**************** > * Stuff that must be initialized for every instance > * (including the first of course). > */ > static int > mymod_vnet_init(const void *unused) > { > return (0); > } > > /********************** > * Called for the removal of the last instance only on module unload. > */ > static void > mymod_uninit(void) > { > } > > /*********************** > * Called for the removal of each instance. > */ > static int > mymod_vnet_uninit(const void *unused) > { > return (0) > } > > mymod_modevent(module_t mod, int type, void *unused) > { > int err =3D 0; > > switch (type) { > case MOD_LOAD: > /* check that loading is ok */ > break; > > case MOD_UNLOAD: > /* check that unloading is ok */ > break; > > case MOD_QUIESCE: > /* warning: try stop processing */ > /* maybe sleep 1 mSec or something to let threads get = out */ > break; > > case MOD_SHUTDOWN: > /* > * this is called once but you may want to shut down > * things in each jail, or something global. > * In that case it's up to us to simulate the = SYSUNINIT() > * or the VNET_SYSUNINIT() > */ > { > VNET_ITERATOR_DECL(vnet_iter); > VNET_LIST_RLOCK(); > VNET_FOREACH(vnet_iter) { > CURVNET_SET(vnet_iter); > mymod_vnet_uninit(NULL); > CURVNET_RESTORE(); > } > VNET_LIST_RUNLOCK(); > } > /* you may need to shutdown something global. */ > mymod_uninit(); > break; > > default: > err =3D EOPNOTSUPP; > break; > } > return err; > } > > static moduledata_t mymodmod =3D { > "mymod", > mymod_modevent, > 0 > }; > > /* define execution order using constants from /sys/sys/kernel.h */ > #define MYMOD_MAJOR_ORDER SI_SUB_PROTO_BEGIN /* for =20 > example */ > #define MYMOD_MODULE_ORDER (SI_ORDER_ANY + 64) /* not =20 > fussy */ > #define MYMOD_SYSINIT_ORDER (MYMOD_MODULE_ORDER + 1) /* a bit =20 > later */ > #define MYMOD_VNET_ORDER (MYMOD_MODULE_ORDER + 2) /* later =20 > still */ > > DECLARE_MODULE(mymod, mymodmod, MYMOD_MAJOR_ORDER, =20 > MYMOD_MODULE_ORDER); > MODULE_DEPEND(mymod, ipfw, 2, 2, 2); /* depend on ipfw version =20 > (exactly) 2 */ > MODULE_VERSION(mymod, 1); > > SYSINIT(mymod_init, MYMOD_MAJOR_ORDER, MYMOD_SYSINIT_ORDER, > mymod_init, NULL); > SYSUNINIT(mymod_uninit, MYMOD_MAJOR_ORDER, MYMOD_SYSINIT_ORDER, > mymod_uninit, NULL); > > VNET_SYSINIT(mymod_vnet_init, MYMOD_MAJOR_ORDER, MYMOD_VNET_ORDER, > mymod_vnet_init, NULL); > VNET_SYSUNINIT(mymod_vnet_uninit, MYMOD_MAJOR_ORDER, MYMOD_VNET_ORDER, > mymod_vnet_uninit, NULL); > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D end sample code =3D=3D=3D=3D=3D=3D=3D > > On BOOT, the order of evaluation will be: > In a NON-VIMAGE kernel where the module is compiled: > MODEVENT, SYSINIT and VNET_SYSINIT both runm with order defined =20= > by their > order declarations. {good foot shooting material if you get it =20 > wrong!} > > In a VIMAGE kernel where the module is compiled in: > MODEVNET, SYSINIT and VNET_SYSINIT all run with order defined by =20= > their > order declarations. AND in addition, the VNET_SYSINIT is > repeated once for every existing or new jail/vnet. > > On loading a vnet enabled kernel module after boot: > MODEVENT("event =3D load"); > SYSINIT() > VNET_SYSINIT() for every existing jail > AND in addition, VNET_SYSINIT being called for each new jail =20= > created. > > On unloading of module: > MODEVENT("event =3D MOD_QUIESCE") > MODEVENT("event =3D MOD_UNLOAD") > VNET_SYSUNINIT called for every jail/vnet > SYSUNINIT > > On system shutdown: > MODEVENT(shutdown) > > NOTICE that while the order of the SYSINIT and VNET_SYSINIT is =20 > reversed from > that of SYSUNINIT and VNET_SYSUNINIT, MODEVENTS do not follow > this rule and thus it is dangerous to initialise and uninitialise > things which are order dependent using MODEVENTs. > > Or, put another way, > Since MODEVENT is called first during module load, it would, by the > assumption that everything is reversed, be easy to assume that =20 > MODEVENT > is called AFTER the SYSINITS during unload. This is in fact not > the case. (and I have the scars to prove it). > > It might be make some sense if the "QUIESCE" was called before the > SYSINIT/SYSUNINIT and the UNLOAD called after.. with a millisecond > sleep between them, but this is not the case either. > > Since initial values are copied into the virtualized variables > on each new instantiatin, it is quite possible to have modules for =20 > which > some of the above methods are not needed, and they may be left out. > (but not the modevent). > > Sometimes there is a need to iterate through the vnets. > See the modevent shutdown handler (above) for an example of how to =20 > do this. > Don't forget the locks. > > In the case where you are loading a new protocol, or domain =20 > (protocol family) > there are some "shortcuts" that are in place to allow you to =20 > maintain a bit > more source compatibility with older revisions of FreeBSD. It must be > added that the sample code above works just fine for protocols, =20 > however > protcols also have an aditional initialization vector which is via the > prtocol structure, which has a pr_init() entry. > When a protocol is registered using pf_proto_register(), the pr_init() > for the protocol is called once for every existing vnet. in addition, > it will be called for each new vnet. The pr_destroy() method will be =20= > called > as well on vnet teardown. The pf_proto_register() funcion can be =20 > called > either from a modevent handler of from the SYSINIT() if you have =20 > one, and > the pf_proto_unregister() called from the SYSUNINIT or the unload > modevent handler. > > If you are adding a whole new protocol domain, (protocol family) then > you should add the VNET_DOMAIN_SET(domainname) (e,g, inet, inet6) > macro. These use VNET_SYSINIT internally to indirectly call the > dom_init() and pr_init() functions for each vnet, (and the =20 > equivalent for > teardown.) In this case one needs to be absolutely sure that both =20 > your > domain and protocol initializers can be called multiple times, once =20= > for > each vnet. One can still add SYSINITs for once only initialization, > or use the modevent handler. I prefer to do as much explicitly > in the SYSINITS and VNET_SYSINITS as then you have no surprises. > > finally: > The command to make a new jail with a new vnet: > jail -c host.hostname=3Dtest path=3D/ vnet command=3D/bin/tcsh > jail -c host.hostname=3Dtest path=3D/ children.max=3D4 vnet = command=3D/bin/=20 > tcsh > (children.max allows hierarchical jail creation). > Note that the command must come last. > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 14:20:42 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1BDD106564A for ; Fri, 21 Aug 2009 14:20:42 +0000 (UTC) (envelope-from john@traktor.dnepro.net) Received: from traktor.dnepro.net (roof1.dnepro.net [212.3.111.66]) by mx1.freebsd.org (Postfix) with ESMTP id 457B58FC16 for ; Fri, 21 Aug 2009 14:20:41 +0000 (UTC) Received: from traktor.dnepro.net (localhost [127.0.0.1]) by traktor.dnepro.net (8.14.3/8.14.3) with ESMTP id n7LEKdtx049433 for ; Fri, 21 Aug 2009 17:20:39 +0300 (EEST) (envelope-from john@traktor.dnepro.net) Received: (from john@localhost) by traktor.dnepro.net (8.14.3/8.14.3/Submit) id n7LEKdYZ049430 for freebsd-net@freebsd.org; Fri, 21 Aug 2009 17:20:39 +0300 (EEST) (envelope-from john) Date: Fri, 21 Aug 2009 17:20:39 +0300 From: Eugene Perevyazko To: freebsd-net@freebsd.org Message-ID: <20090821142039.GA40018@traktor.dnepro.net> Mail-Followup-To: freebsd-net@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: D-Link DGE-560SX (Marvell 88E8061-based) doesn't see link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 14:20:42 -0000 Hello, Freebsd-net subscribers! This is on 7.2-Stable. I've got a D-Link DGE-560SX GigE fiber adapter. It's Marvell 88E8061 so I hoped it will work as msk(4). D-Link has changed PCI Id for the chip: none1@pci0:1:0:0: class=0x020000 card=0x4b021186 chip=0x4b021186 rev=0x10 hdr=0x00 vendor = 'D-Link System Inc' device = 'DGE-560SX PCIe Gigabit Ethernet Adapter' class = network subclass = ethernet So I've patched the driver to recognize new IDs. --- /sys/dev/msk/if_mskreg.h.old 2009-08-14 17:04:09.000000000 +0300 +++ /sys/dev/msk/if_mskreg.h 2009-08-14 17:04:49.000000000 +0300 @@ -149,6 +149,7 @@ */ #define DEVICEID_DLINK_DGE550SX 0x4001 #define DEVICEID_DLINK_DGE560T 0x4b00 +#define DEVICEID_DLINK_DGE560SX 0x4b02 #define BIT_31 (1 << 31) #define BIT_30 (1 << 30) --- /sys/dev/msk/if_msk.c.old 2009-08-14 17:04:03.000000000 +0300 +++ /sys/dev/msk/if_msk.c 2009-08-14 17:12:48.000000000 +0300 @@ -224,7 +224,9 @@ { VENDORID_DLINK, DEVICEID_DLINK_DGE550SX, "D-Link 550SX Gigabit Ethernet" }, { VENDORID_DLINK, DEVICEID_DLINK_DGE560T, - "D-Link 560T Gigabit Ethernet" } + "D-Link 560T Gigabit Ethernet" }, + { VENDORID_DLINK, DEVICEID_DLINK_DGE560SX, + "D-Link 560SX Gigabit Ethernet" } }; static const char *model_name[] = { Then the driver recognizes the card: mskc0: port 0x4000-0x40ff mem 0xdc100000-0xdc103fff irq 16 at device 0.0 on pci1 msk0: on mskc0 msk0: Ethernet address: 00:21:91:52:4f:09 miibus1: on msk0 e1000phy0: PHY 0 on miibus1 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto mskc0: [FILTER] Now I'm stuck because it doesn't see link. # ifconfig msk0 up # ifconfig -m msk0 msk0: flags=8843 metric 0 mtu 1500 options=11a capabilities=11a ether 00:21:91:52:4f:09 media: Ethernet autoselect (100baseTX ) status: no carrier supported media: media autoselect media 1000baseTX mediaopt full-duplex media 1000baseTX media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP media none The adapter is definitely not broken, I've tested it on Win and there it works normally. Also on Win it has no speed/duplex settings, while msk driver allows to change media. I've also tried Marvell's binary if_myk driver, but it doesn't know about d-link. Can anyone propose something to get it working? -- Eugene Perevyazko From owner-freebsd-net@FreeBSD.ORG Fri Aug 21 22:20:13 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8656C106568B for ; Fri, 21 Aug 2009 22:20:13 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.26]) by mx1.freebsd.org (Postfix) with ESMTP id 380D78FC0C for ; Fri, 21 Aug 2009 22:20:13 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 3so638488qwe.7 for ; Fri, 21 Aug 2009 15:20:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:subject :message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=zkAXBGtH+AF8nE5pqJcWFLk9yaLFadzlFHG4f2j/Brc=; b=phIJLhBPsP9D54TuoDS6JAERGGIFagUND4cw8athMrUCYUrjBoZaN2QThDuMknEvZ1 RTAg54fVfPq/hN9KHDUJgB/dJUNnX85oeF5vVELztMtmyoYjEixIqGVfNT+Lgic/bwBL cy0b1YAaw++jm70j8G7a+qdKt03TS2aZRwLnA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=N/HBy/Vp6oWIE3qMq5htyzya1wUpdRxsaRO86i05+ECGwf3k5w3O0Rff8s1Z0k7kev EB3dd4RkspIwW7KesiafyHeeH8x38+6MECsYjaBv/ICz1eTJMtk65qi0h2Jk1ib8gVd2 3Us3ebOONacoEbxtXj9cZaoJ3+ftPYXuNKHus= Received: by 10.224.118.142 with SMTP id v14mr1248964qaq.41.1250893212523; Fri, 21 Aug 2009 15:20:12 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 2sm2695104qwi.23.2009.08.21.15.20.10 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 21 Aug 2009 15:20:11 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 21 Aug 2009 15:19:32 -0700 From: Pyun YongHyeon Date: Fri, 21 Aug 2009 15:19:32 -0700 To: freebsd-net@freebsd.org Message-ID: <20090821221932.GE1262@michelle.cdnetworks.com> References: <20090821142039.GA40018@traktor.dnepro.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090821142039.GA40018@traktor.dnepro.net> User-Agent: Mutt/1.4.2.3i Subject: Re: D-Link DGE-560SX (Marvell 88E8061-based) doesn't see link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 22:20:13 -0000 On Fri, Aug 21, 2009 at 05:20:39PM +0300, Eugene Perevyazko wrote: > Hello, Freebsd-net subscribers! > > This is on 7.2-Stable. > I've got a D-Link DGE-560SX GigE fiber adapter. It's Marvell 88E8061 so > I hoped it will work as msk(4). > D-Link has changed PCI Id for the chip: > none1@pci0:1:0:0: class=0x020000 card=0x4b021186 chip=0x4b021186 rev=0x10 > hdr=0x00 > vendor = 'D-Link System Inc' > device = 'DGE-560SX PCIe Gigabit Ethernet Adapter' > class = network > subclass = ethernet > > > So I've patched the driver to recognize new IDs. > Your patch looks to me. [...] > Then the driver recognizes the card: > > mskc0: port 0x4000-0x40ff mem 0xdc100000-0xdc103fff irq 16 at device 0.0 on pci1 > msk0: on mskc0 > msk0: Ethernet address: 00:21:91:52:4f:09 > miibus1: on msk0 > e1000phy0: PHY 0 on miibus1 > e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This part is problem, you have finer media, so you should have just 1000baseSX. > mskc0: [FILTER] > > Now I'm stuck because it doesn't see link. > # ifconfig msk0 up > # ifconfig -m msk0 > msk0: flags=8843 metric 0 mtu 1500 > options=11a > capabilities=11a > ether 00:21:91:52:4f:09 > media: Ethernet autoselect (100baseTX ) > status: no carrier > supported media: > media autoselect > media 1000baseTX mediaopt full-duplex > media 1000baseTX > media 100baseTX mediaopt full-duplex > media 100baseTX > media 10baseT/UTP mediaopt full-duplex > media 10baseT/UTP > media none > > The adapter is definitely not broken, I've tested it on Win and there it works > normally. Also on Win it has no speed/duplex settings, while msk driver allows > to change media. > ATM there is no easy/clean way to pass driver specific data to mii layer in FreeBSD so e1000phy(4) incorrectly thinks it found copper PHY. Marvell PHYs seem to have no reliable way to know configured media type of PHY hardware unless parent driver(msk) gives hint to it. If you have just 1 NIC which uses e1000phy(4) on your system, modify e1000phy(4) to force it having fiber media by inserting the following line around line 114 in e1000phy.c. sc->mii_flags |= MIIF_HAVEFIBER; > I've also tried Marvell's binary if_myk driver, but it doesn't know about d-link. > > Can anyone propose something to get it working? > > -- > Eugene Perevyazko