From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 14:23:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0CE5ED2B for ; Sun, 28 Apr 2013 14:23:35 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm17-vm2.bullet.mail.ne1.yahoo.com (nm17-vm2.bullet.mail.ne1.yahoo.com [98.138.91.93]) by mx1.freebsd.org (Postfix) with ESMTP id A56301750 for ; Sun, 28 Apr 2013 14:23:34 +0000 (UTC) Received: from [98.138.90.55] by nm17.bullet.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 14:21:42 -0000 Received: from [98.138.89.194] by tm8.bullet.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 14:21:42 -0000 Received: from [127.0.0.1] by omp1052.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 14:21:42 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 223876.99225.bm@omp1052.mail.ne1.yahoo.com Received: (qmail 13108 invoked by uid 60001); 28 Apr 2013 14:21:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1367158902; bh=HDYc9WZlrK3gIhp2UvNV8yPoVozlSf8gZV+3u6/Q41Q=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=WZkh5r5QHxHPGDBIw3yT/zLWTn67z3w2FeEuo5j23Ue3fJEvsdj3rac3AQFDaTI6TluXwfNPncfO6WMJ7zbHliAgDp4Lm9E4KB2wEBZK1fkPYlAbJrnAoCgMCN6BZjEtQu9Ipj313Y8gOcUNmTSI7kGWYT+96UzbrjcfHLhnWCA= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=pG2sMnf6jQte+LcP3a2UFXmc6D94kmLxQY6gXqaqX5tLeMqa3XfQ+b9P6kkNdA5ci1LI75svqfZRPINaHtYDOCVQOMpZlfboBQ6hkEzwFQDE+MLzrivDWKNEAN6domwAgHAsgDhK+bRZDEZ0mP07GZkmDdp09bPiIMUEoaYD7OA=; X-YMail-OSG: nxgdSYkVM1n5saCQnFbd33iXeOLOXWdAHbsnUNthbAUmU6P 7xz4SuiUURY2Qd5QCxz5X537NohW77oMiM51r8LzAJhZXQh6mljhGGnltyGi FZ4tDY48vpzh6xERMSOfkUc6RA5zJdTbt7nZXDgkBHMo1zpya94yGoDttTqW 5_bjNRFvXrv961YhaiqehrPv11DXEsRAlD6Y_2bD5EQY5qAT2N6Tvp4gHzxz zOSv10aSebN2Tw83asVJpLPCiGVyx4FGHKP7nkECO1r9OFF3Gs3bpB752Z_m 9hkCu8hUXsqTicnRSqFzfy8lmpiF_sC3ipD_3Wxi7gsL4HEgUAWYfdg0KXoP 8EojF4Jutu5b0gWQbsmWaNC74QIoYmVnQeuC18VMComrt7GgOzdc.dGxNkT8 YrKj4FjjRmRK3O_5QP5iQBJWBgK5hlV2zCe.4zcLd9YDq8NrVFMdog4huPRx S2D2sUq0W7EfaXsxAYmDxrD45ur5Poqu0AJlOT.vHHyRTpRvEFgTCSCTMeps XZz92OcTjXBC4QiRut1mzJs9KMpyKNw-- Received: from [98.203.118.124] by web121603.mail.ne1.yahoo.com via HTTP; Sun, 28 Apr 2013 07:21:42 PDT X-Rocket-MIMEInfo: 002.001, CgotLS0gT24gRnJpLCA0LzI2LzEzLCAiQ2zDqW1lbnQgSGVybWFubiAobm9kZW5zKSIgPG5vZGVuczIwOTlAZ21haWwuY29tPiB3cm90ZToKCj4gRnJvbTogIkNsw6ltZW50IEhlcm1hbm4gKG5vZGVucykiIDxub2RlbnMyMDk5QGdtYWlsLmNvbT4KPiBTdWJqZWN0OiBIaWdoIENQVSBpbnRlcnJ1cHQgbG9hZCBvbiBpbnRlbCBJMzUwVDQgd2l0aCBpZ2Igb24gOC4zCj4gVG86IGZyZWVic2QtbmV0QGZyZWVic2Qub3JnCj4gRGF0ZTogRnJpZGF5LCBBcHJpbCAyNiwgMjAxMywgNzozMSBBTQo.IEhpIGxpc3QsCj4BMAEBAQE- X-Mailer: YahooMailClassic/15.1.8 YahooMailWebService/0.8.141.536 Message-ID: <1367158902.96158.YahooMailClassic@web121603.mail.ne1.yahoo.com> Date: Sun, 28 Apr 2013 07:21:42 -0700 (PDT) From: Barney Cordoba Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 To: freebsd-net@freebsd.org, =?iso-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= In-Reply-To: <517A657B.7060003@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 14:23:35 -0000 =0A=0A--- On Fri, 4/26/13, "Cl=E9ment Hermann (nodens)" wrote:=0A=0A> From: "Cl=E9ment Hermann (nodens)" = =0A> Subject: High CPU interrupt load on intel I350T4 with igb on 8.3=0A> T= o: freebsd-net@freebsd.org=0A> Date: Friday, April 26, 2013, 7:31 AM=0A> Hi= list,=0A> =0A> We use pf+ALTQ for trafic shaping on some routers.=0A> =0A>= We are switching to new servers : Dell PowerEdge R620 with 2=0A> 8-cores I= ntel Processor (E5-2650L), 8GB RAM and Intel I350T4=0A> (quad port) using i= gb driver. The old hardware is using em=0A> driver, the CPU load is high bu= t mostly due to kernel and a=0A> large pf ruleset.=0A> =0A> On the new hard= ware, we see high CPU Interrupt load (up to=0A> 95%), even though there is = not much trafic currently (peaks=0A> about 150Mbps and 40Kpps). All queues = are used and binded to=0A> a cpu according to top, but a lot of CPU time is= spent on=0A> igb queues (interrupt or wait). The load is fine when we=0A> = stay below 20Kpps.=0A> =0A> We see no mbuf shortage, no dropped packet, but= there is=0A> little margin left on CPU time (about 25% idle at best, most= =0A> of CPU time is spent on interrupts), which is disturbing.=0A> =0A> We = have done some tuning, but to no avail :=0A> =0A> sysctl.conf :=0A> =0A> # = mbufs=0A> kern.ipc.nmbclusters=3D65536=0A> # Sockets=0A> kern.ipc.somaxconn= =3D8192=0A> net.inet.tcp.delayed_ack=3D0=0A> net.inet.tcp.sendspace=3D65535= =0A> net.inet.udp.recvspace=3D65535=0A> net.inet.udp.maxdgram=3D57344=0A> n= et.local.stream.recvspace=3D65535=0A> net.local.stream.sendspace=3D65535=0A= > # IGB=0A> dev.igb.0.rx_processing_limit=3D4096=0A> dev.igb.1.rx_processin= g_limit=3D4096=0A> dev.igb.2.rx_processing_limit=3D4096=0A> dev.igb.3.rx_pr= ocessing_limit=3D4096=0A> =0A> /boot/loader.conf :=0A> =0A> vm.kmem_size=3D= 1G=0A> hw.igb.max_interrupt_rate=3D"32000"=A0 # maximum number of=0A> inter= rupts/sec generated by single igb(4) (default 8000)=0A> hw.igb.txd=3D"2048"= =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 # numbe= r of transmit descriptors allocated by the=0A> driver (2048 limit)=0A> hw.i= gb.rxd=3D"2048"=A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =0A> =A0 # number of receive descriptors allocated by the=0A> driver (2048 = limit)=0A> hw.igb.rx_process_limit=3D"1000"=A0 =A0=A0=A0#=0A> maximum numbe= r of received packets to process at a time, The=0A> default of 100 is=0A> = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 =A0 =A0 =A0 =A0=0A> =A0=A0=A0# too= low for most firewalls. (-1 means=0A> unlimited)=0A> =0A> Kernel HZ is 100= 0.=0A> =0A> The IGB /boot/loader.conf tuning was our last attempt, it=0A> d= idn't change anything.=0A> =0A> Does anyone have any pointer ? How could we= lower CPU=0A> interrupt load ? should we set hw.igb.max_interrupt_rate=0A>= lower instead of higher ?=0A> From what we saw here and there, we should b= e able to do=0A> much better with this hardware.=0A> =0A> =0A> relevant sys= ctl (igb1 and igb2 only, other interfaces are=0A> unused) :=0A> =0A> sysctl= dev.igb | grep -v ": 0$"=0A> dev.igb.1.%desc: Intel(R) PRO/1000 Network Co= nnection=0A> version - 2.3.1=0A> dev.igb.1.%driver: igb=0A> dev.igb.1.%loca= tion: slot=3D0 function=3D1=0A> dev.igb.1.%pnpinfo: vendor=3D0x8086 device= =3D0x1521=0A> subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000=0A> de= v.igb.1.%parent: pci5=0A> dev.igb.1.nvm: -1=0A> dev.igb.1.enable_aim: 1=0A>= dev.igb.1.fc: 3=0A> dev.igb.1.rx_processing_limit: 4096=0A> dev.igb.1.eee_= disabled: 1=0A> dev.igb.1.link_irq: 2=0A> dev.igb.1.device_control: 1209795= 137=0A> dev.igb.1.rx_control: 67141658=0A> dev.igb.1.interrupt_mask: 4=0A> = dev.igb.1.extended_int_mask: 2147483981=0A> dev.igb.1.fc_high_water: 33168= =0A> dev.igb.1.fc_low_water: 33152=0A> dev.igb.1.queue0.interrupt_rate: 714= 28=0A> dev.igb.1.queue0.txd_head: 1318=0A> dev.igb.1.queue0.txd_tail: 1318= =0A> dev.igb.1.queue0.tx_packets: 84663594=0A> dev.igb.1.queue0.rxd_head: 7= 17=0A> dev.igb.1.queue0.rxd_tail: 715=0A> dev.igb.1.queue0.rx_packets: 4389= 9597=0A> dev.igb.1.queue0.rx_bytes: 8905556030=0A> dev.igb.1.queue1.interru= pt_rate: 90909=0A> dev.igb.1.queue1.txd_head: 693=0A> dev.igb.1.queue1.txd_= tail: 693=0A> dev.igb.1.queue1.tx_packets: 57543349=0A> dev.igb.1.queue1.rx= d_head: 1033=0A> dev.igb.1.queue1.rxd_tail: 1032=0A> dev.igb.1.queue1.rx_pa= ckets: 54821897=0A> dev.igb.1.queue1.rx_bytes: 9944955108=0A> dev.igb.1.que= ue2.interrupt_rate: 100000=0A> dev.igb.1.queue2.txd_head: 350=0A> dev.igb.1= .queue2.txd_tail: 350=0A> dev.igb.1.queue2.tx_packets: 62320990=0A> dev.igb= .1.queue2.rxd_head: 1962=0A> dev.igb.1.queue2.rxd_tail: 1939=0A> dev.igb.1.= queue2.rx_packets: 43909016=0A> dev.igb.1.queue2.rx_bytes: 8673941461=0A> d= ev.igb.1.queue3.interrupt_rate: 14925=0A> dev.igb.1.queue3.txd_head: 647=0A= > dev.igb.1.queue3.txd_tail: 647=0A> dev.igb.1.queue3.tx_packets: 58776199= =0A> dev.igb.1.queue3.rxd_head: 692=0A> dev.igb.1.queue3.rxd_tail: 691=0A> = dev.igb.1.queue3.rx_packets: 55138996=0A> dev.igb.1.queue3.rx_bytes: 931021= 7354=0A> dev.igb.1.queue4.interrupt_rate: 100000=0A> dev.igb.1.queue4.txd_h= ead: 1721=0A> dev.igb.1.queue4.txd_tail: 1721=0A> dev.igb.1.queue4.tx_packe= ts: 54337209=0A> dev.igb.1.queue4.rxd_head: 1609=0A> dev.igb.1.queue4.rxd_t= ail: 1598=0A> dev.igb.1.queue4.rx_packets: 46546503=0A> dev.igb.1.queue4.rx= _bytes: 8818182840=0A> dev.igb.1.queue5.interrupt_rate: 11627=0A> dev.igb.1= .queue5.txd_head: 254=0A> dev.igb.1.queue5.txd_tail: 254=0A> dev.igb.1.queu= e5.tx_packets: 53117182=0A> dev.igb.1.queue5.rxd_head: 701=0A> dev.igb.1.qu= eue5.rxd_tail: 685=0A> dev.igb.1.queue5.rx_packets: 43014837=0A> dev.igb.1.= queue5.rx_bytes: 8699057447=0A> dev.igb.1.queue6.interrupt_rate: 55555=0A> = dev.igb.1.queue6.txd_head: 8=0A> dev.igb.1.queue6.txd_tail: 8=0A> dev.igb.1= .queue6.tx_packets: 52654088=0A> dev.igb.1.queue6.rxd_head: 1057=0A> dev.ig= b.1.queue6.rxd_tail: 1041=0A> dev.igb.1.queue6.rx_packets: 45227030=0A> dev= .igb.1.queue6.rx_bytes: 9494489640=0A> dev.igb.1.queue7.interrupt_rate: 523= 5=0A> dev.igb.1.queue7.txd_head: 729=0A> dev.igb.1.queue7.txd_tail: 729=0A>= dev.igb.1.queue7.tx_packets: 61926105=0A> dev.igb.1.queue7.rxd_head: 146= =0A> dev.igb.1.queue7.rxd_tail: 140=0A> dev.igb.1.queue7.rx_packets: 517817= 75=0A> dev.igb.1.queue7.rx_bytes: 8901279226=0A> dev.igb.1.mac_stats.missed= _packets: 1657=0A> dev.igb.1.mac_stats.recv_no_buff: 405=0A> dev.igb.1.mac_= stats.total_pkts_recvd: 384332760=0A> dev.igb.1.mac_stats.good_pkts_recvd: = 384331103=0A> dev.igb.1.mac_stats.bcast_pkts_recvd: 15510=0A> dev.igb.1.mac= _stats.mcast_pkts_recvd: 52957=0A> dev.igb.1.mac_stats.rx_frames_64: 195496= 498=0A> dev.igb.1.mac_stats.rx_frames_65_127: 133346124=0A> dev.igb.1.mac_s= tats.rx_frames_128_255: 5254911=0A> dev.igb.1.mac_stats.rx_frames_256_511: = 9700049=0A> dev.igb.1.mac_stats.rx_frames_512_1023: 16885886=0A> dev.igb.1.= mac_stats.rx_frames_1024_1522: 23647635=0A> dev.igb.1.mac_stats.good_octets= _recvd: 74284029276=0A> dev.igb.1.mac_stats.good_octets_txd: 544536708502= =0A> dev.igb.1.mac_stats.total_pkts_txd: 485327419=0A> dev.igb.1.mac_stats.= good_pkts_txd: 485327419=0A> dev.igb.1.mac_stats.bcast_pkts_txd: 72=0A> dev= .igb.1.mac_stats.mcast_pkts_txd: 52820=0A> dev.igb.1.mac_stats.tx_frames_64= : 57820809=0A> dev.igb.1.mac_stats.tx_frames_65_127: 51586341=0A> dev.igb.1= .mac_stats.tx_frames_128_255: 7050579=0A> dev.igb.1.mac_stats.tx_frames_256= _511: 7887126=0A> dev.igb.1.mac_stats.tx_frames_512_1023: 10130891=0A> dev.= igb.1.mac_stats.tx_frames_1024_1522: 350851673=0A> dev.igb.1.interrupts.ass= erts: 551135045=0A> dev.igb.1.interrupts.rx_pkt_timer: 384326679=0A> dev.ig= b.1.interrupts.tx_queue_empty: 485323376=0A> dev.igb.1.interrupts.tx_queue_= min_thresh: 6324386=0A> dev.igb.1.host.rx_pkt: 4424=0A> dev.igb.1.host.tx_g= ood_pkt: 4043=0A> dev.igb.1.host.rx_good_bytes: 74284030864=0A> dev.igb.1.h= ost.tx_good_bytes: 544536708502=0A> dev.igb.2.%desc: Intel(R) PRO/1000 Netw= ork Connection=0A> version - 2.3.1=0A> dev.igb.2.%driver: igb=0A> dev.igb.2= .%location: slot=3D0 function=3D2=0A> dev.igb.2.%pnpinfo: vendor=3D0x8086 d= evice=3D0x1521=0A> subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000= =0A> dev.igb.2.%parent: pci5=0A> dev.igb.2.nvm: -1=0A> dev.igb.2.enable_aim= : 1=0A> dev.igb.2.fc: 3=0A> dev.igb.2.rx_processing_limit: 4096=0A> dev.igb= .2.eee_disabled: 1=0A> dev.igb.2.link_irq: 2=0A> dev.igb.2.device_control: = 1209795137=0A> dev.igb.2.rx_control: 67141658=0A> dev.igb.2.interrupt_mask:= 4=0A> dev.igb.2.extended_int_mask: 2147483959=0A> dev.igb.2.fc_high_water:= 33168=0A> dev.igb.2.fc_low_water: 33152=0A> dev.igb.2.queue0.interrupt_rat= e: 13698=0A> dev.igb.2.queue0.txd_head: 1618=0A> dev.igb.2.queue0.txd_tail:= 1618=0A> dev.igb.2.queue0.tx_packets: 46401106=0A> dev.igb.2.queue0.rxd_he= ad: 831=0A> dev.igb.2.queue0.rxd_tail: 827=0A> dev.igb.2.queue0.rx_packets:= 69356350=0A> dev.igb.2.queue0.rx_bytes: 68488772907=0A> dev.igb.2.queue1.i= nterrupt_rate: 5405=0A> dev.igb.2.queue1.txd_head: 190=0A> dev.igb.2.queue1= .txd_tail: 190=0A> dev.igb.2.queue1.tx_packets: 55965886=0A> dev.igb.2.queu= e1.rxd_head: 268=0A> dev.igb.2.queue1.rxd_tail: 256=0A> dev.igb.2.queue1.rx= _packets: 58958084=0A> dev.igb.2.queue1.rx_bytes: 69154569937=0A> dev.igb.2= .queue2.interrupt_rate: 83333=0A> dev.igb.2.queue2.txd_head: 568=0A> dev.ig= b.2.queue2.txd_tail: 568=0A> dev.igb.2.queue2.tx_packets: 44974648=0A> dev.= igb.2.queue2.rxd_head: 371=0A> dev.igb.2.queue2.rxd_tail: 219=0A> dev.igb.2= .queue2.rx_packets: 67037407=0A> dev.igb.2.queue2.rx_bytes: 72042326102=0A>= dev.igb.2.queue3.interrupt_rate: 12658=0A> dev.igb.2.queue3.txd_head: 867= =0A> dev.igb.2.queue3.txd_tail: 867=0A> dev.igb.2.queue3.tx_packets: 559624= 67=0A> dev.igb.2.queue3.rxd_head: 85=0A> dev.igb.2.queue3.rxd_tail: 1953=0A= > dev.igb.2.queue3.rx_packets: 60972965=0A> dev.igb.2.queue3.rx_bytes: 7039= 7176035=0A> dev.igb.2.queue4.interrupt_rate: 90909=0A> dev.igb.2.queue4.txd= _head: 1920=0A> dev.igb.2.queue4.txd_tail: 1920=0A> dev.igb.2.queue4.tx_pac= kets: 47660931=0A> dev.igb.2.queue4.rxd_head: 1397=0A> dev.igb.2.queue4.rxd= _tail: 1379=0A> dev.igb.2.queue4.rx_packets: 59110758=0A> dev.igb.2.queue4.= rx_bytes: 68919201478=0A> dev.igb.2.queue5.interrupt_rate: 111111=0A> dev.i= gb.2.queue5.txd_head: 886=0A> dev.igb.2.queue5.txd_tail: 886=0A> dev.igb.2.= queue5.tx_packets: 45103990=0A> dev.igb.2.queue5.rxd_head: 812=0A> dev.igb.= 2.queue5.rxd_tail: 799=0A> dev.igb.2.queue5.rx_packets: 59030312=0A> dev.ig= b.2.queue5.rx_bytes: 69234293962=0A> dev.igb.2.queue6.interrupt_rate: 5208= =0A> dev.igb.2.queue6.txd_head: 1926=0A> dev.igb.2.queue6.txd_tail: 1926=0A= > dev.igb.2.queue6.tx_packets: 46215046=0A> dev.igb.2.queue6.rxd_head: 692= =0A> dev.igb.2.queue6.rxd_tail: 689=0A> dev.igb.2.queue6.rx_packets: 582560= 50=0A> dev.igb.2.queue6.rx_bytes: 68429172749=0A> dev.igb.2.queue7.interrup= t_rate: 50000=0A> dev.igb.2.queue7.txd_head: 126=0A> dev.igb.2.queue7.txd_t= ail: 126=0A> dev.igb.2.queue7.tx_packets: 52451455=0A> dev.igb.2.queue7.rxd= _head: 968=0A> dev.igb.2.queue7.rxd_tail: 885=0A> dev.igb.2.queue7.rx_packe= ts: 65946491=0A> dev.igb.2.queue7.rx_bytes: 70263478849=0A> dev.igb.2.mac_s= tats.missed_packets: 958=0A> dev.igb.2.mac_stats.recv_no_buff: 69=0A> dev.i= gb.2.mac_stats.total_pkts_recvd: 498658079=0A> dev.igb.2.mac_stats.good_pkt= s_recvd: 498657121=0A> dev.igb.2.mac_stats.bcast_pkts_recvd: 16867=0A> dev.= igb.2.mac_stats.mcast_pkts_recvd: 52957=0A> dev.igb.2.mac_stats.rx_frames_6= 4: 59089332=0A> dev.igb.2.mac_stats.rx_frames_65_127: 52880118=0A> dev.igb.= 2.mac_stats.rx_frames_128_255: 7526966=0A> dev.igb.2.mac_stats.rx_frames_25= 6_511: 8468389=0A> dev.igb.2.mac_stats.rx_frames_512_1023: 10434770=0A> dev= .igb.2.mac_stats.rx_frames_1024_1522: 360257545=0A> dev.igb.2.mac_stats.goo= d_octets_recvd: 558910494322=0A> dev.igb.2.mac_stats.good_octets_txd: 84618= 858153=0A> dev.igb.2.mac_stats.total_pkts_txd: 394726904=0A> dev.igb.2.mac_= stats.good_pkts_txd: 394726904=0A> dev.igb.2.mac_stats.bcast_pkts_txd: 48= =0A> dev.igb.2.mac_stats.mcast_pkts_txd: 52821=0A> dev.igb.2.mac_stats.tx_f= rames_64: 196605932=0A> dev.igb.2.mac_stats.tx_frames_65_127: 134602807=0A>= dev.igb.2.mac_stats.tx_frames_128_255: 5705236=0A> dev.igb.2.mac_stats.tx_= frames_256_511: 10267168=0A> dev.igb.2.mac_stats.tx_frames_512_1023: 171654= 96=0A> dev.igb.2.mac_stats.tx_frames_1024_1522: 30380265=0A> dev.igb.2.inte= rrupts.asserts: 465994260=0A> dev.igb.2.interrupts.rx_pkt_timer: 498647546= =0A> dev.igb.2.interrupts.tx_queue_empty: 394720657=0A> dev.igb.2.interrupt= s.tx_queue_min_thresh: 24424555=0A> dev.igb.2.host.rx_pkt: 9575=0A> dev.igb= .2.host.tx_good_pkt: 6248=0A> dev.igb.2.host.rx_good_bytes: 558910513984=0A= > dev.igb.2.host.tx_good_bytes: 84618858217=0A> =0A> =0A> Thanks for your h= elp.=0A> =0A> Cheers,=0A=0AYou're experiencing lock contention=0A=0ATry edi= ting igb.c and setting the queues to 1. The multiqueue=0Aimplementation in = igb has a negative impact.=0A=0AIf you have just 1 system get a dual port 8= 2571 card and use the=0Aem driver.=0A=0ABC From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 15:57:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2E849C80; Sun, 28 Apr 2013 15:57:01 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-vb0-x235.google.com (mail-vb0-x235.google.com [IPv6:2607:f8b0:400c:c02::235]) by mx1.freebsd.org (Postfix) with ESMTP id C11421ACD; Sun, 28 Apr 2013 15:57:00 +0000 (UTC) Received: by mail-vb0-f53.google.com with SMTP id i3so1655947vbh.40 for ; Sun, 28 Apr 2013 08:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=rDO0tYAXh6pa1Wt7Z2xGQuFRbB2ryhJZX61qvZJP1J8=; b=wJH5TWUeUVfcg8eBWYdadYxZ2FAiR7yedDADvSeau6Nlqy/nSeHHGrCCL0n1zaiPen urtDd8VflTMpa5xy0fW/wHPZYZEuz3Fk0kCnFlMc6HzdG+9tlNRj4CXmcZf/tlffn6X7 8klHAktgjOtmYVbc8jSS5mEFFjHuyXh4xCll1UPuye1iTDX4UHZwaLCTY0BpaWzIJ1ge RbvcT5BiV0DzknrkIUPH98eFrM6OYI+Yhf+usdeoTwfKhywzb8g6Cgg5XLXArOFtm/ao tPfdYZlDCb2WjlbMZuhRCS7jaxzo/bEzpxiwqqF7QFimJld04RbbWoPMk+Yj7IqvnCZT ny1w== X-Received: by 10.220.65.1 with SMTP id g1mr32045942vci.63.1367164620234; Sun, 28 Apr 2013 08:57:00 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Sun, 28 Apr 2013 08:56:39 -0700 (PDT) In-Reply-To: <5178F72F.90008@freebsd.org> References: <5178F72F.90008@freebsd.org> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Sun, 28 Apr 2013 17:56:39 +0200 X-Google-Sender-Auth: 1aZvnKSNL-X2Uy4y6x3LNPkXQXU Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 15:57:01 -0000 On Thu, Apr 25, 2013 at 11:28 AM, Andre Oppermann wrote: > > > Again one has to be really careful drawing any firm conclusions from this > as it was measured on a Pentium4 and UP kernel (GENERIC would add WITNESS > and INVARIANT overhead as well). > > The Pentium4 is about the worst micro-architecture when it comes to locks > and easily regresses. At the same time modern Intel Core i[3-7] and AMD64 > may actually improve with these changes. Unless more recent micro-archs > have been shown to exhibit the same regression we can't claim this change > was bad (other than for Pentium4). OK, here are the results of the same bench on another server (HP ProLiant DL320 G5): - Dual Core: Intel Xeon CPU 3050 2.13GHz (2133.45-MHz K8-class CPU) - NIC changed to dual 82571EB Graph: http://gugus69.free.fr/freebsd/benchs/current2/current-pps.png gnuplot data: http://gugus69.free.fr/freebsd/benchs/current2/plot/ ministat data: http://gugus69.free.fr/freebsd/benchs/current2/ministat/ raw data: http://gugus69.free.fr/freebsd/benchs/current2/raw/ Notice the Glebius' explanation regarding a unique one-flow test and the new pf-smp behavior: http://lists.freebsd.org/pipermail/freebsd-net/2013-April/035417.html Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 15:58:59 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6F44BE28 for ; Sun, 28 Apr 2013 15:58:59 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4847C1AFD for ; Sun, 28 Apr 2013 15:58:58 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id rl6so82266pac.17 for ; Sun, 28 Apr 2013 08:58:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=rrcfrJiKq5dEf9gPCgoKoNUPNdBYhLCG5fd7+6QgkK8=; b=Stu8WNjOlco1alo2LWE4O7qo6LAXY+cmDlV6kGShMf1Xjx+4FB8ZHEFR7YovoEL5tG IT5zXTSbCjOOega/YVgCGfcSv7HykQrVDE0aAiGL16tpr7snjqlVVDbRp92Be9TLaxzT mUf10Zy2GocopQAHGAQv6BAolYs1TOKdwlqgKKjRC2yfiKIElJgW7mdFGVFRU0FjvFl/ 8nNOkVZ8zWSZRyetQKQvEQf+dbwZBK8X9a7Oot0AXgeaWrca08Ku+Z+yJV/Jx+xHxb9Y 5MvicoNeYPzcWuLF6+7l7rlnY0r2NXU6WmVVYLBc4PcOFGcUKC/dT1bY/IWAJ/iMoa6P ya4Q== MIME-Version: 1.0 X-Received: by 10.66.159.234 with SMTP id xf10mr39376957pab.203.1367164738523; Sun, 28 Apr 2013 08:58:58 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Sun, 28 Apr 2013 08:58:58 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Sun, 28 Apr 2013 08:58:58 -0700 (PDT) In-Reply-To: <20130426082457.GA33737@onelab2.iet.unipi.it> References: <20130426082457.GA33737@onelab2.iet.unipi.it> Date: Sun, 28 Apr 2013 18:58:58 +0300 Message-ID: Subject: Re: using netmap From: Sami Halabi To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Andreas Nilsson , Eitan Adler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 15:58:59 -0000 Thank you. I've started using pkt-gen in virtualbox, 2 machines singlwe core-i7 3612QM core, with em cards internal. I see that the sender send about 950 kpps but the receiver is far less (around 200kpps max). Any suggestions why this? Sami On Apr 26, 2013 11:23 AM, "Luigi Rizzo" wrote: > On Fri, Apr 26, 2013 at 09:23:35AM +0300, Sami Halabi wrote: > > Hi Eitan, > > Thank your for your response. > > the ioctl is the example was in Luigi netmap page... maybe Luigi can help > > here??????? > > the thing i suggest is take the pkt-gen source from the FreeBSD tree > > tools/tools/netmap/ > > and start modifying that one. The samples on my web page are > 2 years old, and probably wrong in various ways even then. > They were meant to be something to put on a slide, so a lot of > details were missing. pkt-gen.c, on the other hand, is supposed > to build and be usable. > > As others pointed out, the snippets of code you posted have a lot > of very basic programming errors which suggest that the problems > you are seeing are not related to netmap. > > Of course we may be wrong, but the odds are against you, > so it is better to start from something known-working. > > Try to build your code with "cc -O2 -Wall -Werror" so the > compiler will find at least the most egregious bugs > and abort the compilation until you fix them. > > cheers > luigi > > > can you say why the print's are wrong? > > > > > > i fetched wrking headers from other tools without too much checking, > maybe > > some > > are irrelevant but for my tests i didn't worry about it yet, so here is > it: > > =================================================== > > #include > > #include /* signal */ > > #include > > #include > > #include /* PRI* macros */ > > #include /* strcmp */ > > #include /* open */ > > #include /* close */ > > #include /* getifaddrs */ > > > > #include /* PROT_* */ > > #include /* ioctl */ > > #include > > #include /* sockaddr.. */ > > #include /* ntohs */ > > #include > > #include /* sysctl */ > > #include /* timersub */ > > > > #include > > #include /* ifreq */ > > > > #include > > #include > > #include > > > > #include > > #include > > > > #ifndef MY_PCAP /* use the system's pcap if available */ > > > > #ifdef NO_PCAP > > #define PCAP_ERRBUF_SIZE 512 > > typedef void pcap_t; > > struct pcap_pkthdr; > > #define pcap_inject(a,b,c) ((void)a, (void)b, (void)c, -1) > > #define pcap_dispatch(a, b, c, d) (void)c > > #define pcap_open_live(a, b, c, d, e) ((void)e, NULL) > > #else /* !NO_PCAP */ > > #include // XXX do we need it ? > > #endif /* !NO_PCAP */ > > > > #endif // XXX hack > > > > #include /* pthread_* */ > > #include /* le64toh */ > > #include > > > > #include /* pthread w/ affinity */ > > #include /* cpu_set */ > > #include /* LLADDR */ > > =================================================== > > > > > > Thanks in advance, > > Sami > > > > > > On Fri, Apr 26, 2013 at 6:43 AM, Eitan Adler > wrote: > > > > > [ please bottom post or reply inline ] > > > > > > On 25 April 2013 17:48, Sami Halabi wrote: > > > > Okay, > > > > i figured out the includes, now it runs and seg faults: > > > > > > Don't forget to show the working headers ;) > > > > > > > any ideas? > > > > > > > > here is the new code: > > > > int main() { > > > > > > > > struct netmap_if *nifp = NULL; > > > > struct nmreq req; > > > > int i, len, fd; > > > > char *buf, *mem, *txt; > > > > > > > > printf("Starting...\n"); > > > > fd = open("/dev/netmap", 0); > > > > strcpy(req.nr_name, "em0"); // register the interface > > > > printf("em0 registered...\n"); > > > > ioctl(fd, NIOCREGIF, &req); // offset of the structure > > > > > > is req fully initialized? > > > > > > I don't think this ioctl is correct. Everything goes wrong after this > > > as a result. > > > > > > > printf("sysctl passed\n"); > > > > > > Things will not always crash if you make a wrong call. Try checking > > > return codes before printing something like this. > > > > > > > mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); > > > > printf("mem mapped...\n"); > > > > > > > > nifp = NETMAP_IF(mem, req.nr_offset); > > > > printf("nifp mapped...%u\n",(long)nifp); > > > > > > this seems wrong: ^^ ^^^^^^ > > > > > > > > > > for (;;) { > > > > struct pollfd x[1]; > > > > printf("rx ring def...\n"); > > > > struct netmap_ring *ring; > > > > printf("rx ring start...\n"); > > > > ring = NETMAP_RXRING(nifp, 0); > > > > printf("rx ring polled...\n"); > > > > > > > > x[0].fd = fd; > > > > x[0].events = POLLIN; > > > > poll(x, 1, 1000); > > > > for ( ; ring->avail > 0 ; ring->avail--) { > > > > i = ring->cur; > > > > printf("i=%d\n",&i); > > > > > > I think this is wrong. > > > > > > > buf = NETMAP_BUF(ring, i); > > > > printf("buff read...\n"); > > > > //use_data(buf, ring->slot[i].len); > > > > txt = malloc(sizeof(char) * ring->slot[i].len +1); > > > > strncpy(txt,buf,ring->slot[i].len); > > > > txt[ring->slot[i].len]='\0'; > > > > printf("len is: %d\n",&ring->slot[i].len); > > > > > > Also this. > > > > > > > ring->cur = NETMAP_RING_NEXT(ring, i); > > > > } > > > > } > > > > > > > > return 0; > > > > } > > > > > > > > > > > > -- > > > Eitan Adler > > > > > > > > > > > -- > > Sami Halabi > > Information Systems Engineer > > NMS Projects Expert > > FreeBSD SysAdmin Expert > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 17:07:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 782B9266 for ; Sun, 28 Apr 2013 17:07:06 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vb0-x22f.google.com (mail-vb0-x22f.google.com [IPv6:2607:f8b0:400c:c02::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 390581D76 for ; Sun, 28 Apr 2013 17:07:06 +0000 (UTC) Received: by mail-vb0-f47.google.com with SMTP id x14so107289vbb.34 for ; Sun, 28 Apr 2013 10:07:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=JfwfK+AWm0zI9FM21uPJ0iVl9M+jJtLTrnZT459rbys=; b=G1HU1rpOgCzmTtilVHgGxHXwuokSR9eavN+8vsH038NY/c9kCnL2ogaL5pRXX4O+7m Mg298WT3xBoFmm8uoPG0XW5/hDxKLxEkngNhx+4XDTIDA7Y4Vc/mvq4SOv1+Z3lQRQOm DQ6ZxrkzZbW1r/rReG3kK8fFl4L8rP8Ccvll3eG1vC5nbRZdfoY2/xYnGCdkoO6JZr9/ qSIibao2wfb/RiAbkoWA73i1B3DywFkV/08FmRzFZfWvTKGOmZokld/lwhgsLKys1GgP CGmy2WwAYxoEdbLPCyhciq8EiOnozrl1M9Vmeyjk0ci8lrpchGpr95yzlZOG202QhAwt mcow== MIME-Version: 1.0 X-Received: by 10.52.22.202 with SMTP id g10mr26913858vdf.126.1367168825511; Sun, 28 Apr 2013 10:07:05 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Sun, 28 Apr 2013 10:07:05 -0700 (PDT) In-Reply-To: <1367158902.96158.YahooMailClassic@web121603.mail.ne1.yahoo.com> References: <517A657B.7060003@gmail.com> <1367158902.96158.YahooMailClassic@web121603.mail.ne1.yahoo.com> Date: Sun, 28 Apr 2013 10:07:05 -0700 Message-ID: Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 From: Jack Vogel To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , =?ISO-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 17:07:06 -0000 Try setting your queues to 1, run some tests, then try setting your queues to 2, then to 4... its called tuning, and rather than just pontificating about it, which Barney so loves to do, you can discover what works best. I ran tests last week preparing for a new driver version and found the best results came not only while tweaking queues, but also ring size, and I could see changes based on the buf ring size.... There are lots of things that may improve or degrade performance depending on the workload. Jack On Sun, Apr 28, 2013 at 7:21 AM, Barney Cordoba w= rote: > > > --- On Fri, 4/26/13, "Cl=E9ment Hermann (nodens)" > wrote: > > > From: "Cl=E9ment Hermann (nodens)" > > Subject: High CPU interrupt load on intel I350T4 with igb on 8.3 > > To: freebsd-net@freebsd.org > > Date: Friday, April 26, 2013, 7:31 AM > > Hi list, > > > > We use pf+ALTQ for trafic shaping on some routers. > > > > We are switching to new servers : Dell PowerEdge R620 with 2 > > 8-cores Intel Processor (E5-2650L), 8GB RAM and Intel I350T4 > > (quad port) using igb driver. The old hardware is using em > > driver, the CPU load is high but mostly due to kernel and a > > large pf ruleset. > > > > On the new hardware, we see high CPU Interrupt load (up to > > 95%), even though there is not much trafic currently (peaks > > about 150Mbps and 40Kpps). All queues are used and binded to > > a cpu according to top, but a lot of CPU time is spent on > > igb queues (interrupt or wait). The load is fine when we > > stay below 20Kpps. > > > > We see no mbuf shortage, no dropped packet, but there is > > little margin left on CPU time (about 25% idle at best, most > > of CPU time is spent on interrupts), which is disturbing. > > > > We have done some tuning, but to no avail : > > > > sysctl.conf : > > > > # mbufs > > kern.ipc.nmbclusters=3D65536 > > # Sockets > > kern.ipc.somaxconn=3D8192 > > net.inet.tcp.delayed_ack=3D0 > > net.inet.tcp.sendspace=3D65535 > > net.inet.udp.recvspace=3D65535 > > net.inet.udp.maxdgram=3D57344 > > net.local.stream.recvspace=3D65535 > > net.local.stream.sendspace=3D65535 > > # IGB > > dev.igb.0.rx_processing_limit=3D4096 > > dev.igb.1.rx_processing_limit=3D4096 > > dev.igb.2.rx_processing_limit=3D4096 > > dev.igb.3.rx_processing_limit=3D4096 > > > > /boot/loader.conf : > > > > vm.kmem_size=3D1G > > hw.igb.max_interrupt_rate=3D"32000" # maximum number of > > interrupts/sec generated by single igb(4) (default 8000) > > hw.igb.txd=3D"2048" > > > > # number of transmit descriptors allocated by the > > driver (2048 limit) > > hw.igb.rxd=3D"2048" > > > > # number of receive descriptors allocated by the > > driver (2048 limit) > > hw.igb.rx_process_limit=3D"1000" # > > maximum number of received packets to process at a time, The > > default of 100 is > > > > > > > > > > # too low for most firewalls. (-1 means > > unlimited) > > > > Kernel HZ is 1000. > > > > The IGB /boot/loader.conf tuning was our last attempt, it > > didn't change anything. > > > > Does anyone have any pointer ? How could we lower CPU > > interrupt load ? should we set hw.igb.max_interrupt_rate > > lower instead of higher ? > > From what we saw here and there, we should be able to do > > much better with this hardware. > > > > > > relevant sysctl (igb1 and igb2 only, other interfaces are > > unused) : > > > > sysctl dev.igb | grep -v ": 0$" > > dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection > > version - 2.3.1 > > dev.igb.1.%driver: igb > > dev.igb.1.%location: slot=3D0 function=3D1 > > dev.igb.1.%pnpinfo: vendor=3D0x8086 device=3D0x1521 > > subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 > > dev.igb.1.%parent: pci5 > > dev.igb.1.nvm: -1 > > dev.igb.1.enable_aim: 1 > > dev.igb.1.fc: 3 > > dev.igb.1.rx_processing_limit: 4096 > > dev.igb.1.eee_disabled: 1 > > dev.igb.1.link_irq: 2 > > dev.igb.1.device_control: 1209795137 > > dev.igb.1.rx_control: 67141658 > > dev.igb.1.interrupt_mask: 4 > > dev.igb.1.extended_int_mask: 2147483981 > > dev.igb.1.fc_high_water: 33168 > > dev.igb.1.fc_low_water: 33152 > > dev.igb.1.queue0.interrupt_rate: 71428 > > dev.igb.1.queue0.txd_head: 1318 > > dev.igb.1.queue0.txd_tail: 1318 > > dev.igb.1.queue0.tx_packets: 84663594 > > dev.igb.1.queue0.rxd_head: 717 > > dev.igb.1.queue0.rxd_tail: 715 > > dev.igb.1.queue0.rx_packets: 43899597 > > dev.igb.1.queue0.rx_bytes: 8905556030 > > dev.igb.1.queue1.interrupt_rate: 90909 > > dev.igb.1.queue1.txd_head: 693 > > dev.igb.1.queue1.txd_tail: 693 > > dev.igb.1.queue1.tx_packets: 57543349 > > dev.igb.1.queue1.rxd_head: 1033 > > dev.igb.1.queue1.rxd_tail: 1032 > > dev.igb.1.queue1.rx_packets: 54821897 > > dev.igb.1.queue1.rx_bytes: 9944955108 > > dev.igb.1.queue2.interrupt_rate: 100000 > > dev.igb.1.queue2.txd_head: 350 > > dev.igb.1.queue2.txd_tail: 350 > > dev.igb.1.queue2.tx_packets: 62320990 > > dev.igb.1.queue2.rxd_head: 1962 > > dev.igb.1.queue2.rxd_tail: 1939 > > dev.igb.1.queue2.rx_packets: 43909016 > > dev.igb.1.queue2.rx_bytes: 8673941461 > > dev.igb.1.queue3.interrupt_rate: 14925 > > dev.igb.1.queue3.txd_head: 647 > > dev.igb.1.queue3.txd_tail: 647 > > dev.igb.1.queue3.tx_packets: 58776199 > > dev.igb.1.queue3.rxd_head: 692 > > dev.igb.1.queue3.rxd_tail: 691 > > dev.igb.1.queue3.rx_packets: 55138996 > > dev.igb.1.queue3.rx_bytes: 9310217354 > > dev.igb.1.queue4.interrupt_rate: 100000 > > dev.igb.1.queue4.txd_head: 1721 > > dev.igb.1.queue4.txd_tail: 1721 > > dev.igb.1.queue4.tx_packets: 54337209 > > dev.igb.1.queue4.rxd_head: 1609 > > dev.igb.1.queue4.rxd_tail: 1598 > > dev.igb.1.queue4.rx_packets: 46546503 > > dev.igb.1.queue4.rx_bytes: 8818182840 > > dev.igb.1.queue5.interrupt_rate: 11627 > > dev.igb.1.queue5.txd_head: 254 > > dev.igb.1.queue5.txd_tail: 254 > > dev.igb.1.queue5.tx_packets: 53117182 > > dev.igb.1.queue5.rxd_head: 701 > > dev.igb.1.queue5.rxd_tail: 685 > > dev.igb.1.queue5.rx_packets: 43014837 > > dev.igb.1.queue5.rx_bytes: 8699057447 > > dev.igb.1.queue6.interrupt_rate: 55555 > > dev.igb.1.queue6.txd_head: 8 > > dev.igb.1.queue6.txd_tail: 8 > > dev.igb.1.queue6.tx_packets: 52654088 > > dev.igb.1.queue6.rxd_head: 1057 > > dev.igb.1.queue6.rxd_tail: 1041 > > dev.igb.1.queue6.rx_packets: 45227030 > > dev.igb.1.queue6.rx_bytes: 9494489640 > > dev.igb.1.queue7.interrupt_rate: 5235 > > dev.igb.1.queue7.txd_head: 729 > > dev.igb.1.queue7.txd_tail: 729 > > dev.igb.1.queue7.tx_packets: 61926105 > > dev.igb.1.queue7.rxd_head: 146 > > dev.igb.1.queue7.rxd_tail: 140 > > dev.igb.1.queue7.rx_packets: 51781775 > > dev.igb.1.queue7.rx_bytes: 8901279226 > > dev.igb.1.mac_stats.missed_packets: 1657 > > dev.igb.1.mac_stats.recv_no_buff: 405 > > dev.igb.1.mac_stats.total_pkts_recvd: 384332760 > > dev.igb.1.mac_stats.good_pkts_recvd: 384331103 > > dev.igb.1.mac_stats.bcast_pkts_recvd: 15510 > > dev.igb.1.mac_stats.mcast_pkts_recvd: 52957 > > dev.igb.1.mac_stats.rx_frames_64: 195496498 > > dev.igb.1.mac_stats.rx_frames_65_127: 133346124 > > dev.igb.1.mac_stats.rx_frames_128_255: 5254911 > > dev.igb.1.mac_stats.rx_frames_256_511: 9700049 > > dev.igb.1.mac_stats.rx_frames_512_1023: 16885886 > > dev.igb.1.mac_stats.rx_frames_1024_1522: 23647635 > > dev.igb.1.mac_stats.good_octets_recvd: 74284029276 > > dev.igb.1.mac_stats.good_octets_txd: 544536708502 > > dev.igb.1.mac_stats.total_pkts_txd: 485327419 > > dev.igb.1.mac_stats.good_pkts_txd: 485327419 > > dev.igb.1.mac_stats.bcast_pkts_txd: 72 > > dev.igb.1.mac_stats.mcast_pkts_txd: 52820 > > dev.igb.1.mac_stats.tx_frames_64: 57820809 > > dev.igb.1.mac_stats.tx_frames_65_127: 51586341 > > dev.igb.1.mac_stats.tx_frames_128_255: 7050579 > > dev.igb.1.mac_stats.tx_frames_256_511: 7887126 > > dev.igb.1.mac_stats.tx_frames_512_1023: 10130891 > > dev.igb.1.mac_stats.tx_frames_1024_1522: 350851673 > > dev.igb.1.interrupts.asserts: 551135045 > > dev.igb.1.interrupts.rx_pkt_timer: 384326679 > > dev.igb.1.interrupts.tx_queue_empty: 485323376 > > dev.igb.1.interrupts.tx_queue_min_thresh: 6324386 > > dev.igb.1.host.rx_pkt: 4424 > > dev.igb.1.host.tx_good_pkt: 4043 > > dev.igb.1.host.rx_good_bytes: 74284030864 > > dev.igb.1.host.tx_good_bytes: 544536708502 > > dev.igb.2.%desc: Intel(R) PRO/1000 Network Connection > > version - 2.3.1 > > dev.igb.2.%driver: igb > > dev.igb.2.%location: slot=3D0 function=3D2 > > dev.igb.2.%pnpinfo: vendor=3D0x8086 device=3D0x1521 > > subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 > > dev.igb.2.%parent: pci5 > > dev.igb.2.nvm: -1 > > dev.igb.2.enable_aim: 1 > > dev.igb.2.fc: 3 > > dev.igb.2.rx_processing_limit: 4096 > > dev.igb.2.eee_disabled: 1 > > dev.igb.2.link_irq: 2 > > dev.igb.2.device_control: 1209795137 > > dev.igb.2.rx_control: 67141658 > > dev.igb.2.interrupt_mask: 4 > > dev.igb.2.extended_int_mask: 2147483959 > > dev.igb.2.fc_high_water: 33168 > > dev.igb.2.fc_low_water: 33152 > > dev.igb.2.queue0.interrupt_rate: 13698 > > dev.igb.2.queue0.txd_head: 1618 > > dev.igb.2.queue0.txd_tail: 1618 > > dev.igb.2.queue0.tx_packets: 46401106 > > dev.igb.2.queue0.rxd_head: 831 > > dev.igb.2.queue0.rxd_tail: 827 > > dev.igb.2.queue0.rx_packets: 69356350 > > dev.igb.2.queue0.rx_bytes: 68488772907 > > dev.igb.2.queue1.interrupt_rate: 5405 > > dev.igb.2.queue1.txd_head: 190 > > dev.igb.2.queue1.txd_tail: 190 > > dev.igb.2.queue1.tx_packets: 55965886 > > dev.igb.2.queue1.rxd_head: 268 > > dev.igb.2.queue1.rxd_tail: 256 > > dev.igb.2.queue1.rx_packets: 58958084 > > dev.igb.2.queue1.rx_bytes: 69154569937 > > dev.igb.2.queue2.interrupt_rate: 83333 > > dev.igb.2.queue2.txd_head: 568 > > dev.igb.2.queue2.txd_tail: 568 > > dev.igb.2.queue2.tx_packets: 44974648 > > dev.igb.2.queue2.rxd_head: 371 > > dev.igb.2.queue2.rxd_tail: 219 > > dev.igb.2.queue2.rx_packets: 67037407 > > dev.igb.2.queue2.rx_bytes: 72042326102 > > dev.igb.2.queue3.interrupt_rate: 12658 > > dev.igb.2.queue3.txd_head: 867 > > dev.igb.2.queue3.txd_tail: 867 > > dev.igb.2.queue3.tx_packets: 55962467 > > dev.igb.2.queue3.rxd_head: 85 > > dev.igb.2.queue3.rxd_tail: 1953 > > dev.igb.2.queue3.rx_packets: 60972965 > > dev.igb.2.queue3.rx_bytes: 70397176035 > > dev.igb.2.queue4.interrupt_rate: 90909 > > dev.igb.2.queue4.txd_head: 1920 > > dev.igb.2.queue4.txd_tail: 1920 > > dev.igb.2.queue4.tx_packets: 47660931 > > dev.igb.2.queue4.rxd_head: 1397 > > dev.igb.2.queue4.rxd_tail: 1379 > > dev.igb.2.queue4.rx_packets: 59110758 > > dev.igb.2.queue4.rx_bytes: 68919201478 > > dev.igb.2.queue5.interrupt_rate: 111111 > > dev.igb.2.queue5.txd_head: 886 > > dev.igb.2.queue5.txd_tail: 886 > > dev.igb.2.queue5.tx_packets: 45103990 > > dev.igb.2.queue5.rxd_head: 812 > > dev.igb.2.queue5.rxd_tail: 799 > > dev.igb.2.queue5.rx_packets: 59030312 > > dev.igb.2.queue5.rx_bytes: 69234293962 > > dev.igb.2.queue6.interrupt_rate: 5208 > > dev.igb.2.queue6.txd_head: 1926 > > dev.igb.2.queue6.txd_tail: 1926 > > dev.igb.2.queue6.tx_packets: 46215046 > > dev.igb.2.queue6.rxd_head: 692 > > dev.igb.2.queue6.rxd_tail: 689 > > dev.igb.2.queue6.rx_packets: 58256050 > > dev.igb.2.queue6.rx_bytes: 68429172749 > > dev.igb.2.queue7.interrupt_rate: 50000 > > dev.igb.2.queue7.txd_head: 126 > > dev.igb.2.queue7.txd_tail: 126 > > dev.igb.2.queue7.tx_packets: 52451455 > > dev.igb.2.queue7.rxd_head: 968 > > dev.igb.2.queue7.rxd_tail: 885 > > dev.igb.2.queue7.rx_packets: 65946491 > > dev.igb.2.queue7.rx_bytes: 70263478849 > > dev.igb.2.mac_stats.missed_packets: 958 > > dev.igb.2.mac_stats.recv_no_buff: 69 > > dev.igb.2.mac_stats.total_pkts_recvd: 498658079 > > dev.igb.2.mac_stats.good_pkts_recvd: 498657121 > > dev.igb.2.mac_stats.bcast_pkts_recvd: 16867 > > dev.igb.2.mac_stats.mcast_pkts_recvd: 52957 > > dev.igb.2.mac_stats.rx_frames_64: 59089332 > > dev.igb.2.mac_stats.rx_frames_65_127: 52880118 > > dev.igb.2.mac_stats.rx_frames_128_255: 7526966 > > dev.igb.2.mac_stats.rx_frames_256_511: 8468389 > > dev.igb.2.mac_stats.rx_frames_512_1023: 10434770 > > dev.igb.2.mac_stats.rx_frames_1024_1522: 360257545 > > dev.igb.2.mac_stats.good_octets_recvd: 558910494322 > > dev.igb.2.mac_stats.good_octets_txd: 84618858153 > > dev.igb.2.mac_stats.total_pkts_txd: 394726904 > > dev.igb.2.mac_stats.good_pkts_txd: 394726904 > > dev.igb.2.mac_stats.bcast_pkts_txd: 48 > > dev.igb.2.mac_stats.mcast_pkts_txd: 52821 > > dev.igb.2.mac_stats.tx_frames_64: 196605932 > > dev.igb.2.mac_stats.tx_frames_65_127: 134602807 > > dev.igb.2.mac_stats.tx_frames_128_255: 5705236 > > dev.igb.2.mac_stats.tx_frames_256_511: 10267168 > > dev.igb.2.mac_stats.tx_frames_512_1023: 17165496 > > dev.igb.2.mac_stats.tx_frames_1024_1522: 30380265 > > dev.igb.2.interrupts.asserts: 465994260 > > dev.igb.2.interrupts.rx_pkt_timer: 498647546 > > dev.igb.2.interrupts.tx_queue_empty: 394720657 > > dev.igb.2.interrupts.tx_queue_min_thresh: 24424555 > > dev.igb.2.host.rx_pkt: 9575 > > dev.igb.2.host.tx_good_pkt: 6248 > > dev.igb.2.host.rx_good_bytes: 558910513984 > > dev.igb.2.host.tx_good_bytes: 84618858217 > > > > > > Thanks for your help. > > > > Cheers, > > You're experiencing lock contention > > Try editing igb.c and setting the queues to 1. The multiqueue > implementation in igb has a negative impact. > > If you have just 1 system get a dual port 82571 card and use the > em driver. > > BC > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 19:01:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C1F07990 for ; Sun, 28 Apr 2013 19:01:53 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm39-vm5.bullet.mail.ne1.yahoo.com (nm39-vm5.bullet.mail.ne1.yahoo.com [98.138.229.165]) by mx1.freebsd.org (Postfix) with ESMTP id 8A249117D for ; Sun, 28 Apr 2013 19:01:53 +0000 (UTC) Received: from [98.138.226.179] by nm39.bullet.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 18:59:34 -0000 Received: from [98.138.226.169] by tm14.bullet.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 18:59:34 -0000 Received: from [127.0.0.1] by omp1070.mail.ne1.yahoo.com with NNFMP; 28 Apr 2013 18:59:34 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 450441.6386.bm@omp1070.mail.ne1.yahoo.com Received: (qmail 98224 invoked by uid 60001); 28 Apr 2013 18:59:34 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1367175574; bh=8rMK3siu5BQynxA9PuIgErXqZsCKQTqOhlcpTbHQdrE=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=A4EekTjO/gbOEtbEpvVJiJIpqXrAHLweULqEM0Sl3bJmzKXOfIr7va7Ywr0EDX6jKLp7uIqfbgqqhA9L2VecNpveRK3wtpIwmSjmC7MgSqQPkaK0nkY3YDlfipc8cERBfS2DfjT1UiIKvvr4SmQaQtn/3DXwdO2p368pEJ8JE+Q= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=XZe+qWAz33KPd7KXCBVkY1bycSxQ/SIhVl+ZhNsRea7gPOvO7Avt4PrFyeyCtED5R9Ecxngh8qzmRF+jO7iNKeVut/wdZsXLZEm44VOCY62UEXQdF8trzEjglLAEedPamS0LEzQ9xgvMShqX7Tc1HQzQdyi4yRNRIo/lnRNvbEs=; X-YMail-OSG: dWL.W.wVM1kFPy1Of9hk53sbc2Y3eqnwexFMjeK61oM8L.p 2wa9r87hqKaNbNA1Ia0LV75e.EscsbU1kInNUCC8pcxVXNiuukXJfJvVKxx6 YzKaoSG_KpuY_odQBSAc.c0XVQEijRxfEEVIWypIjPN1wxleMdb9WoQ1zV2I NBpla5LP17yYcsxsMNHXaw99qg0kwiVPFMKfEPSGlelsN0dGtx0nwWOJG8bx 3kmxRDqfHhR5ekUpWRP5BDqn5WByYUBULAlcpcBzz8MMmUYgsXMLtfmkutMg lZSr0XFwe5psML4HBxt7503TphvHJg4APdGo2ykThoXHXUMICFk4mmbAVe2T y5Yp75Jozhj7yZNh5nHV7iXDHdboh.Y.NLaALEJPORtDvZ2vf6h4_1vXuDSy 6_dGUVjZyAbGh7nNKJ8eUU6tulkpvh7F9t90t.7QtdfwivNKv_jAP1eWSRjk cLjQMjGKIRuc21cshJ37s11nxvXy7iSyIq.SaQhjoXibQ0qPlT8aMAv_4rXp mdDcpGUoeQFqIJJycmN6cOLMNpCgzO745NeM5V2hCubGj.CUtMiWXfgQOK_O JEK0E_b_9TBtPJsCkwIETHy1S Received: from [98.203.118.124] by web121602.mail.ne1.yahoo.com via HTTP; Sun, 28 Apr 2013 11:59:33 PDT X-Rocket-MIMEInfo: 002.001, VGhlIHBvaW50IG9mIGxpc3RzIGlzIHRvIGJlIGFibGUgdG8gYmVuZWZpdCBmcm9tIG90aGVyJ3MgZXhwZXJpZW5jZXMgc28geW91IGRvbid0IGhhdmUgdG8gd2FzdGUgeW91ciB0aW1lICJ0cnlpbmciIHRoaW5ncyB0aGF0IG90aGVycyBoYXZlIGFscmVhZHkgZG9uZS4NCkknbSBub3QgcG9udGlmaWNhdGluZy4gSSd2ZSBkb25lIHRoZSB0ZXN0cy4gVGhlcmUncyBubyByZWFzb24gZm9yIGV2ZXJ5IHBlcnNvbiB3aG8gaXPCoGhhdmluZyB0byBleGFjdCBzYW1lIHByb2JsZW0gdG8gZG8gdGhlIHNhbWUgdGVzdHMBMAEBAQE- X-Mailer: YahooMailClassic/15.1.8 YahooMailWebService/0.8.141.536 Message-ID: <1367175573.97340.YahooMailClassic@web121602.mail.ne1.yahoo.com> Date: Sun, 28 Apr 2013 11:59:33 -0700 (PDT) From: Barney Cordoba Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 To: Jack Vogel In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , =?iso-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 19:01:53 -0000 The point of lists is to be able to benefit from other's experiences so you= don't have to waste your time "trying" things that others have already don= e. I'm not pontificating. I've done the tests. There's no reason for every per= son who is=A0having to exact same problem to do the same tests over and ove= r, hoping for somemagically different result. The result will always be the= same. Because there's no chance=A0of it=A0working properly by chance. BC --- On Sun, 4/28/13, Jack Vogel wrote: From: Jack Vogel Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 To: "Barney Cordoba" Cc: "FreeBSD Net" , "Cl=E9ment Hermann (nodens)" <= nodens2099@gmail.com> Date: Sunday, April 28, 2013, 1:07 PM Try setting your queues to 1, run some tests, then try settingyour queues t= o 2, then to 4... its called tuning, and rather thanjust pontificating abou= t it, which Barney so loves to do, you can=0Adiscover what works best. I ra= n tests last week preparing for anew driver version and found the best resu= lts came not only whiletweaking queues, but also ring size, and I could see= changes based=0Aon the buf ring size.... =A0There are lots of things that = may improve ordegrade performance depending on the workload. Jack =0A On Sun, Apr 28, 2013 at 7:21 AM, Barney Cordoba = wrote: =0A =0A =0A--- On Fri, 4/26/13, "Cl=E9ment Hermann (nodens)" = wrote: =0A =0A> From: "Cl=E9ment Hermann (nodens)" =0A> Subject: High CPU interrupt load on intel I350T4 with igb on 8.3 =0A> To: freebsd-net@freebsd.org =0A> Date: Friday, April 26, 2013, 7:31 AM =0A> Hi list, =0A> =0A> We use pf+ALTQ for trafic shaping on some routers. =0A> =0A> We are switching to new servers : Dell PowerEdge R620 with 2 =0A> 8-cores Intel Processor (E5-2650L), 8GB RAM and Intel I350T4 =0A> (quad port) using igb driver. The old hardware is using em =0A> driver, the CPU load is high but mostly due to kernel and a =0A> large pf ruleset. =0A> =0A> On the new hardware, we see high CPU Interrupt load (up to =0A> 95%), even though there is not much trafic currently (peaks =0A> about 150Mbps and 40Kpps). All queues are used and binded to =0A> a cpu according to top, but a lot of CPU time is spent on =0A> igb queues (interrupt or wait). The load is fine when we =0A> stay below 20Kpps. =0A> =0A> We see no mbuf shortage, no dropped packet, but there is =0A> little margin left on CPU time (about 25% idle at best, most =0A> of CPU time is spent on interrupts), which is disturbing. =0A> =0A> We have done some tuning, but to no avail : =0A> =0A> sysctl.conf : =0A> =0A> # mbufs =0A> kern.ipc.nmbclusters=3D65536 =0A> # Sockets =0A> kern.ipc.somaxconn=3D8192 =0A> net.inet.tcp.delayed_ack=3D0 =0A> net.inet.tcp.sendspace=3D65535 =0A> net.inet.udp.recvspace=3D65535 =0A> net.inet.udp.maxdgram=3D57344 =0A> net.local.stream.recvspace=3D65535 =0A> net.local.stream.sendspace=3D65535 =0A> # IGB =0A> dev.igb.0.rx_processing_limit=3D4096 =0A> dev.igb.1.rx_processing_limit=3D4096 =0A> dev.igb.2.rx_processing_limit=3D4096 =0A> dev.igb.3.rx_processing_limit=3D4096 =0A> =0A> /boot/loader.conf : =0A> =0A> vm.kmem_size=3D1G =0A> hw.igb.max_interrupt_rate=3D"32000"=A0 # maximum number of =0A> interrupts/sec generated by single igb(4) (default 8000) =0A> hw.igb.txd=3D"2048"=A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 # number of transmit descriptors allocated by the =0A> driver (2048 limit) =0A> hw.igb.rxd=3D"2048"=A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 # number of receive descriptors allocated by the =0A> driver (2048 limit) =0A> hw.igb.rx_process_limit=3D"1000"=A0 =A0=A0=A0# =0A> maximum number of received packets to process at a time, The =0A> default of 100 is =0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0 =A0 =A0 =A0 =A0 =A0 =0A> =A0=A0=A0# too low for most firewalls. (-1 means =0A> unlimited) =0A> =0A> Kernel HZ is 1000. =0A> =0A> The IGB /boot/loader.conf tuning was our last attempt, it =0A> didn't change anything. =0A> =0A> Does anyone have any pointer ? How could we lower CPU =0A> interrupt load ? should we set hw.igb.max_interrupt_rate =0A> lower instead of higher ? =0A> From what we saw here and there, we should be able to do =0A> much better with this hardware. =0A> =0A> =0A> relevant sysctl (igb1 and igb2 only, other interfaces are =0A> unused) : =0A> =0A> sysctl dev.igb | grep -v ": 0$" =0A> dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection =0A> version - 2.3.1 =0A> dev.igb.1.%driver: igb =0A> dev.igb.1.%location: slot=3D0 function=3D1 =0A> dev.igb.1.%pnpinfo: vendor=3D0x8086 device=3D0x1521 =0A> subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 =0A> dev.igb.1.%parent: pci5 =0A> dev.igb.1.nvm: -1 =0A> dev.igb.1.enable_aim: 1 =0A> dev.igb.1.fc: 3 =0A> dev.igb.1.rx_processing_limit: 4096 =0A> dev.igb.1.eee_disabled: 1 =0A> dev.igb.1.link_irq: 2 =0A> dev.igb.1.device_control: 1209795137 =0A> dev.igb.1.rx_control: 67141658 =0A> dev.igb.1.interrupt_mask: 4 =0A> dev.igb.1.extended_int_mask: 2147483981 =0A> dev.igb.1.fc_high_water: 33168 =0A> dev.igb.1.fc_low_water: 33152 =0A> dev.igb.1.queue0.interrupt_rate: 71428 =0A> dev.igb.1.queue0.txd_head: 1318 =0A> dev.igb.1.queue0.txd_tail: 1318 =0A> dev.igb.1.queue0.tx_packets: 84663594 =0A> dev.igb.1.queue0.rxd_head: 717 =0A> dev.igb.1.queue0.rxd_tail: 715 =0A> dev.igb.1.queue0.rx_packets: 43899597 =0A> dev.igb.1.queue0.rx_bytes: 8905556030 =0A> dev.igb.1.queue1.interrupt_rate: 90909 =0A> dev.igb.1.queue1.txd_head: 693 =0A> dev.igb.1.queue1.txd_tail: 693 =0A> dev.igb.1.queue1.tx_packets: 57543349 =0A> dev.igb.1.queue1.rxd_head: 1033 =0A> dev.igb.1.queue1.rxd_tail: 1032 =0A> dev.igb.1.queue1.rx_packets: 54821897 =0A> dev.igb.1.queue1.rx_bytes: 9944955108 =0A> dev.igb.1.queue2.interrupt_rate: 100000 =0A> dev.igb.1.queue2.txd_head: 350 =0A> dev.igb.1.queue2.txd_tail: 350 =0A> dev.igb.1.queue2.tx_packets: 62320990 =0A> dev.igb.1.queue2.rxd_head: 1962 =0A> dev.igb.1.queue2.rxd_tail: 1939 =0A> dev.igb.1.queue2.rx_packets: 43909016 =0A> dev.igb.1.queue2.rx_bytes: 8673941461 =0A> dev.igb.1.queue3.interrupt_rate: 14925 =0A> dev.igb.1.queue3.txd_head: 647 =0A> dev.igb.1.queue3.txd_tail: 647 =0A> dev.igb.1.queue3.tx_packets: 58776199 =0A> dev.igb.1.queue3.rxd_head: 692 =0A> dev.igb.1.queue3.rxd_tail: 691 =0A> dev.igb.1.queue3.rx_packets: 55138996 =0A> dev.igb.1.queue3.rx_bytes: 9310217354 =0A> dev.igb.1.queue4.interrupt_rate: 100000 =0A> dev.igb.1.queue4.txd_head: 1721 =0A> dev.igb.1.queue4.txd_tail: 1721 =0A> dev.igb.1.queue4.tx_packets: 54337209 =0A> dev.igb.1.queue4.rxd_head: 1609 =0A> dev.igb.1.queue4.rxd_tail: 1598 =0A> dev.igb.1.queue4.rx_packets: 46546503 =0A> dev.igb.1.queue4.rx_bytes: 8818182840 =0A> dev.igb.1.queue5.interrupt_rate: 11627 =0A> dev.igb.1.queue5.txd_head: 254 =0A> dev.igb.1.queue5.txd_tail: 254 =0A> dev.igb.1.queue5.tx_packets: 53117182 =0A> dev.igb.1.queue5.rxd_head: 701 =0A> dev.igb.1.queue5.rxd_tail: 685 =0A> dev.igb.1.queue5.rx_packets: 43014837 =0A> dev.igb.1.queue5.rx_bytes: 8699057447 =0A> dev.igb.1.queue6.interrupt_rate: 55555 =0A> dev.igb.1.queue6.txd_head: 8 =0A> dev.igb.1.queue6.txd_tail: 8 =0A> dev.igb.1.queue6.tx_packets: 52654088 =0A> dev.igb.1.queue6.rxd_head: 1057 =0A> dev.igb.1.queue6.rxd_tail: 1041 =0A> dev.igb.1.queue6.rx_packets: 45227030 =0A> dev.igb.1.queue6.rx_bytes: 9494489640 =0A> dev.igb.1.queue7.interrupt_rate: 5235 =0A> dev.igb.1.queue7.txd_head: 729 =0A> dev.igb.1.queue7.txd_tail: 729 =0A> dev.igb.1.queue7.tx_packets: 61926105 =0A> dev.igb.1.queue7.rxd_head: 146 =0A> dev.igb.1.queue7.rxd_tail: 140 =0A> dev.igb.1.queue7.rx_packets: 51781775 =0A> dev.igb.1.queue7.rx_bytes: 8901279226 =0A> dev.igb.1.mac_stats.missed_packets: 1657 =0A> dev.igb.1.mac_stats.recv_no_buff: 405 =0A> dev.igb.1.mac_stats.total_pkts_recvd: 384332760 =0A> dev.igb.1.mac_stats.good_pkts_recvd: 384331103 =0A> dev.igb.1.mac_stats.bcast_pkts_recvd: 15510 =0A> dev.igb.1.mac_stats.mcast_pkts_recvd: 52957 =0A> dev.igb.1.mac_stats.rx_frames_64: 195496498 =0A> dev.igb.1.mac_stats.rx_frames_65_127: 133346124 =0A> dev.igb.1.mac_stats.rx_frames_128_255: 5254911 =0A> dev.igb.1.mac_stats.rx_frames_256_511: 9700049 =0A> dev.igb.1.mac_stats.rx_frames_512_1023: 16885886 =0A> dev.igb.1.mac_stats.rx_frames_1024_1522: 23647635 =0A> dev.igb.1.mac_stats.good_octets_recvd: 74284029276 =0A> dev.igb.1.mac_stats.good_octets_txd: 544536708502 =0A> dev.igb.1.mac_stats.total_pkts_txd: 485327419 =0A> dev.igb.1.mac_stats.good_pkts_txd: 485327419 =0A> dev.igb.1.mac_stats.bcast_pkts_txd: 72 =0A> dev.igb.1.mac_stats.mcast_pkts_txd: 52820 =0A> dev.igb.1.mac_stats.tx_frames_64: 57820809 =0A> dev.igb.1.mac_stats.tx_frames_65_127: 51586341 =0A> dev.igb.1.mac_stats.tx_frames_128_255: 7050579 =0A> dev.igb.1.mac_stats.tx_frames_256_511: 7887126 =0A> dev.igb.1.mac_stats.tx_frames_512_1023: 10130891 =0A> dev.igb.1.mac_stats.tx_frames_1024_1522: 350851673 =0A> dev.igb.1.interrupts.asserts: 551135045 =0A> dev.igb.1.interrupts.rx_pkt_timer: 384326679 =0A> dev.igb.1.interrupts.tx_queue_empty: 485323376 =0A> dev.igb.1.interrupts.tx_queue_min_thresh: 6324386 =0A> dev.igb.1.host.rx_pkt: 4424 =0A> dev.igb.1.host.tx_good_pkt: 4043 =0A> dev.igb.1.host.rx_good_bytes: 74284030864 =0A> dev.igb.1.host.tx_good_bytes: 544536708502 =0A> dev.igb.2.%desc: Intel(R) PRO/1000 Network Connection =0A> version - 2.3.1 =0A> dev.igb.2.%driver: igb =0A> dev.igb.2.%location: slot=3D0 function=3D2 =0A> dev.igb.2.%pnpinfo: vendor=3D0x8086 device=3D0x1521 =0A> subvendor=3D0x8086 subdevice=3D0x5001 class=3D0x020000 =0A> dev.igb.2.%parent: pci5 =0A> dev.igb.2.nvm: -1 =0A> dev.igb.2.enable_aim: 1 =0A> dev.igb.2.fc: 3 =0A> dev.igb.2.rx_processing_limit: 4096 =0A> dev.igb.2.eee_disabled: 1 =0A> dev.igb.2.link_irq: 2 =0A> dev.igb.2.device_control: 1209795137 =0A> dev.igb.2.rx_control: 67141658 =0A> dev.igb.2.interrupt_mask: 4 =0A> dev.igb.2.extended_int_mask: 2147483959 =0A> dev.igb.2.fc_high_water: 33168 =0A> dev.igb.2.fc_low_water: 33152 =0A> dev.igb.2.queue0.interrupt_rate: 13698 =0A> dev.igb.2.queue0.txd_head: 1618 =0A> dev.igb.2.queue0.txd_tail: 1618 =0A> dev.igb.2.queue0.tx_packets: 46401106 =0A> dev.igb.2.queue0.rxd_head: 831 =0A> dev.igb.2.queue0.rxd_tail: 827 =0A> dev.igb.2.queue0.rx_packets: 69356350 =0A> dev.igb.2.queue0.rx_bytes: 68488772907 =0A> dev.igb.2.queue1.interrupt_rate: 5405 =0A> dev.igb.2.queue1.txd_head: 190 =0A> dev.igb.2.queue1.txd_tail: 190 =0A> dev.igb.2.queue1.tx_packets: 55965886 =0A> dev.igb.2.queue1.rxd_head: 268 =0A> dev.igb.2.queue1.rxd_tail: 256 =0A> dev.igb.2.queue1.rx_packets: 58958084 =0A> dev.igb.2.queue1.rx_bytes: 69154569937 =0A> dev.igb.2.queue2.interrupt_rate: 83333 =0A> dev.igb.2.queue2.txd_head: 568 =0A> dev.igb.2.queue2.txd_tail: 568 =0A> dev.igb.2.queue2.tx_packets: 44974648 =0A> dev.igb.2.queue2.rxd_head: 371 =0A> dev.igb.2.queue2.rxd_tail: 219 =0A> dev.igb.2.queue2.rx_packets: 67037407 =0A> dev.igb.2.queue2.rx_bytes: 72042326102 =0A> dev.igb.2.queue3.interrupt_rate: 12658 =0A> dev.igb.2.queue3.txd_head: 867 =0A> dev.igb.2.queue3.txd_tail: 867 =0A> dev.igb.2.queue3.tx_packets: 55962467 =0A> dev.igb.2.queue3.rxd_head: 85 =0A> dev.igb.2.queue3.rxd_tail: 1953 =0A> dev.igb.2.queue3.rx_packets: 60972965 =0A> dev.igb.2.queue3.rx_bytes: 70397176035 =0A> dev.igb.2.queue4.interrupt_rate: 90909 =0A> dev.igb.2.queue4.txd_head: 1920 =0A> dev.igb.2.queue4.txd_tail: 1920 =0A> dev.igb.2.queue4.tx_packets: 47660931 =0A> dev.igb.2.queue4.rxd_head: 1397 =0A> dev.igb.2.queue4.rxd_tail: 1379 =0A> dev.igb.2.queue4.rx_packets: 59110758 =0A> dev.igb.2.queue4.rx_bytes: 68919201478 =0A> dev.igb.2.queue5.interrupt_rate: 111111 =0A> dev.igb.2.queue5.txd_head: 886 =0A> dev.igb.2.queue5.txd_tail: 886 =0A> dev.igb.2.queue5.tx_packets: 45103990 =0A> dev.igb.2.queue5.rxd_head: 812 =0A> dev.igb.2.queue5.rxd_tail: 799 =0A> dev.igb.2.queue5.rx_packets: 59030312 =0A> dev.igb.2.queue5.rx_bytes: 69234293962 =0A> dev.igb.2.queue6.interrupt_rate: 5208 =0A> dev.igb.2.queue6.txd_head: 1926 =0A> dev.igb.2.queue6.txd_tail: 1926 =0A> dev.igb.2.queue6.tx_packets: 46215046 =0A> dev.igb.2.queue6.rxd_head: 692 =0A> dev.igb.2.queue6.rxd_tail: 689 =0A> dev.igb.2.queue6.rx_packets: 58256050 =0A> dev.igb.2.queue6.rx_bytes: 68429172749 =0A> dev.igb.2.queue7.interrupt_rate: 50000 =0A> dev.igb.2.queue7.txd_head: 126 =0A> dev.igb.2.queue7.txd_tail: 126 =0A> dev.igb.2.queue7.tx_packets: 52451455 =0A> dev.igb.2.queue7.rxd_head: 968 =0A> dev.igb.2.queue7.rxd_tail: 885 =0A> dev.igb.2.queue7.rx_packets: 65946491 =0A> dev.igb.2.queue7.rx_bytes: 70263478849 =0A> dev.igb.2.mac_stats.missed_packets: 958 =0A> dev.igb.2.mac_stats.recv_no_buff: 69 =0A> dev.igb.2.mac_stats.total_pkts_recvd: 498658079 =0A> dev.igb.2.mac_stats.good_pkts_recvd: 498657121 =0A> dev.igb.2.mac_stats.bcast_pkts_recvd: 16867 =0A> dev.igb.2.mac_stats.mcast_pkts_recvd: 52957 =0A> dev.igb.2.mac_stats.rx_frames_64: 59089332 =0A> dev.igb.2.mac_stats.rx_frames_65_127: 52880118 =0A> dev.igb.2.mac_stats.rx_frames_128_255: 7526966 =0A> dev.igb.2.mac_stats.rx_frames_256_511: 8468389 =0A> dev.igb.2.mac_stats.rx_frames_512_1023: 10434770 =0A> dev.igb.2.mac_stats.rx_frames_1024_1522: 360257545 =0A> dev.igb.2.mac_stats.good_octets_recvd: 558910494322 =0A> dev.igb.2.mac_stats.good_octets_txd: 84618858153 =0A> dev.igb.2.mac_stats.total_pkts_txd: 394726904 =0A> dev.igb.2.mac_stats.good_pkts_txd: 394726904 =0A> dev.igb.2.mac_stats.bcast_pkts_txd: 48 =0A> dev.igb.2.mac_stats.mcast_pkts_txd: 52821 =0A> dev.igb.2.mac_stats.tx_frames_64: 196605932 =0A> dev.igb.2.mac_stats.tx_frames_65_127: 134602807 =0A> dev.igb.2.mac_stats.tx_frames_128_255: 5705236 =0A> dev.igb.2.mac_stats.tx_frames_256_511: 10267168 =0A> dev.igb.2.mac_stats.tx_frames_512_1023: 17165496 =0A> dev.igb.2.mac_stats.tx_frames_1024_1522: 30380265 =0A> dev.igb.2.interrupts.asserts: 465994260 =0A> dev.igb.2.interrupts.rx_pkt_timer: 498647546 =0A> dev.igb.2.interrupts.tx_queue_empty: 394720657 =0A> dev.igb.2.interrupts.tx_queue_min_thresh: 24424555 =0A> dev.igb.2.host.rx_pkt: 9575 =0A> dev.igb.2.host.tx_good_pkt: 6248 =0A> dev.igb.2.host.rx_good_bytes: 558910513984 =0A> dev.igb.2.host.tx_good_bytes: 84618858217 =0A> =0A> =0A> Thanks for your help. =0A> =0A> Cheers, =0A =0AYou're experiencing lock contention =0A =0ATry editing igb.c and setting the queues to 1. The multiqueue =0Aimplementation in igb has a negative impact. =0A =0AIf you have just 1 system get a dual port 82571 card and use the =0Aem driver. =0A =0ABC =0A_______________________________________________ =0Afreebsd-net@freebsd.org mailing list =0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-net =0ATo unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" =0A =0A From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 22:33:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 01E4AB7 for ; Sun, 28 Apr 2013 22:33:00 +0000 (UTC) (envelope-from spinney@tilera.com) Received: from USMAMAIL.TILERA.COM (usmamail.tilera.com [12.216.194.151]) by mx1.freebsd.org (Postfix) with ESMTP id A6FDA1795 for ; Sun, 28 Apr 2013 22:33:00 +0000 (UTC) Received: from USMAEXCH1.tad.internal.tilera.com ([fe80::709c:7a3e:ae82:7a6e]) by USMAExch2.tad.internal.tilera.com ([fe80::408c:3921:ab63:6a87%11]) with mapi; Sun, 28 Apr 2013 18:31:51 -0400 From: Barry Spinney To: "freebsd-net@freebsd.org" Subject: TF_NEEDSYN flag never set. Thread-Topic: TF_NEEDSYN flag never set. Thread-Index: Ac5EXUkf2910zLpuSFy6dwGLhSsWmQ== Date: Sun, 28 Apr 2013 22:31:48 +0000 Message-ID: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Barry Spinney X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 22:33:01 -0000 I am sorry if this is a dumb question, but I was trying to understand the F= reeBSD TCP stack, and In particular I was trying to understand the use of the TF_NEEDSYN flag= . This flag is referenced a number of times in tcp_input.c and tcp_output.c, but I don'= t think that it can ever be set. In particular grepping through the "../src/sys/netinet", one discovers that= the only code that can set this flag is lines 1018 and 1020 of tcp_input.c. But, it appe= ars to me that none of the lines in tcp_input.c from 999 thru 1023 are even reachable! Th= e reason they are not reachable is because just ahead of them are the following lines: if (!syncache_add(&inc, &to, th, &so, m)) goto drop; if (so =3D=3D NULL) { ... // uninteresting lines, but no gotos return; } ... /unreachable code here Studying syncache_add (in file tcp_syncache.c), reveals three return stat= ements. One of the returns, returns the value 0, which will cause the "goto drop"= to be executed. The other two returns, return both the value 1 AND set "*sop =3D NULL", w= hich should cause the following "if (so =3D=3D NULL)" to execute the subsequent return stat= ement. Is this intentional? (i.e. dead code awaiting future development?), or a bu= g? Or I am going blind to something obvious? Thanx Barry Spinney. (p.s. I doubt it matters which version of code, but to be precise this is f= rom the /pub/FreeBSD/development/tarballs named "src_stable_6.tar.gz" dated "4/21/2= 013 01:15 AM", gotten from ftp1.us.freebsd.org) From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 23:05:03 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 12F9B2EA for ; Sun, 28 Apr 2013 23:05:03 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id DC2311A5A for ; Sun, 28 Apr 2013 23:05:02 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id 9so6767263iec.36 for ; Sun, 28 Apr 2013 16:05:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=L0kIrN91jkT4TcDOuvecUQGhyxCg9qUqL7QifSyv0Ek=; b=SScj9re72y9Xf40Ua74yX1Sxop+uqoLAJQY7YKxBwJFXvZWGWd7qnI2nu2aKJI7BMl 7sPypIqaVCP5Vcgcnv/CNDmBvzMhCEPSiYn8GJ8kRQVDTgDR8eW7bCMOHCF3k/1kp3eM 2zQrgdM548Kgv+8DF4j4MypSOuMBGwvvYVsDH4o7dxyRNKE0pscOdz7nmu/vUeftkjCh FbrLt9pNOyw8DdHBkua0YdwvnNosfpD9tUWaTTIaGAwznDxhQf7vOX8I9dzTnMtRM/6n Qz/japhqmgwecibdDksoRdvvslpRcQPQOOH1BaEErPVDfYTn07lahIXuLyPwnl/UAtZv y4Lg== X-Received: by 10.42.64.69 with SMTP id f5mr28046489ici.29.1367190302674; Sun, 28 Apr 2013 16:05:02 -0700 (PDT) Received: from gloom ([66.11.160.35]) by mx.google.com with ESMTPSA id c2sm14726452igv.1.2013.04.28.16.05.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 28 Apr 2013 16:05:01 -0700 (PDT) Date: Sun, 28 Apr 2013 19:04:54 -0400 From: Mark Johnston To: Barry Spinney Subject: Re: TF_NEEDSYN flag never set. Message-ID: <20130428230454.GA31215@gloom> References: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 23:05:03 -0000 On Sun, Apr 28, 2013 at 10:31:48PM +0000, Barry Spinney wrote: > I am sorry if this is a dumb question, but I was trying to understand the FreeBSD TCP stack, > and In particular I was trying to understand the use of the TF_NEEDSYN flag. This flag > is referenced a number of times in tcp_input.c and tcp_output.c, but I don't think that > it can ever be set. > > In particular grepping through the "../src/sys/netinet", one discovers that the only code > that can set this flag is lines 1018 and 1020 of tcp_input.c. But, it appears to me that > none of the lines in tcp_input.c from 999 thru 1023 are even reachable! The reason they > are not reachable is because just ahead of them are the following lines: > > if (!syncache_add(&inc, &to, th, &so, m)) > goto drop; > if (so == NULL) { > ... // uninteresting lines, but no gotos > return; > } > ... /unreachable code here > > > Studying syncache_add (in file tcp_syncache.c), reveals three return statements. > One of the returns, returns the value 0, which will cause the "goto drop" to be executed. > The other two returns, return both the value 1 AND set "*sop = NULL", which should cause > the following "if (so == NULL)" to execute the subsequent return statement. > > Is this intentional? (i.e. dead code awaiting future development?), or a bug? > Or I am going blind to something obvious? > > Thanx Barry Spinney. > > (p.s. I doubt it matters which version of code, but to be precise this is from the > /pub/FreeBSD/development/tarballs named "src_stable_6.tar.gz" dated "4/21/2013 01:15 AM", > gotten from ftp1.us.freebsd.org) That tarball presumably contains sources for the stable branch of FreeBSD 6, which probably isn't what you're looking for - the last release from that branch was in 2008. The relevant code in FreeBSD-CURRENT is different and your observations don't seem to apply there. Based on a comment added in r156125, you seem to be correct though. :) http://svnweb.freebsd.org/base?view=revision&revision=156125 I'd suggest fetching src_current.tar.gz from the FTP same directory and looking at that instead - you're more likely to get replies to questions about the current tip of development. From owner-freebsd-net@FreeBSD.ORG Sun Apr 28 23:29:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DA032B8A for ; Sun, 28 Apr 2013 23:29:56 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-lb0-f181.google.com (mail-lb0-f181.google.com [209.85.217.181]) by mx1.freebsd.org (Postfix) with ESMTP id 649761B61 for ; Sun, 28 Apr 2013 23:29:56 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id 13so3019218lba.26 for ; Sun, 28 Apr 2013 16:29:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :x-gm-message-state; bh=qEz6K6Z8kRZ8Xio5Qq7D+zUcjAI+6bL4jEHjOhTLsKI=; b=n5Ymc5NylrBDeF+i8B4kSu1aiy/hRCvqAEhgpeHHimFYpnAgL2MGYCQ38i2e492XdB VmwnTIXXsXsOYANMVw6M4FQubLfDCxIAWk2JijCHuxnAtuMvUlfrf0E9fziawHsDX6mH ujpnR4vZ30bzPG3Abjdwx54fEzzNAF4l7/IiOJnlDw59IgkDyTPKip03Uy5IdRBkRwhz KRrmyf5WGlzi1iD+s8RmxXtTU38INVb8Lom2oZek6v5EHPKjvkAfBFAXi2sZ7XiEZsSB wEDQdpXk4fFArIuRbMgEB73B3M/ypZdhQiuop8u4zlQSJ74L4wUEQLsrSpvQ36tgs3QD FBeQ== X-Received: by 10.112.162.40 with SMTP id xx8mr25808390lbb.30.1367191789177; Sun, 28 Apr 2013 16:29:49 -0700 (PDT) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.152.28.196 with HTTP; Sun, 28 Apr 2013 16:29:29 -0700 (PDT) In-Reply-To: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> References: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> From: Juli Mallett Date: Sun, 28 Apr 2013 16:29:29 -0700 X-Google-Sender-Auth: sTiKz4uM_BVqaIkd2TnKY6pzg8g Message-ID: Subject: Re: TF_NEEDSYN flag never set. To: Barry Spinney Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnXRY7RJIDdVFOPm3+xqe1fNL+vbraE+mGD6NA1Sb/HK5ADGx/mX0wkxyHjvHpXP/Hk7L3Y Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 23:29:56 -0000 On Sun, Apr 28, 2013 at 3:31 PM, Barry Spinney wrote: > (p.s. I doubt it matters which version of code, but to be precise this is from the > /pub/FreeBSD/development/tarballs named "src_stable_6.tar.gz" dated "4/21/2013 01:15 AM", > gotten from ftp1.us.freebsd.org) Barry, of course the version of code matters :) You might try using FXR for these sorts of lookups in the future: http://fxr.watson.org/fxr/ident?i=TF_NEEDSYN Or just check out -CURRENT using Subversion. Or are you actually working on 6.x and so need an answer for 6.x? Thanks, Juli. From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 04:04:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C49A8CE6 for ; Mon, 29 Apr 2013 04:04:06 +0000 (UTC) (envelope-from spinney@tilera.com) Received: from USMAMAIL.TILERA.COM (usmamail.tilera.com [12.216.194.151]) by mx1.freebsd.org (Postfix) with ESMTP id 76FF611F4 for ; Mon, 29 Apr 2013 04:04:06 +0000 (UTC) Received: from USMAEXCH1.tad.internal.tilera.com ([fe80::709c:7a3e:ae82:7a6e]) by USMAExch2.tad.internal.tilera.com ([fe80::408c:3921:ab63:6a87%11]) with mapi; Mon, 29 Apr 2013 00:04:05 -0400 From: Barry Spinney To: Mark Johnston Subject: RE: TF_NEEDSYN flag never set. Thread-Topic: TF_NEEDSYN flag never set. Thread-Index: Ac5EXUkf2910zLpuSFy6dwGLhSsWmQAKQlUAAAC53xA= Date: Mon, 29 Apr 2013 04:04:04 +0000 Message-ID: <73BC01C897E9F642AC962BF311A45ACB100B0081@USMAExch1.tad.internal.tilera.com> References: <73BC01C897E9F642AC962BF311A45ACB100B0057@USMAExch1.tad.internal.tilera.com> <20130428230454.GA31215@gloom> In-Reply-To: <20130428230454.GA31215@gloom> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" , "juli@clockworksquid.com" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 04:04:06 -0000 Yes, I switched over to using src_current.tar.gz, and that specific bug see= ms to be gone. Thanx a lot. BUT, the original reason I was looking so closely at this code, still remai= ns a mystery to me. In particular, the comment on lines 2716-2717 of tcp_input.c, along with th= e following statement, seems to be wrong to me. Specifically: /* * If no data (only SYN) was ACK'd, * skip rest of ACK processing. */ if (acked =3D=3D 0) goto step6; This comment implies to me that if the current rcvd pkt is a (say pure) AC= K, that happens to ACK my SYN-ACK (here it is assumed that the stack is operat= ing as a server, and we previously rcvd a SYN, sent a SYN-ACK, and are now rcvi= ng the subsequent ACK), then acked should be 0 and we should "goto step6". However my analysis of the code seems to imply that in this case "acked" wi= ll have the value 1, which will proceed to execute in the following code, most of w= hich seems to expect some real data bytes to have been ack'd. Now, I am not sure tha= t continuing past this goto necessarily causes harm, but it sure seems unexpe= cted, and in the future someone else might add code here assuming that the current AC= K acks some real data bytes. Regarding my analysis, that acked is 1 in this case, one can examine syncac= he_socket (called by syncache_expand when we promote a syncache entry to a full entry= upon receipt of the aforementioned ACK pkt), on line 821 of tcp_syncache.c. This invoke= s the tcp_sendseqinit macro (defined on line 62 of tcp_seq.h), which sets "snd_un= a" to be "iss". Also line 817 of tcp_syncache.c sets the t_state to be TCPS_SYN_RECEIVED. One can also see that snd_una MUST be iss, at least by noticing that the in= coming ackNum must of course be equal to "iss + 1" (since it acks the SYN which takes a s= equence number), and seeing that line 1916 of tcp_input.c would drop the pkt if ackNum were = <=3D snd_una=20 (assuming the t_state is still TCPS_SYN_RECEIVED - which more boring analys= is seems to confirm). By the way line 2398 of tcp_input.c will change the t_state from TCPS_SYN_R= ECEIVED to TCPS_ESTABLISHED. Also notice line 2438 of tcp_input.c, which would cause = the code to go down the duplicate ACK pathway if the ackNum were <=3D snd_una (the equa= lity be the key here). Finally we reach line 2661 of tcp_input.c which is: acked =3D BYTES_THIS_ACK(tp, th); Where BYTES_THIS_ACK is just "ackNum - snd_una" (see line 261 of tcp_var.h)= . But since snd_una=3D=3Diss and ackNum=3Diss+1, "acked" must be 1. Thanx Barry. (p.s. the reason I am curious is I have been doing some work with a tcp sta= ck that was original derived from some FreeBSD software, though with non-trivial ch= anges, and I have been trying to understand its workings). =20 -----Original Message----- From: Mark Johnston [mailto:markjdb@gmail.com]=20 Sent: Sunday, April 28, 2013 7:05 PM To: Barry Spinney Cc: freebsd-net@freebsd.org Subject: Re: TF_NEEDSYN flag never set. On Sun, Apr 28, 2013 at 10:31:48PM +0000, Barry Spinney wrote: > I am sorry if this is a dumb question, but I was trying to understand=20 > the FreeBSD TCP stack, and In particular I was trying to understand=20 > the use of the TF_NEEDSYN flag. This flag is referenced a number of=20 > times in tcp_input.c and tcp_output.c, but I don't think that it can ever= be set. >=20 > In particular grepping through the "../src/sys/netinet", one discovers=20 > that the only code that can set this flag is lines 1018 and 1020 of=20 > tcp_input.c. But, it appears to me that none of the lines in=20 > tcp_input.c from 999 thru 1023 are even reachable! The reason they are n= ot reachable is because just ahead of them are the following lines: >=20 > if (!syncache_add(&inc, &to, th, &so, m)) > goto drop; > if (so =3D=3D NULL) { > ... // uninteresting lines, but no gotos > return; > } > ... /unreachable code here >=20 >=20 > Studying syncache_add (in file tcp_syncache.c), reveals three return st= atements. > One of the returns, returns the value 0, which will cause the "goto dro= p" to be executed. > The other two returns, return both the value 1 AND set "*sop =3D NULL",= which should cause > the following "if (so =3D=3D NULL)" to execute the subsequent return st= atement. >=20 > Is this intentional? (i.e. dead code awaiting future development?), or a = bug? > Or I am going blind to something obvious? >=20 > Thanx Barry Spinney. >=20 > (p.s. I doubt it matters which version of code, but to be precise this=20 > is from the /pub/FreeBSD/development/tarballs named=20 > "src_stable_6.tar.gz" dated "4/21/2013 01:15 AM", gotten from=20 > ftp1.us.freebsd.org) That tarball presumably contains sources for the stable branch of FreeBSD 6= , which probably isn't what you're looking for - the last release from that= branch was in 2008. The relevant code in FreeBSD-CURRENT is different and your observations don= 't seem to apply there. Based on a comment added in r156125, you seem to be= correct though. :) http://svnweb.freebsd.org/base?view=3Drevision&revision=3D156125 I'd suggest fetching src_current.tar.gz from the FTP same directory and loo= king at that instead - you're more likely to get replies to questions about= the current tip of development. From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 06:40:59 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A646E9E6 for ; Mon, 29 Apr 2013 06:40:59 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 35248193A for ; Mon, 29 Apr 2013 06:40:58 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3T6euRI006224; Mon, 29 Apr 2013 10:40:56 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3T6etTg006223; Mon, 29 Apr 2013 10:40:55 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 29 Apr 2013 10:40:55 +0400 From: Gleb Smirnoff To: Kajetan Staszkiewicz Subject: Re: pf performance? Message-ID: <20130429064055.GH76816@FreeBSD.org> References: <5176E5C1.9090601@soe.ucsc.edu> <517974DA.5090809@soe.ucsc.edu> <201304260021.11209.vegeta@tuxpowered.net> <201304262150.17215.vegeta@tuxpowered.net> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <201304262150.17215.vegeta@tuxpowered.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org, Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 06:40:59 -0000 Kajetan, On Fri, Apr 26, 2013 at 09:50:17PM +0200, Kajetan Staszkiewicz wrote: K> > > > How do you count the 140kpps value? One interface, both, in, out? I'd K> > > > like to relate this somehow to my values. K> > > K> > > Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx K> > > spike to 150kpps occasionally. K> > K> > Unfortunately at this moment I have no single machine with such traffic, K> > although maybe I can aggregate some traffic later and check the cpu usage K> > then. K> K> OK, got my CPU usage for 30/40kpps, 55/340Mbit/s (public side, in/out). K> K> CPU is 4-core E5540 @ 2.53GHz with HT disabled. K> K> Cpu usage looks more or less like this: K> CPU 0: 0.0% user, 0.0% nice, 0.4% system, 3.9% interrupt, 95.7% idle K> CPU 1: 0.0% user, 0.0% nice, 1.2% system, 18.1% interrupt, 80.7% idle K> CPU 2: 0.0% user, 0.0% nice, 1.6% system, 5.5% interrupt, 92.9% idle K> CPU 3: 0.0% user, 0.0% nice, 0.8% system, 26.8% interrupt, 72.4% idle K> K> Public network card is pinned to cpu 2, internal to cpu 3, each card has only a K> single irq. Netisr threads are limited to cpu 0 and 1, I use deferred netisr. K> K> So yes, I have 2x less pps than you, but also I have quite a slower cpu and K> there still seems to be much cpu power left. When it comes to contention, the dependence of CPU utilization from amount of data processed isn't linear. While you have low enough pps, a time spent by a CPU in pf is accounted as time of that CPU, and that's all. But if pps are higher, then while that CPU was working in pf, couple more CPUs were spinning on pf lock, and now their CPU time is also utilized and accounted (although they did nothing). -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 08:41:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C487A8DF for ; Mon, 29 Apr 2013 08:41:53 +0000 (UTC) (envelope-from bounces1@mail.emobimail.com) Received: from mail.emobimail.com (mail.emobimail.com [183.82.9.2]) by mx1.freebsd.org (Postfix) with ESMTP id 7B1851FC5 for ; Mon, 29 Apr 2013 08:41:53 +0000 (UTC) Received: from mail.emobimail.com (localhost [127.0.0.1]) by mail.emobimail.com (Postfix) with ESMTP id 2BA6AE80E49 for ; Mon, 29 Apr 2013 14:11:30 +0530 (IST) X-DKIM: Sendmail DKIM Filter v2.8.3 mail.emobimail.com 2BA6AE80E49 To: freebsd-net@freebsd.org Subject: SpaceDart.com Offers Pay Per Click Campaign! Message-ID: Date: Mon, 29 Apr 2013 13:42:03 +0530 From: "Spacedart" MIME-Version: 1.0 X-Mailer-LID: 527 X-Mailer-RecptId: 5464996 X-Mailer-SID: 550 X-Mailer-Sent-By: 40 Content-Type: text/plain; format=flowed; charset="UTF-8" Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: promo@spacedart.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 08:41:53 -0000 Greetings, SpaceDart.com is India’s real estate destination. Users can find and post properties as well as search to buy/sale/rent, both new or resale Flats,homes and villas. Affiliates can leverage SpaceDart.com’s presence as the premiere real estate company in India to grow this affiliate program. Here are the program specifics: SpaceDart.com provides tracking links that you can put on your site. Visitors click on the links and search for property on SpaceDart.com. We happily pay our affiliates for the traffic referred to us. Sign up Today! http://mail.emobimail.com/link.php?M=5464996&N=550&L=271&F=T If your site reaches a Indian audience and you are promoting other real estate affiliate programs then you need to be promoting SpaceDart.com. Feel free to contact us if you have questions or if you need anything, we are here to help. Thanks! Uday Bhaskar Affiliate Manager www.spacedart.com e-mail: udayabhaskar@spacedart.com From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 09:24:29 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2B51DEE7 for ; Mon, 29 Apr 2013 09:24:29 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id D02A91310 for ; Mon, 29 Apr 2013 09:24:28 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.6/8.14.6) with ESMTP id r3T9OOEM055115 for ; Mon, 29 Apr 2013 16:24:24 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <517E3C43.905@rdtc.ru> Date: Mon, 29 Apr 2013 16:24:19 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: "net@freebsd.org" Subject: lagg+vlan+igb kernel bug Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 09:24:29 -0000 Hi! Today I discovered really subtle bug in connection of lagg(4), vlan(4) and igb(4) drivers. Here is what I have in /etc/rc.conf: cloned_interfaces="lagg0 lagg1" ifconfig_lagg1="laggproto lacp laggport igb0 laggport igb1" #ifconfig_lagg1_alias0="link 00:1b:21:63:5b:d1" ifconfig_igb0="-wol" ifconfig_igb1="-wol" My lagg1 has no IP addresses but lots of vlans created on it. These vlans carry PPPoE frames only. This configuration works just fine. But, it I uncomment the line changing MAC-address of lagg1 (ifconfig_lagg1_alias0), I still have igb0 working just fine (input/output/broadcast/unicast - all OK) but igp1 stops accepting unicasts. More precisely: if I bring igb0 link down, lagg1 excludes it from aggregation group, then tcpdump -ni igb1 shows that no unicasts come in, broadcasts only. I use switch port mirroring feature to make sure that switch directs unicasts to igb1. However, if I manually recreate lagg1 and assign MAC to lagg1 first and assign igb0/igb1 later, both ports run just fine and this issue does not rise. It seems, lagg(4) fails to reprogramm igb1 with new MAC in first case? In both cases, ifconfig shows single MAC address for lagg1 and both its ports. Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 11:06:48 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B6F6E3BB for ; Mon, 29 Apr 2013 11:06:48 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id A777F1928 for ; Mon, 29 Apr 2013 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3TB6m7T018208 for ; Mon, 29 Apr 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3TB6mbb018206 for freebsd-net@FreeBSD.org; Mon, 29 Apr 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Apr 2013 11:06:48 GMT Message-Id: <201304291106.r3TB6mbb018206@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/178116 net [tcp] [panic] Kernel panic: general protection fault i o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177417 net [ip6] Invalid protocol value in ipsec6_common_input_cb o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod o kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176667 net [libalias] [patch] libalias locks on uninitalized data o kern/176596 net [firewire] [ip6] Crash with IPv6 and Firewire o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176446 net [netinet] [patch] Concurrency in ixgbe driving out-of- o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176097 net [lagg] [patch] lagg/lacp broken when aggregated interf o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o bin/175974 net ppp(8): logic issue o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos o kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169664 net [bgp] Wrongful replacement of interface connected net o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/169438 net [ipsec] ipv4-in-ipv6 tunnel mode IPsec does not work p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167947 net [setfib] [patch] arpresolve checks only the default FI o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/167059 net [tcp] [panic] System does panic in in_pcbbind() and ha o kern/166940 net [ipfilter] [panic] Double fault in kern 8.2 o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166372 net [patch] [ipfilter] drops UDP packets with zero checksu o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165963 net [panic] [ipfilter] ipfilter/nat NULL pointer deference p kern/165903 net mbuf leak o kern/165643 net [net] [patch] Missing vnet restores in net/if_ethersub o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162926 net [ipfilter] Infinite loop in ipfilter with fragmented I o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel p kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipfilter] There is no ippool start script/ipfilter ma o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [ipfilter] [rc.d] [patch] No good way to set ipfilter o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipfilter] FreeBSD 6.1+VPN+ipnat+ipf: port mapping doe o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipfilter] [panic] Kernel Panic Trap No 12 Page Fault o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipfilter] [patch] ipfilter drops OOW packets under 6. o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipfilter] [patch] wrong behaviour with skip rule insi o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87521 net [ipfilter] [panic] using ipfilter "auth" keyword leads o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipfilter] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation s kern/77195 net [ipfilter] [patch] ipfilter ioctl SIOCGNATL does not m o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipfilter] ipfilter ipnat problem with h323 proxy supp o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipfilter] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipfilter] [ppp] Interactive use of user PPP and ipfil f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 463 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 12:11:10 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 51732CA3 for ; Mon, 29 Apr 2013 12:11:10 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id EAC6310BF for ; Mon, 29 Apr 2013 12:11:09 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.6/8.14.6) with ESMTP id r3TCB5ax057103 for ; Mon, 29 Apr 2013 19:11:05 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <517E6354.7050401@rdtc.ru> Date: Mon, 29 Apr 2013 19:11:00 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: "net@freebsd.org" Subject: Re: lagg+vlan+igb kernel bug References: <517E3C43.905@rdtc.ru> In-Reply-To: <517E3C43.905@rdtc.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 12:11:10 -0000 On 29.04.2013 16:24, Eugene Grosbein wrote: > Hi! > > Today I discovered really subtle bug in connection of lagg(4), vlan(4) and igb(4) drivers. > > Here is what I have in /etc/rc.conf: > > cloned_interfaces="lagg0 lagg1" > ifconfig_lagg1="laggproto lacp laggport igb0 laggport igb1" > #ifconfig_lagg1_alias0="link 00:1b:21:63:5b:d1" > ifconfig_igb0="-wol" > ifconfig_igb1="-wol" > > My lagg1 has no IP addresses but lots of vlans created on it. > These vlans carry PPPoE frames only. This configuration works just fine. > > But, it I uncomment the line changing MAC-address of lagg1 (ifconfig_lagg1_alias0), > I still have igb0 working just fine (input/output/broadcast/unicast - all OK) > but igp1 stops accepting unicasts. > > More precisely: if I bring igb0 link down, lagg1 excludes it from aggregation group, > then tcpdump -ni igb1 shows that no unicasts come in, broadcasts only. > > I use switch port mirroring feature to make sure that switch directs unicasts to igb1. > > However, if I manually recreate lagg1 and assign MAC to lagg1 first and > assign igb0/igb1 later, both ports run just fine and this issue does not rise. > > It seems, lagg(4) fails to reprogramm igb1 with new MAC in first case? > In both cases, ifconfig shows single MAC address for lagg1 and both its ports. Forgot to mention, I use 8.3-STABLE/amd64 with dual-port 82576 adapter and stock igb(4) driver version 2.3.1. Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Tue Apr 30 01:17:33 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 130C14CA for ; Tue, 30 Apr 2013 01:17:33 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id 901FC11F6 for ; Tue, 30 Apr 2013 01:17:32 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id s47so3661wey.22 for ; Mon, 29 Apr 2013 18:17:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=bQ3/2h++Beo8WD7daUIJk4sueQfs0F9PS5SVafpv+NE=; b=Pxzudpf4JesjMLuFJJJxn+qz7i2Cfcp2ZgZkzSLHpKjFliWSi+YP7qzHAzDFl8OOTC ktQWsH/UGirFOMpFmcj9QmJldrRJ6jt6JCIForWGViFvWWY0z5hKtlNDojs8mcAzMjnx KMmZnT/Mwi5FP/wgeEEUbTYEC/XIFtZOyC4ZgVzbC5JSx0XKb7JVfO2qZwEUVjFfA5HP /XnncqwQr5vOZBUMiusyxmcjSjPDQudyuoK8QBiVtp5UCwU0Aw31Ov8YhdjUAOy//Bxh CNPVUE7xs+lJFxQCh2GbrgM2Y/nchSd1tj8RCqR8yDDnOpYL6/vbRRcZYw9xcGtWvrpn kuUQ== MIME-Version: 1.0 X-Received: by 10.180.24.69 with SMTP id s5mr21211417wif.34.1367284651074; Mon, 29 Apr 2013 18:17:31 -0700 (PDT) Received: by 10.194.82.138 with HTTP; Mon, 29 Apr 2013 18:17:30 -0700 (PDT) Date: Mon, 29 Apr 2013 21:17:30 -0400 Message-ID: Subject: in_lltable_rtcheck From: Juan Mojica To: FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2013 01:17:33 -0000 Why is the code comparing the entire sockaddr structure instead of just the relevant fields? We have a flood of the log message below when transitioning an IP address from one port to another. And this triggers other behavior as well. Through GDB, we can see that the addresses are in the same subnet. The problem is that the sin_port port fields in the l3addr and in sa do not match. Is there a reason sin_port should be compared here? I can come up with a patch, that's not an issue, but I wanted to confirm with others first. if (!(rt->rt_flags & RTF_HOST) && rt->rt_ifp != ifp) { const char *sa, *mask, *addr, *lim; int len; mask = (const char *)rt_mask(rt); /* * Just being extra cautious to avoid some custom * code getting into trouble. */ if (mask == NULL) { RTFREE_LOCKED(rt); return (EINVAL); } sa = (const char *)rt_key(rt); addr = (const char *)l3addr; len = ((const struct sockaddr_in *)l3addr)->sin_len; lim = addr + len; for ( ; addr < lim; sa++, mask++, addr++) { if ((*sa ^ *addr) & *mask) { #ifdef DIAGNOSTIC log(LOG_INFO, "IPv4 address: \"%s\" is not on the network\n", inet_ntoa(((const struct sockaddr_in *)l3addr)->sin_addr)); #endif RTFREE_LOCKED(rt); return (EINVAL); } } } Regards -- Juan Mojica Email: jmojica@gmail.com From owner-freebsd-net@FreeBSD.ORG Tue Apr 30 01:46:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3072E8E9; Tue, 30 Apr 2013 01:46:22 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from plsvl-mailgw-02.bluecoat.com (plsvl-mailgw-02.bluecoat.com [199.91.133.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0EB9C12FB; Tue, 30 Apr 2013 01:46:21 +0000 (UTC) Received: from PWSVL-EXCHTS-01.internal.cacheflow.com (unknown [10.2.2.122]) by plsvl-mailgw-02.bluecoat.com (Postfix) with ESMTP id 358A02010F; Mon, 29 Apr 2013 18:40:57 -0700 (PDT) Received: from PWSVL-EXCMBX-04.internal.cacheflow.com ([fe80::c596:c77:dd67:b72d]) by PWSVL-EXCHTS-01.internal.cacheflow.com ([fe80::5c50:e2ba:8115:4223%20]) with mapi id 14.01.0289.001; Mon, 29 Apr 2013 18:40:56 -0700 From: "Li, Qing" To: Juan Mojica , FreeBSD Net Subject: RE: in_lltable_rtcheck Thread-Topic: in_lltable_rtcheck Thread-Index: AQHORUCD21qRbZKIH0GzdfKaIfmSsJjt+vYI Date: Tue, 30 Apr 2013 01:40:55 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [199.91.133.85] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "qingli@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2013 01:46:22 -0000 The problem you described here seemed familiar so I checked into the svn hi= story,=0A= and found I have in fact fixed this issue in other parts of the code.=0A= =0A= Please see =0A= http://svnweb.freebsd.org/base?view=3Drevision&revision=3D186708=0A= =0A= So I think similar fix should be applied here as well.=0A= =0A= --Qing=0A= =0A= =0A= ________________________________________=0A= From: owner-freebsd-net@freebsd.org [owner-freebsd-net@freebsd.org] on beha= lf of Juan Mojica [jmojica@gmail.com]=0A= Sent: Monday, April 29, 2013 6:17 PM=0A= To: FreeBSD Net=0A= Subject: in_lltable_rtcheck=0A= =0A= Why is the code comparing the entire sockaddr structure instead of just the= =0A= relevant fields? We have a flood of the log message below when=0A= transitioning an IP address from one port to another. And this triggers=0A= other behavior as well.=0A= =0A= Through GDB, we can see that the addresses are in the same subnet. The=0A= problem is that the sin_port port fields in the l3addr and in sa do not=0A= match.=0A= =0A= Is there a reason sin_port should be compared here?=0A= =0A= I can come up with a patch, that's not an issue, but I wanted to confirm=0A= with others first.=0A= =0A= =0A= if (!(rt->rt_flags & RTF_HOST) && rt->rt_ifp !=3D ifp) {=0A= const char *sa, *mask, *addr, *lim;=0A= int len;=0A= =0A= mask =3D (const char *)rt_mask(rt);=0A= /*=0A= * Just being extra cautious to avoid some custom=0A= * code getting into trouble.=0A= */=0A= if (mask =3D=3D NULL) {=0A= RTFREE_LOCKED(rt);=0A= return (EINVAL);=0A= }=0A= =0A= sa =3D (const char *)rt_key(rt);=0A= addr =3D (const char *)l3addr;=0A= len =3D ((const struct sockaddr_in *)l3addr)->sin_len;=0A= lim =3D addr + len;=0A= =0A= for ( ; addr < lim; sa++, mask++, addr++) {=0A= if ((*sa ^ *addr) & *mask) {=0A= #ifdef DIAGNOSTIC=0A= log(LOG_INFO, "IPv4 address: \"%s\" is not = on the network\n",=0A= inet_ntoa(((const struct sockaddr_in *)= l3addr)->sin_addr));=0A= #endif=0A= RTFREE_LOCKED(rt);=0A= return (EINVAL);=0A= }=0A= }=0A= }=0A= =0A= =0A= Regards=0A= =0A= --=0A= Juan Mojica=0A= Email: jmojica@gmail.com=0A= _______________________________________________=0A= freebsd-net@freebsd.org mailing list=0A= http://lists.freebsd.org/mailman/listinfo/freebsd-net=0A= To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A= From owner-freebsd-net@FreeBSD.ORG Tue Apr 30 05:50:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B4860DDF for ; Tue, 30 Apr 2013 05:50:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 955321C96 for ; Tue, 30 Apr 2013 05:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3U5o1Cg054339 for ; Tue, 30 Apr 2013 05:50:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3U5o1wR054331; Tue, 30 Apr 2013 05:50:01 GMT (envelope-from gnats) Date: Tue, 30 Apr 2013 05:50:01 GMT Message-Id: <201304300550.r3U5o1wR054331@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Nate Denning Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Nate Denning List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2013 05:50:01 -0000 The following reply was made to PR kern/178116; it has been noted by GNATS. From: Nate Denning To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Mon, 29 Apr 2013 23:46:58 -0600 --089e013a16ec95f6f204db8d8750 Content-Type: text/plain; charset=ISO-8859-1 The em panics continue. I've received them from both em0 and em1. Below is a new variation not related to tcp_do_segment. Let me know if I can provide more info. Thanks, Nate --- core.txt --- Fatal trap 9: general protection fault while in kernel mode cpuid = 4; apic id = 32 instruction pointer = 0x20:0xffffffff8097dc20 stack pointer = 0x28:0xffffff83428cd9b0 frame pointer = 0x28:0xffffff83428cd9c0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em0 que) trap number = 9 panic: general protection fault cpuid = 4 KDB: stack backtrace: #0 0xffffffff80952f46 at kdb_backtrace+0x66 #1 0xffffffff8091a2de at panic+0x1ce #2 0xffffffff80ca8b80 at trap_fatal+0x290 #3 0xffffffff80ca9391 at trap+0x241 #4 0xffffffff80c92813 at calltrap+0x8 #5 0xffffffff809cf257 at bpf_mtap+0x37 #6 0xffffffff809d9549 at ether_nh_input+0x259 #7 0xffffffff809e2578 at netisr_dispatch_src+0x218 #8 0xffffffff804dd248 at em_rxeof+0x1c8 #9 0xffffffff804dd6f8 at em_handle_que+0x48 #10 0xffffffff8095f9b4 at taskqueue_run_locked+0x74 #11 0xffffffff80960966 at taskqueue_thread_loop+0x46 #12 0xffffffff808e83af at fork_exit+0x11f #13 0xffffffff80c92d3e at fork_trampoline+0xe Uptime: 17h44m42s Dumping 2486 out of 8158 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/ipl.ko...Reading symbols from /boot/kernel/ipl.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipl.ko Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from /boot/kernel/accf_http.ko.symbols...done. done. Loaded symbols for /boot/kernel/accf_http.ko Reading symbols from /boot/kernel/accf_data.ko...Reading symbols from /boot/kernel/accf_data.ko.symbols...done. done. Loaded symbols for /boot/kernel/accf_data.ko #0 doadump (textdump=) at pcpu.h:229 229 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=) at pcpu.h:229 #1 0xffffffff80919db4 in kern_reboot (howto=260) at /usr/src-9-stable/sys/kern/kern_shutdown.c:449 #2 0xffffffff8091a2b7 in panic (fmt=0x1
) at /usr/src-9-stable/sys/kern/kern_shutdown.c:637 #3 0xffffffff80ca8b80 in trap_fatal (frame=0x9, eva=) at /usr/src-9-stable/sys/amd64/amd64/trap.c:878 #4 0xffffffff80ca9391 in trap (frame=0xffffff83428cd900) at /usr/src-9-stable/sys/amd64/amd64/trap.c:605 #5 0xffffffff80c92813 in calltrap () at /usr/src-9-stable/sys/amd64/amd64/exception.S:228 #6 0xffffffff8097dc20 in m_length (m0=0x3b4d5ae18672812f, last=0x0) at /usr/src-9-stable/sys/kern/uipc_mbuf.c:1459 #7 0xffffffff809cf257 in bpf_mtap (bp=0xfffffe000605ec00, m=0xfffffe0036056600) at /usr/src-9-stable/sys/net/bpf.c:2110 #8 0xffffffff809d9549 in ether_nh_input (m=) at /usr/src-9-stable/sys/net/if_ethersubr.c:636 #9 0xffffffff809e2578 in netisr_dispatch_src (proto=9, source=, m=) at /usr/src-9-stable/sys/net/netisr.c:1013 #10 0xffffffff804dd248 in em_rxeof (rxr=0xfffffe0006056a00, count=98, done=0x0) at /usr/src-9-stable/sys/dev/e1000/if_em.c:4515 #11 0xffffffff804dd6f8 in em_handle_que (context=, pending=) at /usr/src-9-stable/sys/dev/e1000/if_em.c:1518 #12 0xffffffff8095f9b4 in taskqueue_run_locked (queue=0xfffffe000605fb00) at /usr/src-9-stable/sys/kern/subr_taskqueue.c:312 #13 0xffffffff80960966 in taskqueue_thread_loop (arg=) at /usr/src-9-stable/sys/kern/subr_taskqueue.c:501 #14 0xffffffff808e83af in fork_exit ( callout=0xffffffff80960920 , arg=0xffffff8000aae730, frame=0xffffff83428cdc40) at /usr/src-9-stable/sys/kern/kern_fork.c:988 #15 0xffffffff80c92d3e in fork_trampoline () at /usr/src-9-stable/sys/amd64/amd64/exception.S:602 #16 0x0000000000000000 in ?? () (kgdb) --089e013a16ec95f6f204db8d8750 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
The em panics continue. I've received them from both e= m0 and em1. Below is a new variation not related to tcp_do_segment. Let me = know if I can provide more info.

Thanks,

Nate

--- core.txt ---

<= div>
Fatal trap 9: general protection fault while in kernel mode
<= div>cpuid =3D 4; apic id =3D 32
instruction pointer =A0 =A0 =3D 0= x20:0xffffffff8097dc20
stack pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xffffff83428cd9b0
frame pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xffffff83428cd9c0
c= ode segment =A0 =A0 =A0 =A0 =A0 =A0=3D base 0x0, limit 0xfffff, type 0x1b
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =3D DPL 0, pres 1,= long 1, def32 0, gran 1
processor eflags =A0 =A0 =A0 =A0=3D interrupt enabled, resume, IOPL = =3D 0
current process =A0 =A0 =A0 =A0 =3D 0 (em0 que)
t= rap number =A0 =A0 =A0 =A0 =A0 =A0 =3D 9
panic: general protectio= n fault
cpuid =3D 4
KDB: stack backtrace:
#0 0xffffffff80952f46 at kdb_backtrace+0x66=
#1 0xffffffff8091a2de at panic+0x1ce
#2 0xffffffff80ca= 8b80 at trap_fatal+0x290
#3 0xffffffff80ca9391 at trap+0x241
#4 0xffffffff80c92813 at calltrap+0x8
#5 0xffffffff809cf257 = at bpf_mtap+0x37
#6 0xffffffff809d9549 at ether_nh_input+0x259
#7 0xffffffff809e2578 at netisr_dispatch_src+0x218
#8 0xf= fffffff804dd248 at em_rxeof+0x1c8
#9 0xffffffff804dd6f8 at em_handle_que+0x48
#10 0xffffffff80= 95f9b4 at taskqueue_run_locked+0x74
#11 0xffffffff80960966 at tas= kqueue_thread_loop+0x46
#12 0xffffffff808e83af at fork_exit+0x11f=
#13 0xffffffff80c92d3e at fork_trampoline+0xe
Uptime: 17h44m= 42s
Dumping 2486 out of 8158 MB:..1%..11%..21%..31%..41%..51%..61= %..71%..81%..91%

Reading symbols from /boot/kernel= /zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Readi= ng symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/ke= rnel/opensolaris.ko.symbols...done.
done.
Loaded symbol= s for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/ipl.ko...Reading symbols from /boot/= kernel/ipl.ko.symbols...done.
done.
Loaded symbols for = /boot/kernel/ipl.ko
Reading symbols from /boot/kernel/accf_http.k= o...Reading symbols from /boot/kernel/accf_http.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/accf_http.ko
Reading symbols from /boot/kernel/accf_data.ko...Reading symbols from /boo= t/kernel/accf_data.ko.symbols...done.
done.
Loaded symb= ols for /boot/kernel/accf_data.ko
#0 =A0doadump (textdump=3D<value optimized out>) at pcpu.h:229
229 =A0 =A0 pcpu.h: No such file or directory.
=A0 =A0 = =A0 =A0 in pcpu.h
(kgdb) #0 =A0doadump (textdump=3D<value= optimized out>) at pcpu.h:229
#1 =A00xffffffff80919db4 in kern_reboot (howto=3D260)
=A0 = =A0 at /usr/src-9-stable/sys/kern/kern_shutdown.c:449
#2 =A00xfff= fffff8091a2b7 in panic (fmt=3D0x1 <Address 0x1 out of bounds>)
<= div>=A0 =A0 at /usr/src-9-stable/sys/kern/kern_shutdown.c:637
#3 =A00xffffffff80ca8b80 in trap_fatal (frame=3D0x9, eva=3D<value o= ptimized out>)
=A0 =A0 at /usr/src-9-stable/sys/amd64/amd64/tr= ap.c:878
#4 =A00xffffffff80ca9391 in trap (frame=3D0xffffff83428c= d900)
=A0 =A0 at /usr/src-9-stable/sys/amd64/amd64/trap.c:605
#5 = =A00xffffffff80c92813 in calltrap ()
=A0 =A0 at /usr/src-9-stable= /sys/amd64/amd64/exception.S:228
#6 =A00xffffffff8097dc20 in m_le= ngth (m0=3D0x3b4d5ae18672812f, last=3D0x0)
=A0 =A0 at /usr/src-9-stable/sys/kern/uipc_mbuf.c:1459
#7 = =A00xffffffff809cf257 in bpf_mtap (bp=3D0xfffffe000605ec00,=A0
= =A0 =A0 m=3D0xfffffe0036056600) at /usr/src-9-stable/sys/net/bpf.c:2110
#8 =A00xffffffff809d9549 in ether_nh_input (m=3D<value optimized = out>)
=A0 =A0 at /usr/src-9-stable/sys/net/if_ethersubr.c:636
#9 = =A00xffffffff809e2578 in netisr_dispatch_src (proto=3D9,=A0
=A0 = =A0 source=3D<value optimized out>, m=3D<value optimized out>)<= /div>
=A0 =A0 at /usr/src-9-stable/sys/net/netisr.c:1013
#10 0xffffffff804dd248 in em_rxeof (rxr=3D0xfffffe0006056a00, count=3D= 98,=A0
=A0 =A0 done=3D0x0) at /usr/src-9-stable/sys/dev/e1000/if_= em.c:4515
#11 0xffffffff804dd6f8 in em_handle_que (context=3D<= value optimized out>,=A0
=A0 =A0 pending=3D<value optimized out>)
=A0 =A0 at /u= sr/src-9-stable/sys/dev/e1000/if_em.c:1518
#12 0xffffffff8095f9b4= in taskqueue_run_locked (queue=3D0xfffffe000605fb00)
=A0 =A0 at = /usr/src-9-stable/sys/kern/subr_taskqueue.c:312
#13 0xffffffff80960966 in taskqueue_thread_loop (arg=3D<value optim= ized out>)
=A0 =A0 at /usr/src-9-stable/sys/kern/subr_taskqueu= e.c:501
#14 0xffffffff808e83af in fork_exit (
=A0 =A0 c= allout=3D0xffffffff80960920 <taskqueue_thread_loop>,=A0
=A0 =A0 arg=3D0xffffff8000aae730, frame=3D0xffffff83428cdc40)
=A0 =A0 at /usr/src-9-stable/sys/kern/kern_fork.c:988
#15 0xfff= fffff80c92d3e in fork_trampoline ()
=A0 =A0 at /usr/src-9-stable/= sys/amd64/amd64/exception.S:602
#16 0x0000000000000000 in ?? ()
(kgdb)=A0
<= div>
--089e013a16ec95f6f204db8d8750-- From owner-freebsd-net@FreeBSD.ORG Tue Apr 30 13:22:41 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 890D5199; Tue, 30 Apr 2013 13:22:41 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) by mx1.freebsd.org (Postfix) with ESMTP id F128012B0; Tue, 30 Apr 2013 13:22:40 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hm14so3905415wib.17 for ; Tue, 30 Apr 2013 06:22:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=sYO21latCbIbOt1o85M1nLnmUwAEF7Sbkz1phOH5yaY=; b=fq858XOQl2yE/eC084HLYjlH624zqV0iOWOTUXiMUDg0fHiMnD/6mTAvJ9NJFdT7e4 MA+O+eCCu5g65IrDioYrvMUdfhzScbl53gEfgXIF9vydQG/jxiXgLDXQVZsSza0KMR6c bvfeFe6i6vD4u16UT2vnH6XEwqD20J/lf9BPPLAmQDZ0tAy1i/mnh3vnyduDQxpD/iZI j5yaEzgGFbaDpJ0FGoMfQhSAzZw4+X9JBJNneH8tFuXFbZWNEL4rwd05S9tsPUsQ2NCm 3Awzthi6HialsYJRY3A5A7VziFnVaEQoMZqU3srjk4pGlqJjqA/MWA3szFE+C7lnbxck fZ9Q== MIME-Version: 1.0 X-Received: by 10.180.94.196 with SMTP id de4mr24481232wib.23.1367328160126; Tue, 30 Apr 2013 06:22:40 -0700 (PDT) Received: by 10.194.82.138 with HTTP; Tue, 30 Apr 2013 06:22:40 -0700 (PDT) In-Reply-To: References: Date: Tue, 30 Apr 2013 09:22:40 -0400 Message-ID: Subject: Re: in_lltable_rtcheck From: Juan Mojica To: "Li, Qing" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , "qingli@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2013 13:22:41 -0000 Great! Thanks Qing. I'll come up with a patch and reply. -Juan On Mon, Apr 29, 2013 at 9:40 PM, Li, Qing wrote: > The problem you described here seemed familiar so I checked into the svn > history, > and found I have in fact fixed this issue in other parts of the code. > > Please see > http://svnweb.freebsd.org/base?view=revision&revision=186708 > > So I think similar fix should be applied here as well. > > --Qing > > > ________________________________________ > From: owner-freebsd-net@freebsd.org [owner-freebsd-net@freebsd.org] on > behalf of Juan Mojica [jmojica@gmail.com] > Sent: Monday, April 29, 2013 6:17 PM > To: FreeBSD Net > Subject: in_lltable_rtcheck > > Why is the code comparing the entire sockaddr structure instead of just the > relevant fields? We have a flood of the log message below when > transitioning an IP address from one port to another. And this triggers > other behavior as well. > > Through GDB, we can see that the addresses are in the same subnet. The > problem is that the sin_port port fields in the l3addr and in sa do not > match. > > Is there a reason sin_port should be compared here? > > I can come up with a patch, that's not an issue, but I wanted to confirm > with others first. > > > if (!(rt->rt_flags & RTF_HOST) && rt->rt_ifp != ifp) { > const char *sa, *mask, *addr, *lim; > int len; > > mask = (const char *)rt_mask(rt); > /* > * Just being extra cautious to avoid some custom > * code getting into trouble. > */ > if (mask == NULL) { > RTFREE_LOCKED(rt); > return (EINVAL); > } > > sa = (const char *)rt_key(rt); > addr = (const char *)l3addr; > len = ((const struct sockaddr_in *)l3addr)->sin_len; > lim = addr + len; > > for ( ; addr < lim; sa++, mask++, addr++) { > if ((*sa ^ *addr) & *mask) { > #ifdef DIAGNOSTIC > log(LOG_INFO, "IPv4 address: \"%s\" is not > on the network\n", > inet_ntoa(((const struct sockaddr_in > *)l3addr)->sin_addr)); > #endif > RTFREE_LOCKED(rt); > return (EINVAL); > } > } > } > > > Regards > > -- > Juan Mojica > Email: jmojica@gmail.com > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Juan Mojica Email: jmojica@gmail.com From owner-freebsd-net@FreeBSD.ORG Tue Apr 30 15:44:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DE43EDDD for ; Tue, 30 Apr 2013 15:44:12 +0000 (UTC) (envelope-from wind@sourcearmory.com) Received: from sender1.zohomail.com (sender1.zohomail.com [72.5.230.103]) by mx1.freebsd.org (Postfix) with ESMTP id B6A7B1AE5 for ; Tue, 30 Apr 2013 15:44:12 +0000 (UTC) Received: from localhost (host-181-114-67-151.supernet.com.bo [181.114.67.151]) by mx.zohomail.com with SMTPS id 1367335475070591.8895044950285; Tue, 30 Apr 2013 08:24:35 -0700 (PDT) Message-ID: <517FE225.6080703@sourcearmory.com> Date: Tue, 30 Apr 2013 11:24:21 -0400 From: wind@sourcearmory.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Capture packets before kernel process Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ZohoMailClient: External X-Zoho-Virus-Status: 2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2013 15:44:12 -0000 Hi! I need some help, currently I'm working in a project where I want to capture and process some network packets before the kernel. I have searched but I have found nothing. Is there some way to capture the packets before the kernel ? From owner-freebsd-net@FreeBSD.ORG Wed May 1 00:02:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A16A0B9F for ; Wed, 1 May 2013 00:02:08 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-oa0-f49.google.com (mail-oa0-f49.google.com [209.85.219.49]) by mx1.freebsd.org (Postfix) with ESMTP id 705FC1FC5 for ; Wed, 1 May 2013 00:02:08 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id j1so1059965oag.36 for ; Tue, 30 Apr 2013 17:02:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=lJeAsrYlM9zespFU7FF3Ot0SmBR7kyrPs/RscwA9k4U=; b=BLti+4tsmcT2mYozGpjaZkYIoj1U9vr4WRnJkM4td4OkN+uEfrf7CaD8YNwDp6IMxx XGtzuzrwfhQ/bvbQvkHJFZQz7/XRhCmJ+PXI8fu3cQeFW0oZEkj75z4R5iATNj0A66py QL+cdiHKVwtEYEnX23zkpTzWjw3ny1r93W0mAjdr8VbSr/3B2Kfi08p+J5JQM3ZBBOcU 6pp/jLStDTQVQdX8R4ih4MPFcP7AtcuhLEY+WoC2co8GGiMjGZkKsjTME54DiP/bB/A0 eGNaiL72KJRpY0jRmoTSOQ5FEoCVInZI9hRtBH3jrDE/7LnQjCmpwnLtkf3UuCNBSVdi +IlA== MIME-Version: 1.0 X-Received: by 10.182.84.135 with SMTP id z7mr144074oby.35.1367366527529; Tue, 30 Apr 2013 17:02:07 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.76.12.167 with HTTP; Tue, 30 Apr 2013 17:02:07 -0700 (PDT) In-Reply-To: <517FE225.6080703@sourcearmory.com> References: <517FE225.6080703@sourcearmory.com> Date: Tue, 30 Apr 2013 17:02:07 -0700 X-Google-Sender-Auth: q516WpsLutqykUgTuaIirs1Sg-Y Message-ID: Subject: Re: Capture packets before kernel process From: Kevin Oberman To: wind@sourcearmory.com Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 00:02:08 -0000 On Tue, Apr 30, 2013 at 8:24 AM, wrote: > Hi! > > I need some help, currently I'm working in a project where I want to > capture and process some network packets before the kernel. I have searched > but I have found nothing. > > Is there some way to capture the packets before the kernel ? > > This is a rather odd question. The device drivers which are the codes that do all direct communication with the interfaces are part of the kernel in most all operating systems. This is technically not required, but I have not run into an OS that did not work this way in many years. (Digital's IAS used user mode handlers to talk to interfaces, but it has been obsolete for a quarter century.) Even there, the kernel contained the basic interrupt routine (very simple) as a part of the kernel to hand the data to the handler. If you want to see the raw data, the PCAP code will capture the data very early after it is received by the kernel, but the kernel still must do this as it and only actually can "talk" to the interface and receive data. -- R. Kevin Oberman, Network Engineer E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Wed May 1 00:13:46 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 87EA3EA2 for ; Wed, 1 May 2013 00:13:46 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by mx1.freebsd.org (Postfix) with ESMTP id 12A841021 for ; Wed, 1 May 2013 00:13:45 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id o10so1067451lbi.3 for ; Tue, 30 Apr 2013 17:13:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :x-gm-message-state; bh=jsRzQ1p9tXGnNyvcKIQKA1rq71qVpl9u8/36YRZQG1Q=; b=QCmn3xzGWrQ4j2Z240owqb1yRiGimJZr2ulMFCzoYLeSEBvcCnC1a9FIR5xScvpTPD 4KWUzcB9VzXlRXKjwmMFjNb/jWcMw20ZCYHahW0k6FnzjgNaxjvt39Fq89B30RasrF0Z baMpyjY5PayG2b1yopMslCtVQBwnYZi7jHAWHj7lPwv2+1WwEf4RodMejPheM2IJRo5W w4FHmXlVlv4neyjqVzBX/xd/MxfmTRfFT8ulSIUZ0UB7IhV+8LQaambowLAP/yYGFZSR evLlCRqp4A45FJwKVHedsTImK6ZvQlAe+EdrEcVEKs3hQ8ENF9XQZoJyzZtHF8gA8tR9 fCYw== X-Received: by 10.112.199.230 with SMTP id jn6mr378805lbc.131.1367367218894; Tue, 30 Apr 2013 17:13:38 -0700 (PDT) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.152.28.196 with HTTP; Tue, 30 Apr 2013 17:13:18 -0700 (PDT) In-Reply-To: <517FE225.6080703@sourcearmory.com> References: <517FE225.6080703@sourcearmory.com> From: Juli Mallett Date: Tue, 30 Apr 2013 17:13:18 -0700 X-Google-Sender-Auth: zfmkB7SGepCSpizm8UUEZcTMgMM Message-ID: Subject: Re: Capture packets before kernel process To: wind@sourcearmory.com Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnqKKDDSaXZQTdjkWBC4PeygjAOFxdNe6ijsITV5z6gb66Y3fjb1VIfR1vgMJT7jM3ECbPe Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 00:13:46 -0000 On Tue, Apr 30, 2013 at 8:24 AM, wrote: > I need some help, currently I'm working in a project where I want to capture > and process some network packets before the kernel. I have searched but I > have found nothing. > > Is there some way to capture the packets before the kernel ? You probably should look at netmap, although you need supported hardware: - http://info.iet.unipi.it/~luigi/netmap/ It can even be configured so that you can pass some packets on to the regular kernel network stack, although it's not clear if that's something you want/need. Thanks, Juli. From owner-freebsd-net@FreeBSD.ORG Wed May 1 03:18:48 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DAC42786 for ; Wed, 1 May 2013 03:18:48 +0000 (UTC) (envelope-from wind@sourcearmory.com) Received: from sender1.zohomail.com (sender1.zohomail.com [72.5.230.100]) by mx1.freebsd.org (Postfix) with ESMTP id AFC3815C4 for ; Wed, 1 May 2013 03:18:48 +0000 (UTC) Received: from localhost (host-190-11-70-139.supernet.com.bo [190.11.70.139]) by mx.zohomail.com with SMTPS id 1367378327283880.9932561767561; Tue, 30 Apr 2013 20:18:47 -0700 (PDT) Message-ID: <51808987.4090800@sourcearmory.com> Date: Tue, 30 Apr 2013 23:18:31 -0400 From: wind@sourcearmory.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Capture packets before kernel process References: <517FE225.6080703@sourcearmory.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ZohoMailClient: External X-Zoho-Virus-Status: 2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 03:18:48 -0000 It looks like what I've been looking for, thanks Juli. From owner-freebsd-net@FreeBSD.ORG Wed May 1 15:10:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 413FA2D2 for ; Wed, 1 May 2013 15:10:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 32DA81363 for ; Wed, 1 May 2013 15:10:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r41FA1vo048645 for ; Wed, 1 May 2013 15:10:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r41FA1Ue048644; Wed, 1 May 2013 15:10:01 GMT (envelope-from gnats) Date: Wed, 1 May 2013 15:10:01 GMT Message-Id: <201305011510.r41FA1Ue048644@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Gleb Smirnoff Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Gleb Smirnoff List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 15:10:01 -0000 The following reply was made to PR kern/178116; it has been noted by GNATS. From: Gleb Smirnoff To: Nate Denning Cc: bug-followup@FreeBSD.org Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Wed, 1 May 2013 18:02:06 +0400 Nate, do you run any additional network modules: ipfw, pf, netgraph, accept filters, etc? How your system differes from a default installation? Is it possible for you to run with INVARIANTS option in the kernel? The option adds additional debugging, thus hurts system performance, but with it we can obtain a more informative crashdump. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed May 1 15:30:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 79A89853 for ; Wed, 1 May 2013 15:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 67C7214F3 for ; Wed, 1 May 2013 15:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r41FU18k053582 for ; Wed, 1 May 2013 15:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r41FU03a053581; Wed, 1 May 2013 15:30:00 GMT (envelope-from gnats) Date: Wed, 1 May 2013 15:30:00 GMT Message-Id: <201305011530.r41FU03a053581@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Nate Denning Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Nate Denning List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 15:30:01 -0000 The following reply was made to PR kern/178116; it has been noted by GNATS. From: Nate Denning To: Gleb Smirnoff Cc: bug-followup@FreeBSD.org Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Wed, 1 May 2013 09:26:04 -0600 On May 1, 2013, at 8:02 AM, Gleb Smirnoff wrote: > Nate, >=20 > do you run any additional network modules: ipfw, pf, netgraph, > accept filters, etc? How your system differes from a default > installation? >=20 Yes, ipfilter, accf_http and accf_data (accf is for Apache). No ipfw, = pf, or netgraph. Output of kldstat: Id Refs Address Size Name 1 15 0xffffffff80200000 1558e18 kernel 2 1 0xffffffff81759000 2324e0 zfs.ko 3 2 0xffffffff8198c000 84e8 opensolaris.ko 4 1 0xffffffff81a12000 330db ipl.ko 5 1 0xffffffff81a46000 163a accf_http.ko 6 1 0xffffffff81a48000 cda accf_data.ko IPv4 is configured natively and IPv6 over a gif tunnel, with ipfilter = rules setup for both. Other than all that I'm not seeing anything = related to networking that is not default. > Is it possible for you to run with INVARIANTS option in the kernel? > The option adds additional debugging, thus hurts system performance, > but with it we can obtain a more informative crashdump. >=20 Yes, I can try that. Thanks, Nate From owner-freebsd-net@FreeBSD.ORG Wed May 1 15:30:02 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 69940854 for ; Wed, 1 May 2013 15:30:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 5C35914F5 for ; Wed, 1 May 2013 15:30:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r41FU21s053589 for ; Wed, 1 May 2013 15:30:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r41FU2fA053588; Wed, 1 May 2013 15:30:02 GMT (envelope-from gnats) Date: Wed, 1 May 2013 15:30:02 GMT Message-Id: <201305011530.r41FU2fA053588@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Gleb Smirnoff Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Gleb Smirnoff List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 15:30:02 -0000 The following reply was made to PR kern/178116; it has been noted by GNATS. From: Gleb Smirnoff To: Nate Denning Cc: bug-followup@FreeBSD.org Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Wed, 1 May 2013 19:27:22 +0400 Nate, On Wed, May 01, 2013 at 09:26:04AM -0600, Nate Denning wrote: N> > do you run any additional network modules: ipfw, pf, netgraph, N> > accept filters, etc? How your system differes from a default N> > installation? N> N> Yes, ipfilter, accf_http and accf_data (accf is for Apache). No ipfw, pf, or netgraph. Output of kldstat: I would suspect ipfilter. :( Is it possible for you to rewrite your rules to ipfw or pf and try running with that? -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed May 1 15:40:01 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7207AA43 for ; Wed, 1 May 2013 15:40:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 63EAE1578 for ; Wed, 1 May 2013 15:40:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r41Fe1Kp055200 for ; Wed, 1 May 2013 15:40:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r41Fe1UA055199; Wed, 1 May 2013 15:40:01 GMT (envelope-from gnats) Date: Wed, 1 May 2013 15:40:01 GMT Message-Id: <201305011540.r41Fe1UA055199@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Nate Denning Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Nate Denning List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 15:40:01 -0000 The following reply was made to PR kern/178116; it has been noted by GNATS. From: Nate Denning To: Gleb Smirnoff Cc: bug-followup@FreeBSD.org Subject: Re: kern/178116: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Wed, 1 May 2013 09:32:08 -0600 On May 1, 2013, at 9:27 AM, Gleb Smirnoff wrote: > Nate, >=20 > On Wed, May 01, 2013 at 09:26:04AM -0600, Nate Denning wrote: > N> > do you run any additional network modules: ipfw, pf, netgraph, > N> > accept filters, etc? How your system differes from a default > N> > installation? > N>=20 > N> Yes, ipfilter, accf_http and accf_data (accf is for Apache). No = ipfw, pf, or netgraph. Output of kldstat: >=20 > I would suspect ipfilter. :( >=20 > Is it possible for you to rewrite your rules to ipfw or pf and try > running with that? >=20 Certainly, I'll switch to pf and see how that goes. Thanks, Nate From owner-freebsd-net@FreeBSD.ORG Wed May 1 23:16:11 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2D8102A2 for ; Wed, 1 May 2013 23:16:11 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm15-vm3.bullet.mail.ne1.yahoo.com (nm15-vm3.bullet.mail.ne1.yahoo.com [98.138.91.145]) by mx1.freebsd.org (Postfix) with ESMTP id CDE55186F for ; Wed, 1 May 2013 23:16:10 +0000 (UTC) Received: from [98.138.226.180] by nm15.bullet.mail.ne1.yahoo.com with NNFMP; 01 May 2013 23:13:44 -0000 Received: from [98.138.86.156] by tm15.bullet.mail.ne1.yahoo.com with NNFMP; 01 May 2013 23:13:44 -0000 Received: from [127.0.0.1] by omp1014.mail.ne1.yahoo.com with NNFMP; 01 May 2013 23:13:44 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 276788.40099.bm@omp1014.mail.ne1.yahoo.com Received: (qmail 52429 invoked by uid 60001); 1 May 2013 23:13:44 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1367450024; bh=JQbIeWoRCakAYxJVTe27Nh/CcV06IugibgEZ319zSGQ=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=E+aILbe6CmCQWzZ7fEKBuACNHM8zgU+9K6Vc0AvM54f4gN8mX2i/upEjwesKsbjq3gyaL9MhB5hfm6JXqranTy89GqS/76ncGBJpvumjhuwepbHflfpLNN/JiXVLScKPnGd+Qt24XkmXCgFGxlB3RRnVozmhbLgePOLQDP6vHEI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=BbGZ49Uz8UNHuwJA2yv/urr3/T/LaLgdii/a8u2iI2/pHqAkgJJszU/HGjBfNaeEksL+mbEASvJok9CQF+tSmXdugO6UP6PFB7J50YM7MKdNmsFcDT1CwIzOIJ988TNTGw2YoIlSW0IClagrVz7oFby1int+KFLewckWs3TeVDM=; X-YMail-OSG: 5IyDktYVM1mrB.NXLVUxQJDZQ0AxpL3VQqbar.MeKEKTZMd G6PmZ2fFULGzqvN86fmGcot374nK_vuLa2_VOp4BupIi66.FV_OLmkX.dBoS MDb8RnrQQW1u8i8BlKTtJ.REQyLPJJyZazMcPBKKi3UgbQyqUBRBv6hHaNf3 I.RsZ8QN1cW5eXy4Ffec6iiP4EwihPO7J9KyTbyi8aiJD.1LmlWAa.jNsJ8F MpilRHcLAmpVGO0G2OFCzy0_H51TQBbW5KFxGybDc6SjcLaZWdUlQre.aKrK Ck1.TReVSZO4U5xvmK.fRDKptdKcNlQHQEsjcR3doleFNpWkKAYN7sPNdd8w JsM0OfDgB5PtGYxbzwVS.Ir8jIUOGlmnNQIsU34q5KzXI_4MDtKi_xBONsIx NjnKGg2TgVaof2P5o55DDwiLYOoym9AKYMrSaCvByFQg89Nuq22SI4a8EVK8 7C6EnUx.VH9XzHMac8aQ_5eFA3oppa8TyGzBDAUucqoMBOD3q7biPfOblxQo OEwY2G3g- Received: from [98.203.118.124] by web121605.mail.ne1.yahoo.com via HTTP; Wed, 01 May 2013 16:13:43 PDT X-Rocket-MIMEInfo: 002.001, Ci0tLSBPbiBUdWUsIDQvMzAvMTMsIHdpbmRAc291cmNlYXJtb3J5LmNvbSA8d2luZEBzb3VyY2Vhcm1vcnkuY29tPiB3cm90ZToKCj4gRnJvbTogd2luZEBzb3VyY2Vhcm1vcnkuY29tIDx3aW5kQHNvdXJjZWFybW9yeS5jb20.Cj4gU3ViamVjdDogQ2FwdHVyZSBwYWNrZXRzIGJlZm9yZSBrZXJuZWwgcHJvY2Vzcwo.IFRvOiBmcmVlYnNkLW5ldEBmcmVlYnNkLm9yZwo.IERhdGU6IFR1ZXNkYXksIEFwcmlsIDMwLCAyMDEzLCAxMToyNCBBTQo.IEhpIQo.IAo.IEkgbmVlZCBzb21lIGhlbHAsIGN1cnJlbnRseSABMAEBAQE- X-Mailer: YahooMailClassic/15.1.8 YahooMailWebService/0.8.141.536 Message-ID: <1367450023.38176.YahooMailClassic@web121605.mail.ne1.yahoo.com> Date: Wed, 1 May 2013 16:13:43 -0700 (PDT) From: Barney Cordoba Subject: Re: Capture packets before kernel process To: freebsd-net@freebsd.org, wind@sourcearmory.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 23:16:11 -0000 --- On Tue, 4/30/13, wind@sourcearmory.com wrote: > From: wind@sourcearmory.com > Subject: Capture packets before kernel process > To: freebsd-net@freebsd.org > Date: Tuesday, April 30, 2013, 11:24 AM > Hi! > > I need some help, currently I'm working in a project where I > want to capture and process some network packets before the > kernel. I have searched but I have found nothing. > > Is there some way to capture the packets before the kernel > ? You want to wedge your code to the if_input routine. Then pass the mbuf to the original if_input routine. BC From owner-freebsd-net@FreeBSD.ORG Thu May 2 03:03:25 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5FFF7665 for ; Thu, 2 May 2013 03:03:25 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) by mx1.freebsd.org (Postfix) with ESMTP id EDB791109 for ; Thu, 2 May 2013 03:03:24 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id u7so106697wey.16 for ; Wed, 01 May 2013 20:03:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=Y2pQC18xtJg2wBxKQYdoXN3dPVbaU/C0PUYpZcZeDm8=; b=HsFxewYouKmICc5z9LrLbd60EKSdnANwmdCweWXJJ6OaPUTA+sHixDiLTqMZNqqUVt 4X8hQp4yOOwhO9rJfB0+3P84t9s72piJODhMmwE7x1JCkJu+iRx23AbW4eGUxPCC2gBr wBwzqkZ7GWC3yMW74LSbFJZN/rnlz+ko3Dau15q22xmQ+rJHu0fPRF9ZUU62+lXyhThs Gs8AEO5wwWn/ZIy8gWJWaDo6GshpqHkjfBxQNY6YTsD27KTqlPWZu74kQ10h+vQRrQI+ zKnKXg/8UoZbmTM2OLJrLT0MKMngYpCdjHeQJuEJc4ZDaeWOjg1l5dJIfZBo5FG23RWJ AZNQ== MIME-Version: 1.0 X-Received: by 10.180.105.195 with SMTP id go3mr32338366wib.2.1367463804133; Wed, 01 May 2013 20:03:24 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Wed, 1 May 2013 20:03:24 -0700 (PDT) Date: Wed, 1 May 2013 20:03:24 -0700 Message-ID: Subject: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: freebsd-net@freebsd.org Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 03:03:25 -0000 Hi folks, I am checking to see if there are any known bugs with respect to this in FreeBSD 8.0. Situation is that Samba 3.6.6 uses writev to a non-blocking socket to get the SMB2 requests on the wire. Intermittently, we see the writev return EINVAL even though the data has gotten on the wire. This I have verified by grabbing a capture and comparing the SMB Sequence number in the last outgoing packet on the wire vs the in-memory contents when we get EINVAL. Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN on the four-element IOVEC and then we get EINVAL when retrying on a smaller IOVEC. Where should I look to check if there is some path where this might be happening? Is this even the correct mailing list? --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Thu May 2 03:28:58 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 28645950 for ; Thu, 2 May 2013 03:28:58 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id F3DD011C2 for ; Thu, 2 May 2013 03:28:57 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-237-17.lns20.per1.internode.on.net [121.45.237.17]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r423SqGN018386 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 1 May 2013 20:28:54 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5181DD6F.3090404@freebsd.org> Date: Thu, 02 May 2013 11:28:47 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Barney Cordoba Subject: Re: Capture packets before kernel process References: <1367450023.38176.YahooMailClassic@web121605.mail.ne1.yahoo.com> In-Reply-To: <1367450023.38176.YahooMailClassic@web121605.mail.ne1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, wind@sourcearmory.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 03:28:58 -0000 On 5/2/13 7:13 AM, Barney Cordoba wrote: > --- On Tue, 4/30/13, wind@sourcearmory.com wrote: > >> From: wind@sourcearmory.com >> Subject: Capture packets before kernel process >> To: freebsd-net@freebsd.org >> Date: Tuesday, April 30, 2013, 11:24 AM >> Hi! >> >> I need some help, currently I'm working in a project where I >> want to capture and process some network packets before the >> kernel. I have searched but I have found nothing. >> >> Is there some way to capture the packets before the kernel >> ? > You want to wedge your code to the if_input routine. Then pass the mbuf > to the original if_input routine. there is a netgraph hook there. man netgraph > > BC > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu May 2 04:35:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 116C9937 for ; Thu, 2 May 2013 04:35:05 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 017DC138D for ; Thu, 2 May 2013 04:35:04 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id DAAA91A3C1A; Wed, 1 May 2013 21:34:57 -0700 (PDT) Message-ID: <5181ECDF.1040905@mu.org> Date: Wed, 01 May 2013 21:34:39 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Richard Sharpe Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire References: In-Reply-To: Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 04:35:05 -0000 On 5/1/13 8:03 PM, Richard Sharpe wrote: > Hi folks, > > I am checking to see if there are any known bugs with respect to this > in FreeBSD 8.0. > > Situation is that Samba 3.6.6 uses writev to a non-blocking socket to > get the SMB2 requests on the wire. > > Intermittently, we see the writev return EINVAL even though the data > has gotten on the wire. This I have verified by grabbing a capture and > comparing the SMB Sequence number in the last outgoing packet on the > wire vs the in-memory contents when we get EINVAL. > > Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN > on the four-element IOVEC and then we get EINVAL when retrying on a > smaller IOVEC. > > Where should I look to check if there is some path where this might be > happening? Is this even the correct mailing list? > What does the iovec look like when you get EINVAL? Can you sanity check it? Is there anything special about it? (zero length vecs?) I think there are a few "maxvals" that if overrun cause EINVAL to be returned. example is if your iovec is somehow huge or has many, many elements. -Alfred From owner-freebsd-net@FreeBSD.ORG Thu May 2 04:47:24 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A24B7A5E for ; Thu, 2 May 2013 04:47:24 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) by mx1.freebsd.org (Postfix) with ESMTP id 3D3F313D8 for ; Thu, 2 May 2013 04:47:24 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id ey16so166603wid.8 for ; Wed, 01 May 2013 21:47:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=BT8YULLk2tqYZbjn+54NdnrZNlsnOqkKs22o1kKfLtU=; b=ghE/RNJbXYyRVW/0ARXSgcqE7EhcSBVQVWLqy7lxE0CM+hKqymrVdmultSr68WgzsU uyenof/240f000sA1Pn5dXYHKSXFSquonNIQvZAzjby98gGEeLbpDkPVRv7rUS0xftKJ GSsTtdisp0473QkfjqGm+zL+f30hTCEbIDbI0LcrQRuyTjGl9oLHyyFppeGqylqaznmO Sm++Pk9s2RR77E5VtFtEUmK5nV2/a6XiB5uH8OlJdOjduFrcmjp+eqxEOyntOVW3QIop 7zPwSzHF762qKviiAWrluWQdAdyZD2LrNeSBtdTBEFBUAl/zF8bVxmDPrkWxo5ioLLvX TSbQ== MIME-Version: 1.0 X-Received: by 10.194.11.70 with SMTP id o6mr5582448wjb.29.1367470043486; Wed, 01 May 2013 21:47:23 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Wed, 1 May 2013 21:47:23 -0700 (PDT) In-Reply-To: <5181ECDF.1040905@mu.org> References: <5181ECDF.1040905@mu.org> Date: Wed, 1 May 2013 21:47:23 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Alfred Perlstein Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 04:47:24 -0000 On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: > On 5/1/13 8:03 PM, Richard Sharpe wrote: >> Hi folks, >> >> I am checking to see if there are any known bugs with respect to this >> in FreeBSD 8.0. >> >> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >> get the SMB2 requests on the wire. >> >> Intermittently, we see the writev return EINVAL even though the data >> has gotten on the wire. This I have verified by grabbing a capture and >> comparing the SMB Sequence number in the last outgoing packet on the >> wire vs the in-memory contents when we get EINVAL. >> >> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >> on the four-element IOVEC and then we get EINVAL when retrying on a >> smaller IOVEC. >> >> Where should I look to check if there is some path where this might be >> happening? Is this even the correct mailing list? >> > What does the iovec look like when you get EINVAL? Can you sanity check > it? Is there anything special about it? (zero length vecs?) > > I think there are a few "maxvals" that if overrun cause EINVAL to be > returned. example is if your iovec is somehow huge or has many, many > elements. It usually consists of four elements. All of the elements are accessible because I checked some of the data. Well, actually, I did not check the 65536 bytes of the last element, but they seemed to go out on the wire, although I should check the last offset. The lengths were usually: 4, 64, 8 or 16, 65536. So, I do need to check the last element. I think it all went out on the wire. With 4,64,16 and 65536 byte segments that is 65620 bytes and they were all accounted for and the SMB2 sequence number matched what was in memory. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Thu May 2 07:21:11 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3446AC15 for ; Thu, 2 May 2013 07:21:11 +0000 (UTC) (envelope-from ze87a1b827serrofq-arg=serrofq.bet@bounce.twitter.com) Received: from ham-cannon.twitter.com (ham-cannon.twitter.com [199.59.148.235]) by mx1.freebsd.org (Postfix) with ESMTP id 1D7061A6F for ; Thu, 2 May 2013 07:21:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; d=twitter.com; s=dkim-201303; c=relaxed/relaxed; q=dns/txt; i=@twitter.com; t=1367479265; h=From:Subject:Date:To; bh=/N5Z6lnDfVxyY5y3vWC+gK3MUY0=; b=iTAJPXLbLm5Qwxlf9lmVz2E+ImM98QB5JJpAU3iCGjl+hkJQWiZdDpdYclBerlE3 dWUk2HPfHRVihFA9Z7TjyZu5egg2MA0j/ONGY89FTG6R2uLvkkgH/XbCP5PU8AeV C7+grHW8MAAnMXI6cPg9fzhgclE3WV6tOwn81uX8WfA=; X-MSFBL: ZnJlZWJzZC1uZXRAZnJlZWJzZC5vcmdAc21mMS1iZmYtMDYtc3IxLTE2NUBFdmVy eXRoaW5nQA== Date: Thu, 02 May 2013 07:21:05 +0000 From: "jabbaarbarelly (via Twitter)" To: freebsd-net@freebsd.org Subject: jabbaarbarelly sent you an invitation MIME-Version: 1.0 Message-Id: <20130502072111.3446AC15@hub.freebsd.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 07:21:11 -0000 jabbaarbarelly sent you an invitation Twitter helps you stay connected with what's happening right now and with the people and organizations you care about. Accept invitation https://twitter.com/i/6332863a-e7fd-4844-9401-c5bff963cc81 ------------------------ This message was sent by Twitter on behalf of Twitter users who entered your email address to invite you to Twitter. Unsubscribe: https://twitter.com/i/o?t=1&iid=c52b3013-5d30-4d50-8302-8b3d136674f4&uid=0&c=ZNV%2BN6G7N7ilWCy7kazRVC1CjOBTEr2%2B&nid=9+26 Need help? https://support.twitter.com From owner-freebsd-net@FreeBSD.ORG Thu May 2 13:48:42 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DBDBCDFA for ; Thu, 2 May 2013 13:48:42 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) by mx1.freebsd.org (Postfix) with ESMTP id 7684C1DAD for ; Thu, 2 May 2013 13:48:42 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id k13so579527wgh.31 for ; Thu, 02 May 2013 06:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=umIcw3C+uJGn0/mLhIApaJ3h6SS6XXfklbRNezFxi0Y=; b=NvJ9sXJBXr4VCcCTM/6hw2UaW3Tc2bfkq/18ImPtvOpSS7T7Q0rYxPbyYpGCUu+mOt 8AVFGh2sG+Qn/KyFqReX7+qLc3VJPl3gfjc1atvQVG12rcsHnVEDT+zLqPLi/Q0/nPQ7 viBncvFjR9i/DERB5dGa8As42Xw0ns+UsFvqUBQFZlHxHTZeKDm4OfXw+tHXPTvJKrhI crnneVx1PJJzwKOGiul6n0geQ5kiW3DM4iEHq66jvW7Xqvl4qk8SNq8sKNWQsOl2FJ4S ycK5uY9Z7Ceovh/i1BLuIrTbIohLY1xbMz7RqFfokBDvtxzsNpimSyixC0EMw8IA9HSZ GTFw== MIME-Version: 1.0 X-Received: by 10.180.90.203 with SMTP id by11mr7897378wib.10.1367502521672; Thu, 02 May 2013 06:48:41 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Thu, 2 May 2013 06:48:41 -0700 (PDT) In-Reply-To: <5181ECDF.1040905@mu.org> References: <5181ECDF.1040905@mu.org> Date: Thu, 2 May 2013 06:48:41 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Alfred Perlstein Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 13:48:42 -0000 On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: > On 5/1/13 8:03 PM, Richard Sharpe wrote: >> Hi folks, >> >> I am checking to see if there are any known bugs with respect to this >> in FreeBSD 8.0. >> >> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >> get the SMB2 requests on the wire. >> >> Intermittently, we see the writev return EINVAL even though the data >> has gotten on the wire. This I have verified by grabbing a capture and >> comparing the SMB Sequence number in the last outgoing packet on the >> wire vs the in-memory contents when we get EINVAL. >> >> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >> on the four-element IOVEC and then we get EINVAL when retrying on a >> smaller IOVEC. >> >> Where should I look to check if there is some path where this might be >> happening? Is this even the correct mailing list? >> > What does the iovec look like when you get EINVAL? Can you sanity check > it? Is there anything special about it? (zero length vecs?) > > I think there are a few "maxvals" that if overrun cause EINVAL to be > returned. example is if your iovec is somehow huge or has many, many > elements. Can anyone tell me the call graph down to the TCP code? --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Thu May 2 14:20:17 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5329430D; Thu, 2 May 2013 14:20:17 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) by mx1.freebsd.org (Postfix) with ESMTP id A4F0C1F7F; Thu, 2 May 2013 14:20:16 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hq12so657534wib.1 for ; Thu, 02 May 2013 07:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=B9f1BRXp+FrZmyhbzAjwFKIPKS1cONtUo/kiyiSd9Do=; b=xmtHlgf1c4NOJd+95D4QA1ZmkeZqp/AjA1GI6VZQ9WsI0UT2bmktdDRqo67w9xDGbC WQ21Eih32OxVr0B6/ttprHh6FVcdDtoVPC1MKZHJWoPwRvZgSJiHY5EPSXx9vChWYX1d l5a8YZZEG7EynWSxFdUbtS80PpAVVyfSBmbGTcZH9J7iBCF6MPYA/7zwJhQRc117iSL5 6wsHlERXz277R5l6qKiHEB8f+P1kUqou9BRvV28zgfi1kZI3hJwltineviVATv/m9CH4 HW2bpQnUuR2thSkVmHU1aznmCPRNJ5l4fboB9uaMsZrW99rPhJYNeEiAo1xF6M6Y0NlE mAOw== MIME-Version: 1.0 X-Received: by 10.194.109.227 with SMTP id hv3mr8273405wjb.32.1367504415684; Thu, 02 May 2013 07:20:15 -0700 (PDT) Received: by 10.194.82.138 with HTTP; Thu, 2 May 2013 07:20:15 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 May 2013 10:20:15 -0400 Message-ID: Subject: Re: in_lltable_rtcheck From: Juan Mojica To: "Li, Qing" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , "qingli@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 14:20:17 -0000 It looks like the same/similar problem is in sys/net/if.c struct ifaddr * ifaof_ifpforaddr(struct sockaddr *addr, struct ifnet *ifp) ... cp = addr->sa_data; cp2 = ifa->ifa_addr->sa_data; cp3 = ifa->ifa_netmask->sa_data; cplim = ifa->ifa_netmask->sa_len + (char *)ifa->ifa_netmask; for (; cp3 < cplim; cp3++) if ((*cp++ ^ *cp2++) & *cp3) break; if (cp3 == cplim) goto done; ... This is intended to be to be agnostic to the v3 protocol, but the unintended consequence is that it compares the in_port_t field as well. -Juan On Tue, Apr 30, 2013 at 9:22 AM, Juan Mojica wrote: > Great! Thanks Qing. I'll come up with a patch and reply. > > -Juan > > > On Mon, Apr 29, 2013 at 9:40 PM, Li, Qing wrote: > >> The problem you described here seemed familiar so I checked into the svn >> history, >> and found I have in fact fixed this issue in other parts of the code. >> >> Please see >> http://svnweb.freebsd.org/base?view=revision&revision=186708 >> >> So I think similar fix should be applied here as well. >> >> --Qing >> >> >> ________________________________________ >> From: owner-freebsd-net@freebsd.org [owner-freebsd-net@freebsd.org] on >> behalf of Juan Mojica [jmojica@gmail.com] >> Sent: Monday, April 29, 2013 6:17 PM >> To: FreeBSD Net >> Subject: in_lltable_rtcheck >> >> Why is the code comparing the entire sockaddr structure instead of just >> the >> relevant fields? We have a flood of the log message below when >> transitioning an IP address from one port to another. And this triggers >> other behavior as well. >> >> Through GDB, we can see that the addresses are in the same subnet. The >> problem is that the sin_port port fields in the l3addr and in sa do not >> match. >> >> Is there a reason sin_port should be compared here? >> >> I can come up with a patch, that's not an issue, but I wanted to confirm >> with others first. >> >> >> if (!(rt->rt_flags & RTF_HOST) && rt->rt_ifp != ifp) { >> const char *sa, *mask, *addr, *lim; >> int len; >> >> mask = (const char *)rt_mask(rt); >> /* >> * Just being extra cautious to avoid some custom >> * code getting into trouble. >> */ >> if (mask == NULL) { >> RTFREE_LOCKED(rt); >> return (EINVAL); >> } >> >> sa = (const char *)rt_key(rt); >> addr = (const char *)l3addr; >> len = ((const struct sockaddr_in *)l3addr)->sin_len; >> lim = addr + len; >> >> for ( ; addr < lim; sa++, mask++, addr++) { >> if ((*sa ^ *addr) & *mask) { >> #ifdef DIAGNOSTIC >> log(LOG_INFO, "IPv4 address: \"%s\" is >> not on the network\n", >> inet_ntoa(((const struct sockaddr_in >> *)l3addr)->sin_addr)); >> #endif >> RTFREE_LOCKED(rt); >> return (EINVAL); >> } >> } >> } >> >> >> Regards >> >> -- >> Juan Mojica >> Email: jmojica@gmail.com >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Juan Mojica > Email: jmojica@gmail.com > -- Juan Mojica Email: jmojica@gmail.com From owner-freebsd-net@FreeBSD.ORG Thu May 2 14:53:37 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 70461D45 for ; Thu, 2 May 2013 14:53:37 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from aussmtpmrkpc120.us.dell.com (aussmtpmrkpc120.us.dell.com [143.166.82.159]) by mx1.freebsd.org (Postfix) with ESMTP id 45B331124 for ; Thu, 2 May 2013 14:53:36 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.87,597,1363150800"; d="scan'208";a="29489904" Message-ID: <51827DAA.2020009@vangyzen.net> Date: Thu, 2 May 2013 09:52:26 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Richard Sharpe Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire References: <5181ECDF.1040905@mu.org> In-Reply-To: Content-Type: text/plain; charset="Big5" Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 14:53:37 -0000 On 05/02/2013 08:48, Richard Sharpe wrote: > On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>> Hi folks, >>> >>> I am checking to see if there are any known bugs with respect to this >>> in FreeBSD 8.0. >>> >>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>> get the SMB2 requests on the wire. >>> >>> Intermittently, we see the writev return EINVAL even though the data >>> has gotten on the wire. This I have verified by grabbing a capture and >>> comparing the SMB Sequence number in the last outgoing packet on the >>> wire vs the in-memory contents when we get EINVAL. >>> >>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>> on the four-element IOVEC and then we get EINVAL when retrying on a >>> smaller IOVEC. >>> >>> Where should I look to check if there is some path where this might be >>> happening? Is this even the correct mailing list? >>> >> What does the iovec look like when you get EINVAL? Can you sanity check >> it? Is there anything special about it? (zero length vecs?) >> >> I think there are a few "maxvals" that if overrun cause EINVAL to be >> returned. example is if your iovec is somehow huge or has many, many >> elements. > Can anyone tell me the call graph down to the TCP code? > writev kern/sys_generic.c kern_writev dofilewrite fo_write in sys/file.h soo_write in kern/sys_socket.c sosend in kern/uipc_socket.c sosend_generic tcp_usr_send in netinet/tcp_usrreq.c Eric From owner-freebsd-net@FreeBSD.ORG Thu May 2 16:03:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id ED69A882 for ; Thu, 2 May 2013 16:03:22 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) by mx1.freebsd.org (Postfix) with ESMTP id 84CE91682 for ; Thu, 2 May 2013 16:03:22 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id l13so766634wie.12 for ; Thu, 02 May 2013 09:03:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=b9lRJMzukmMiIhbCW0MRzZ6ZQFeOzcbPqCZ+55wjQIA=; b=dG4GRYAEBEp0nLWffXfJDXG0D1AkN9kG/rS1E/tMI13kUbT8il0w0QY+rmOBAuWKPq LQw8ZZCYSQzkaN1a+GbKOtEmRFqwpbyI24jIEvB6N9rV9jaPpIlmcoyDlKdp9NHcTQqT QyMjKy7nb75DJddJm+eEDEii+Wrv/35KbVNVpMTOkD7QtU6hxyG1ghFyuKdkNMzMJ2wx 8jLIAwMnFVaOIqCSZYDDWLB1Z3uP3xgynzC12QQQHvUb6iGA/VyZFp5Ribm4JP2DyjDx 27qXpqh3wSz4dabmeTPw/qhYyVv8HNJ0PHDlXcmBOjoKryIrDaOFVVz4Yk60DygZGaaY XvBg== MIME-Version: 1.0 X-Received: by 10.180.21.167 with SMTP id w7mr8731060wie.2.1367510601592; Thu, 02 May 2013 09:03:21 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Thu, 2 May 2013 09:03:21 -0700 (PDT) In-Reply-To: <51827DAA.2020009@vangyzen.net> References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> Date: Thu, 2 May 2013 09:03:21 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Eric van Gyzen Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 16:03:23 -0000 On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote: > On 05/02/2013 08:48, Richard Sharpe wrote: >> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>> Hi folks, >>>> >>>> I am checking to see if there are any known bugs with respect to this >>>> in FreeBSD 8.0. >>>> >>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>>> get the SMB2 requests on the wire. >>>> >>>> Intermittently, we see the writev return EINVAL even though the data >>>> has gotten on the wire. This I have verified by grabbing a capture and >>>> comparing the SMB Sequence number in the last outgoing packet on the >>>> wire vs the in-memory contents when we get EINVAL. >>>> >>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>> smaller IOVEC. >>>> >>>> Where should I look to check if there is some path where this might be >>>> happening? Is this even the correct mailing list? >>>> >>> What does the iovec look like when you get EINVAL? Can you sanity check >>> it? Is there anything special about it? (zero length vecs?) >>> >>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>> returned. example is if your iovec is somehow huge or has many, many >>> elements. >> Can anyone tell me the call graph down to the TCP code? >> > > writev kern/sys_generic.c > kern_writev > dofilewrite > fo_write in sys/file.h > soo_write in kern/sys_socket.c > sosend in kern/uipc_socket.c > sosend_generic > tcp_usr_send in netinet/tcp_usrreq.c Thanks for that. The only places it seems that EINVAL could be coming from are the last two, and they seem to be valid error cases and one of those seem like it could never happen in this path (kern_writev sends down flags set to zero). Since it has the feel of a race, I need to find out of we have a race with signal handlers in some way and whether or not we are doing something weird with control messages or something on the socket. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Fri May 3 00:00:25 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 82E6744A for ; Fri, 3 May 2013 00:00:25 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) by mx1.freebsd.org (Postfix) with ESMTP id 1DF9F1CE2 for ; Fri, 3 May 2013 00:00:24 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id y10so1146030wgg.20 for ; Thu, 02 May 2013 17:00:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=Isqf5lGTNSXTCsNkgvqIK/dCZ8ZG20o/OrtwmrPSfE8=; b=HZHpDQ4GkOS5lJQ9HzD8WwijtbStR7kGoPrxQqvgOuQzrSvPdh+yz0N8gWrgCVEOdv 4mo0r3eTzkeKvxeliFqQIVJvsp8eCMt4Z98fT8/fOCQW9Sla++ZH7gHau0VbzeMMWyCZ HJSz4lTKUnOB7Ad+AIXO9VBwBeOTTfpUUuMR/0vgMW3qY5E+b6ZPJHbiI7sV++wk6Dau aNhinOYXbYixy6zSCNRIdfvhjBKfPlwHxPNxJRmLfCkj/fmNZQdXborA3xC/DsQPwfa+ tEKoFDPBspQj6G6Gh4M6sTyoFRwmt3sM0On0OF4OpfeKLcdpe8y4Atypiv76nPueVhn/ 57BA== MIME-Version: 1.0 X-Received: by 10.194.11.70 with SMTP id o6mr10852405wjb.29.1367539223875; Thu, 02 May 2013 17:00:23 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Thu, 2 May 2013 17:00:23 -0700 (PDT) In-Reply-To: <51827DAA.2020009@vangyzen.net> References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> Date: Thu, 2 May 2013 17:00:23 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Eric van Gyzen Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 00:00:25 -0000 On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote: > On 05/02/2013 08:48, Richard Sharpe wrote: >> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>> Hi folks, >>>> >>>> I am checking to see if there are any known bugs with respect to this >>>> in FreeBSD 8.0. >>>> >>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>>> get the SMB2 requests on the wire. >>>> >>>> Intermittently, we see the writev return EINVAL even though the data >>>> has gotten on the wire. This I have verified by grabbing a capture and >>>> comparing the SMB Sequence number in the last outgoing packet on the >>>> wire vs the in-memory contents when we get EINVAL. >>>> >>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>> smaller IOVEC. >>>> >>>> Where should I look to check if there is some path where this might be >>>> happening? Is this even the correct mailing list? >>>> >>> What does the iovec look like when you get EINVAL? Can you sanity check >>> it? Is there anything special about it? (zero length vecs?) >>> >>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>> returned. example is if your iovec is somehow huge or has many, many >>> elements. >> Can anyone tell me the call graph down to the TCP code? >> > > writev kern/sys_generic.c > kern_writev > dofilewrite > fo_write in sys/file.h > soo_write in kern/sys_socket.c > sosend in kern/uipc_socket.c > sosend_generic > tcp_usr_send in netinet/tcp_usrreq.c Is there a tool that generates call graphs? I have been able to demonstrate that I am getting EINVAL returned from writev kern/sys_generic.c, kern_writev, dofilewrite and soo_write, but when I add printfs to sosend/sosend_generic it becomes very hard to provoke this problem. I wonder if the compiler is generating code that allows some functions to leave variables in registers but they are not treated as volatile. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Fri May 3 00:14:43 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A950C5F0 for ; Fri, 3 May 2013 00:14:43 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) by mx1.freebsd.org (Postfix) with ESMTP id 823231D69 for ; Fri, 3 May 2013 00:14:43 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id mc17so615805pbc.28 for ; Thu, 02 May 2013 17:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=8/AetLsgEILFUCe34wuAZn1qr9Fwm4EBh7BvXKub7+o=; b=bajKZ4rgvWnFjjAgNjkaLN9+oAV/9Tku715hGtmTLNP9C+eRB8J4InosjUvLeUYQq7 3335k0K2BGAYtweDbSfeTwjJtgMCF+8Pz37tVa68rq8X+cM5Eu7gkS9lgXZ9uiNGuBeA cj7fud/9+yN/ce3pdSx92IbnJahO63LesJlWw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=8/AetLsgEILFUCe34wuAZn1qr9Fwm4EBh7BvXKub7+o=; b=dSUiNrck3YCgwDuJOmMpCfyTEAQKF+yMKZdPs5qq8hyXBOeBmQS43Q0F4xPqzspKo1 EVFiXIrZy6kK6sJr4gpMHv+sbr9g5Za0O7i9+N2CtHb1AzfLp9evza4IZSKxX9UbiMt4 m1UCZ/eT9wtwPElGEj3qcT0LIV5jHuhCs7NItLhYkRSSR1BJiXTlr456Q8Hck9/9hcoA 570uqVZXocCBh3XQmE+/A7rMZM6jkfNiZTtRpmcijAoBq6WAUA5CFShFLfslvFsUxufc ezZCAsRD3PR7rR3hdD++pTfMZEG1RCVrCsv/+rhegqLrc2A1Y4AYM2VsNHl5J5ksbHSV zZOA== X-Received: by 10.68.220.106 with SMTP id pv10mr11519204pbc.52.1367540083189; Thu, 02 May 2013 17:14:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.159.97 with HTTP; Thu, 2 May 2013 17:14:12 -0700 (PDT) In-Reply-To: References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> From: Eitan Adler Date: Thu, 2 May 2013 20:14:12 -0400 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire To: Richard Sharpe Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlSrpahw05K/pq+hzvj9hvpn+GIoDbkC3yB4YlRfuaNRb6NDB6Dk2vcf5QR8s+hzvLKXUhi Cc: freebsd-net@freebsd.org, Eric van Gyzen , Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 00:14:43 -0000 On 2 May 2013 20:00, Richard Sharpe wrote: > On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote: >> On 05/02/2013 08:48, Richard Sharpe wrote: >>> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >>>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>>> Hi folks, >>>>> >>>>> I am checking to see if there are any known bugs with respect to this >>>>> in FreeBSD 8.0. >>>>> >>>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>>>> get the SMB2 requests on the wire. >>>>> >>>>> Intermittently, we see the writev return EINVAL even though the data >>>>> has gotten on the wire. This I have verified by grabbing a capture and >>>>> comparing the SMB Sequence number in the last outgoing packet on the >>>>> wire vs the in-memory contents when we get EINVAL. >>>>> >>>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>>> smaller IOVEC. >>>>> >>>>> Where should I look to check if there is some path where this might be >>>>> happening? Is this even the correct mailing list? >>>>> >>>> What does the iovec look like when you get EINVAL? Can you sanity check >>>> it? Is there anything special about it? (zero length vecs?) >>>> >>>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>>> returned. example is if your iovec is somehow huge or has many, many >>>> elements. >>> Can anyone tell me the call graph down to the TCP code? >>> >> >> writev kern/sys_generic.c >> kern_writev >> dofilewrite >> fo_write in sys/file.h >> soo_write in kern/sys_socket.c >> sosend in kern/uipc_socket.c >> sosend_generic >> tcp_usr_send in netinet/tcp_usrreq.c > > Is there a tool that generates call graphs? gprof in base valgrind --callgrind doxygen can also do this. there are probably others. -- Eitan Adler From owner-freebsd-net@FreeBSD.ORG Fri May 3 00:18:40 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id AA6A26AB for ; Fri, 3 May 2013 00:18:40 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-da0-x233.google.com (mail-da0-x233.google.com [IPv6:2607:f8b0:400e:c00::233]) by mx1.freebsd.org (Postfix) with ESMTP id 826291D87 for ; Fri, 3 May 2013 00:18:40 +0000 (UTC) Received: by mail-da0-f51.google.com with SMTP id h15so504638dan.10 for ; Thu, 02 May 2013 17:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=D9haoijPIEOYLLIm1TqC0uk5Go4wMJGsQb2857+kk9Q=; b=pZR4vWs5XabpJ8FJTfrWOVxzBASTzhoLLSwEwL8+QJVpfh0uxBMDj3qugHYxkdxROx NxfULwvHNQjVjFyUhoXGpQeeKPgaIiNNobtMuaPqohh57ChhVFX10lYW0VqWYuZUJ3fK LrI5rpY/M5P9chpTiIIdWNqRNH8SKuGYatBn4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=D9haoijPIEOYLLIm1TqC0uk5Go4wMJGsQb2857+kk9Q=; b=YE/ZPBJSG1EiWTV67l/wngeUr4XAsWPsNotjrXzFLbLkdkaMqejIHePtPFT2cF4/Gx KRoVN3niWhphzt6IfvtTCCDdowwE7Ove2nln5VcPxlFSDE9q+DrdOYgIfpJ9o8UXIJsM QWnGpaYg01t1Q48UTsxRbf2eRxQta7DgLSEzOZPFUHm8ZxIBPK5fiOKCIPd+7vvhN2dt QQnvnZvGBMfGxlNTYoY0fNGgL6K4+PMdAIXqCWKvusVkk/NKNBWCaWI5iXyc0TpIhYSE oeC6B2AhX6duMWjBJ81Jgp3Anjr0QcWiH+vCP8vXUt721Ss/zxw3PfOzxdlfhlPPNp2I RhBw== X-Received: by 10.68.97.130 with SMTP id ea2mr11079703pbb.129.1367540320185; Thu, 02 May 2013 17:18:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.159.97 with HTTP; Thu, 2 May 2013 17:18:10 -0700 (PDT) In-Reply-To: References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> From: Eitan Adler Date: Thu, 2 May 2013 20:18:10 -0400 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire To: Richard Sharpe Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQkkfvSQTiSHxVNBEtzCL3RZJBSAxtOx0dPUSbY/f5UsgVMHHd0OlQPSF81v/NqqvfNilqHr Cc: freebsd-net@freebsd.org, Eric van Gyzen , Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 00:18:40 -0000 On 2 May 2013 20:14, Eitan Adler wrote: > gprof in base > valgrind --callgrind > doxygen can also do this. > I was reminded this was about the kernel, not userland, sorry for the noise. -- Eitan Adler From owner-freebsd-net@FreeBSD.ORG Fri May 3 07:28:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 828846D4 for ; Fri, 3 May 2013 07:28:09 +0000 (UTC) (envelope-from arisangelo33@gmail.com) Received: from mail-la0-x243.google.com (mail-la0-x243.google.com [IPv6:2a00:1450:4010:c03::243]) by mx1.freebsd.org (Postfix) with ESMTP id 151361AE2 for ; Fri, 3 May 2013 07:28:08 +0000 (UTC) Received: by mail-la0-f67.google.com with SMTP id ee20so444225lab.10 for ; Fri, 03 May 2013 00:28:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=PUUuns20OG/ofi6fwy160wRgaJg0SchaVQsX3roUvo4=; b=pGEXetahvLttUAgyzZdaGCzYZwT6QxXkeVS/z1s4vxzZ80zBB/V/MjNS8BWxxnWCur mj+Rr4JjrV9LEWArzJSEEIX3GRqdFlg1Xx9vpo3qmB/v+fqtee1rMAe8z0MPHt28Oq+E veG+xvfbZgCQJgxtqnLrzMfoW3OouKHhh6s37wEbBWAU5IvK7dK1H52VIIFMQ7j5zUK1 m37arv8UBPAqv0645am6v2BkJ+tL1VLt4DwRGCrfyJr6b629nIF3rc08a4HNk4QzCEZn Ou7qjw+z/Ps2nzORDxfy3C5fIz5VZVe3ZxT3/yu0AkH8ZFj2XGLq+NqxJnWwuRKH3mYE znWw== MIME-Version: 1.0 X-Received: by 10.112.201.196 with SMTP id kc4mr3898179lbc.24.1367566087960; Fri, 03 May 2013 00:28:07 -0700 (PDT) Received: by 10.112.202.9 with HTTP; Fri, 3 May 2013 00:28:07 -0700 (PDT) Date: Fri, 3 May 2013 09:28:07 +0200 Message-ID: Subject: Calculation of inflight data From: Aris Angelo To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 07:28:09 -0000 Hi, I am trying to implement an extension to the FreeBSD TCP stack. In order to do that, I have a question regarding the calculation of the "pipe" variable, the amount of data that the sender calculates as being inflight. I am puzzled for the case when no SACK is negotiated and used. My idea would be that in this case the following is correct (during a partial ack): pipe = tp->snd_max - th->th_ack; But when looking at the tcp_output code, I can see that the off variable, which is used as pipe to determine later how much data to send ( len = snd_cwnd- off); ), is calculated as: off = tp->snd_nxt - tp->snd_una; Obviously snd_una is used since there is no info on tcp_output for the ack header, but I think using more up to date information is better (although less data would be injected to the network). But why is snd_nxt instead of snd_max used? In case of a partial ack, snd_nxt is readjusted for retransmit, that means, that it's closer to snd_una. What is your opinion, how should this variable be calculated? Thank you, Aris Angelogiannopoulos From owner-freebsd-net@FreeBSD.ORG Fri May 3 14:40:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 724A2C34 for ; Fri, 3 May 2013 14:40:12 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from aussmtpmrkps320.us.dell.com (aussmtpmrkps320.us.dell.com [143.166.224.254]) by mx1.freebsd.org (Postfix) with ESMTP id 4633E14E4 for ; Fri, 3 May 2013 14:40:11 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.87,605,1363150800"; d="scan'208";a="27221275" Message-ID: <5183CC06.9020806@vangyzen.net> Date: Fri, 3 May 2013 09:39:02 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Richard Sharpe Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> In-Reply-To: Content-Type: text/plain; charset="Big5" Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 14:40:12 -0000 On 05/02/2013 19:00, Richard Sharpe wrote: > On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote: >> On 05/02/2013 08:48, Richard Sharpe wrote: >>> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote: >>>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>>> Hi folks, >>>>> >>>>> I am checking to see if there are any known bugs with respect to this >>>>> in FreeBSD 8.0. >>>>> >>>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket to >>>>> get the SMB2 requests on the wire. >>>>> >>>>> Intermittently, we see the writev return EINVAL even though the data >>>>> has gotten on the wire. This I have verified by grabbing a capture and >>>>> comparing the SMB Sequence number in the last outgoing packet on the >>>>> wire vs the in-memory contents when we get EINVAL. >>>>> >>>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>>> smaller IOVEC. >>>>> >>>>> Where should I look to check if there is some path where this might be >>>>> happening? Is this even the correct mailing list? >>>>> >>>> What does the iovec look like when you get EINVAL? Can you sanity check >>>> it? Is there anything special about it? (zero length vecs?) >>>> >>>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>>> returned. example is if your iovec is somehow huge or has many, many >>>> elements. >>> Can anyone tell me the call graph down to the TCP code? >>> >> writev kern/sys_generic.c >> kern_writev >> dofilewrite >> fo_write in sys/file.h >> soo_write in kern/sys_socket.c >> sosend in kern/uipc_socket.c >> sosend_generic >> tcp_usr_send in netinet/tcp_usrreq.c > Is there a tool that generates call graphs? I'm not aware of one that works in the kernel--other than the kernel itself, of course. With DDB compiled in, you could set a breakpoint on, say, tcp_output, and show the call stack with bt. Also, take a look at stack(9). > I have been able to demonstrate that I am getting EINVAL returned from > writev kern/sys_generic.c, kern_writev, dofilewrite and soo_write, > but when I add printfs to sosend/sosend_generic it becomes very hard > to provoke this problem. So, either relocating code or changing the timing has changed the behavior--a Heisenbug. If your code looks like this: if (error == EINVAL) printf("you are here\n"); You might add __predict_false, like this: if (__predict_false(error == EINVAL)) printf("you are here\n"); That /might/ reduce the impact on runtime behavior. > I wonder if the compiler is generating code that allows some functions > to leave variables in registers but they are not treated as volatile. From owner-freebsd-net@FreeBSD.ORG Fri May 3 17:18:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 99FAD3AA for ; Fri, 3 May 2013 17:18:51 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) by mx1.freebsd.org (Postfix) with ESMTP id 3377E12A1 for ; Fri, 3 May 2013 17:18:51 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id m6so838369wiv.15 for ; Fri, 03 May 2013 10:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=Rra2ZoFCtDCFJO83ABHxnb3Lve7Br0QfZBYgy5Im6Mc=; b=pV+wSdqn6Lq0e+liSQaLG+AKo+s9lwfmZ3tPyFntGC7S/jR/n/AOP4YEtB+z19EWLh SGWYxYWmgSFhZLgJGgqDp3Ko6mOWrNzy9ahIlrMeI+As/RGJFCNF4yJ+lsAOoUHuoO3m EkfulzjcZzuu187hUOpxowl5bFJq0IFauwOrX1kc74BNlelWu/KRPCJ3emKfJLTgrudd BMSNXa4ed66vhSHxv9L3DIotpI7ESPpJVbg8kp4KP7wsr2Drs2TitPekaQSMdEzrdt/s uUZerg9ezPx7+b0Acc7MUVMMfPs+ewUYm3XNlL7aXnkIJyjDw1fVlrliosREAE1YUUCs OHjw== MIME-Version: 1.0 X-Received: by 10.180.182.110 with SMTP id ed14mr41643366wic.6.1367601530294; Fri, 03 May 2013 10:18:50 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Fri, 3 May 2013 10:18:50 -0700 (PDT) In-Reply-To: <5183CC06.9020806@vangyzen.net> References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> <5183CC06.9020806@vangyzen.net> Date: Fri, 3 May 2013 10:18:50 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Eric van Gyzen Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 17:18:51 -0000 On Fri, May 3, 2013 at 7:39 AM, Eric van Gyzen wrote: > On 05/02/2013 19:00, Richard Sharpe wrote: >> On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrote= : >>> On 05/02/2013 08:48, Richard Sharpe wrote: >>>> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrote= : >>>>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>>>> Hi folks, >>>>>> >>>>>> I am checking to see if there are any known bugs with respect to thi= s >>>>>> in FreeBSD 8.0. >>>>>> >>>>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket t= o >>>>>> get the SMB2 requests on the wire. >>>>>> >>>>>> Intermittently, we see the writev return EINVAL even though the data >>>>>> has gotten on the wire. This I have verified by grabbing a capture a= nd >>>>>> comparing the SMB Sequence number in the last outgoing packet on the >>>>>> wire vs the in-memory contents when we get EINVAL. >>>>>> >>>>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAIN >>>>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>>>> smaller IOVEC. >>>>>> >>>>>> Where should I look to check if there is some path where this might = be >>>>>> happening? Is this even the correct mailing list? >>>>>> >>>>> What does the iovec look like when you get EINVAL? Can you sanity che= ck >>>>> it? Is there anything special about it? (zero length vecs?) >>>>> >>>>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>>>> returned. example is if your iovec is somehow huge or has many, many >>>>> elements. >>>> Can anyone tell me the call graph down to the TCP code? >>>> >>> writev kern/sys_generic.c >>> kern_writev >>> dofilewrite >>> fo_write in sys/file.h >>> soo_write in kern/sys_socket.c >>> sosend in kern/uipc_socket.c >>> sosend_generic >>> tcp_usr_send in netinet/tcp_usrreq.c >> Is there a tool that generates call graphs? > > I'm not aware of one that works in the kernel--other than the kernel > itself, of course. With DDB compiled in, you could set a breakpoint on, > say, tcp_output, and show the call stack with bt. > > Also, take a look at stack(9). > >> I have been able to demonstrate that I am getting EINVAL returned from >> writev kern/sys_generic.c, kern_writev, dofilewrite and soo_write, >> but when I add printfs to sosend/sosend_generic it becomes very hard >> to provoke this problem. > > So, either relocating code or changing the timing has changed the > behavior--a Heisenbug. > > If your code looks like this: > > if (error =3D=3D EINVAL) > printf("you are here\n"); > > You might add __predict_false, like this: > > if (__predict_false(error =3D=3D EINVAL)) > printf("you are here\n"); > > That /might/ reduce the impact on runtime behavior. Thanks for that. The problem does not appear to be in the TCP or IP layers. Rather, it appears to be in the ixgbe driver. The problem takes a little more effort to provoke, but simple printfs are doing the job so far. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Fri May 3 23:41:47 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 76AA0658 for ; Fri, 3 May 2013 23:41:47 +0000 (UTC) (envelope-from feenberg@nber.org) Received: from mail2.nber.org (mail2.nber.org [66.251.72.79]) by mx1.freebsd.org (Postfix) with ESMTP id 3BD871BB6 for ; Fri, 3 May 2013 23:41:46 +0000 (UTC) Received: from nber6 (nber6.nber.org [66.251.72.76]) by mail2.nber.org (8.14.4/8.14.4) with ESMTP id r43Nfiri073210; Fri, 3 May 2013 19:41:44 -0400 (EDT) (envelope-from feenberg@nber.org) Date: Fri, 3 May 2013 19:27:23 -0400 (EDT) From: Daniel Feenberg X-X-Sender: feenberg@nber6 To: freebsd-net@freebsd.org Subject: Restarting exports disturbs clients Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Anti-Virus: Kaspersky Anti-Virus for Linux Mail Server 5.6.39/RELEASE, bases: 20130503 #9879565, check: 20130503 clean X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 23:41:47 -0000 When we change the exportfs file on our FreeBSD 9.1 fileserver and signal mountd to reread the file: kill -HUP `cat /var/run/mountd.pid` it kills the jobs on clients that have files open on the fileserver. They terminate with an I/O error. The same thing happens if NFS is restarted. This is pretty inconvenient for users (and us). Is there a way around this? We have noticed that a Linux fileserver can restart nfs without distrubing clients (other than a short pause). The Linux restart doesn't restart the locking mechanism - is that the difference? We could do without locks, even without NFSv4, for that matter, if it would let us change exports without disturbing users. Perhaps there there is an NFS shutdown procedure that we should be using? Daniel Feenberg NBER From owner-freebsd-net@FreeBSD.ORG Sat May 4 02:01:20 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6CB376E2 for ; Sat, 4 May 2013 02:01:20 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 04D3A1F04 for ; Sat, 4 May 2013 02:01:19 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id c11so2066170wgh.34 for ; Fri, 03 May 2013 19:01:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=qewzZ7VUpLRErM+qVz4bFgS9MaYGYBv2Y6ZFuXtkSf0=; b=CVzEVL7PbiippmQLNhvbGX1jZTaQTFcdkGXcRbqkVnjjwiB64bgxOnArhA7HKoMkUp Tq6+C+JX5KQ4upcgFnnx+l8f1qia5E8hLsqu7PM7zPeMNacTa3UaR+M15q75pg//pBOT pboczX0qtpMZq+iY9DBaeccKpSyPq9ARFgQ+pNge678cflTj9SAz2qGhMRf1i3KOLM0q oK/C39rTG74AieyWOlPaAdNmw3LdZUqZ3bvk62qW3WHpKvfeG4g2kUEr+92ZQIJAL+Oh xVZMsfu1AukRQFozUdArY9EGlgEVb5xy4qLBQxYf2UZHTK20KSgJKVugM48iTl/oKp5P l7pQ== MIME-Version: 1.0 X-Received: by 10.180.90.203 with SMTP id by11mr524821wib.10.1367632878993; Fri, 03 May 2013 19:01:18 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Fri, 3 May 2013 19:01:18 -0700 (PDT) In-Reply-To: References: <5181ECDF.1040905@mu.org> <51827DAA.2020009@vangyzen.net> <5183CC06.9020806@vangyzen.net> Date: Fri, 3 May 2013 19:01:18 -0700 Message-ID: Subject: Re: Seeing EINVAL from writev on 8.0 to a non-blocking socket even though the data seems to hit the wire From: Richard Sharpe To: Eric van Gyzen Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Alfred Perlstein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 02:01:20 -0000 On Fri, May 3, 2013 at 10:18 AM, Richard Sharpe wrote: > On Fri, May 3, 2013 at 7:39 AM, Eric van Gyzen wrote: >> On 05/02/2013 19:00, Richard Sharpe wrote: >>> On Thu, May 2, 2013 at 7:52 AM, Eric van Gyzen wrot= e: >>>> On 05/02/2013 08:48, Richard Sharpe wrote: >>>>> On Wed, May 1, 2013 at 9:34 PM, Alfred Perlstein wrot= e: >>>>>> On 5/1/13 8:03 PM, Richard Sharpe wrote: >>>>>>> Hi folks, >>>>>>> >>>>>>> I am checking to see if there are any known bugs with respect to th= is >>>>>>> in FreeBSD 8.0. >>>>>>> >>>>>>> Situation is that Samba 3.6.6 uses writev to a non-blocking socket = to >>>>>>> get the SMB2 requests on the wire. >>>>>>> >>>>>>> Intermittently, we see the writev return EINVAL even though the dat= a >>>>>>> has gotten on the wire. This I have verified by grabbing a capture = and >>>>>>> comparing the SMB Sequence number in the last outgoing packet on th= e >>>>>>> wire vs the in-memory contents when we get EINVAL. >>>>>>> >>>>>>> Sometimes it occurs on a four-element IOVEC, sometimes we get EAGAI= N >>>>>>> on the four-element IOVEC and then we get EINVAL when retrying on a >>>>>>> smaller IOVEC. >>>>>>> >>>>>>> Where should I look to check if there is some path where this might= be >>>>>>> happening? Is this even the correct mailing list? >>>>>>> >>>>>> What does the iovec look like when you get EINVAL? Can you sanity ch= eck >>>>>> it? Is there anything special about it? (zero length vecs?) >>>>>> >>>>>> I think there are a few "maxvals" that if overrun cause EINVAL to be >>>>>> returned. example is if your iovec is somehow huge or has many, many >>>>>> elements. >>>>> Can anyone tell me the call graph down to the TCP code? >>>>> >>>> writev kern/sys_generic.c >>>> kern_writev >>>> dofilewrite >>>> fo_write in sys/file.h >>>> soo_write in kern/sys_socket.c >>>> sosend in kern/uipc_socket.c >>>> sosend_generic >>>> tcp_usr_send in netinet/tcp_usrreq.c >>> Is there a tool that generates call graphs? >> >> I'm not aware of one that works in the kernel--other than the kernel >> itself, of course. With DDB compiled in, you could set a breakpoint on, >> say, tcp_output, and show the call stack with bt. >> >> Also, take a look at stack(9). >> >>> I have been able to demonstrate that I am getting EINVAL returned from >>> writev kern/sys_generic.c, kern_writev, dofilewrite and soo_write, >>> but when I add printfs to sosend/sosend_generic it becomes very hard >>> to provoke this problem. >> >> So, either relocating code or changing the timing has changed the >> behavior--a Heisenbug. >> >> If your code looks like this: >> >> if (error =3D=3D EINVAL) >> printf("you are here\n"); >> >> You might add __predict_false, like this: >> >> if (__predict_false(error =3D=3D EINVAL)) >> printf("you are here\n"); >> >> That /might/ reduce the impact on runtime behavior. > > Thanks for that. The problem does not appear to be in the TCP or IP > layers. Rather, it appears to be in the ixgbe driver. > > The problem takes a little more effort to provoke, but simple printfs > are doing the job so far. The version of the ixgbe driver we are using seems to set the max size of a dma element to 65535 (IXGBE_TSO_SIZE) and, even though large numbers of iovecs are sent where the last element is 65536 bytes in size, sometimes this causes EINVAL to be returned ... --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Sat May 4 11:25:33 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B9002342 for ; Sat, 4 May 2013 11:25:33 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 310F91185 for ; Sat, 4 May 2013 11:25:32 +0000 (UTC) Received: (qmail 22357 invoked from network); 4 May 2013 10:41:48 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 4 May 2013 10:41:48 -0000 Message-ID: <5184D724.2070703@freebsd.org> Date: Sat, 04 May 2013 11:38:44 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Aris Angelo Subject: Re: Calculation of inflight data References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 11:25:33 -0000 On 03.05.2013 09:28, Aris Angelo wrote: > Hi, > > I am trying to implement an extension to the FreeBSD TCP stack. In order to > do that, I have a question regarding the calculation of the "pipe" > variable, the amount of data that the sender calculates as being inflight. > I am puzzled for the case when no SACK is negotiated and used. > > My idea would be that in this case the following is correct (during a > partial ack): > > pipe = tp->snd_max - th->th_ack; Actually pipe would decrease by one MSS for every duplicate ACK received, though we can't be sure that it really represents a full MSS due to smaller segments being possible with TCP_NODELAY. > But when looking at the tcp_output code, I can see that the off variable, > which is used as pipe to determine later how much data to send ( len = > snd_cwnd- off); ), is calculated as: > > off = tp->snd_nxt - tp->snd_una; off specifies the offset to send from in the send socket buffer and is not equal to pipe. > Obviously snd_una is used since there is no info on tcp_output for the ack > header, but I think using more up to date information is better (although > less data would be injected to the network). > > But why is snd_nxt instead of snd_max used? In case of a partial ack, > snd_nxt is readjusted for retransmit, that means, that it's closer to > snd_una. What is your opinion, how should this variable be calculated? Unfortunately the pipe value isn't really calculated at the moment. -- Andre From owner-freebsd-net@FreeBSD.ORG Sat May 4 11:31:26 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3AC5D861 for ; Sat, 4 May 2013 11:31:26 +0000 (UTC) (envelope-from jhellenthal@dataix.net) Received: from mail-gg0-x232.google.com (mail-gg0-x232.google.com [IPv6:2607:f8b0:4002:c02::232]) by mx1.freebsd.org (Postfix) with ESMTP id EDB051291 for ; Sat, 4 May 2013 11:31:25 +0000 (UTC) Received: by mail-gg0-f178.google.com with SMTP id l2so378128ggn.37 for ; Sat, 04 May 2013 04:31:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=x-received:subject:from:content-type:x-mailer:message-id:date:to :content-transfer-encoding:mime-version; bh=rX/kLOBTj0VORsryRiH88nN3Us6TjpSFx7THPBpFGGU=; b=PGYcCt3j/JJaDgTN/AqJ7zsC5wQERH9qNBHdsCvnjRUJcR2Eh/i9UxgH8kkYOkTpo/ ew7kr4UIrenTSR+D3xcq45hEFVJ9U45LdL4xPHT44Uqr+6IVHQhEkZ1s++VJ/X/BTbge bYRlEBrrrbhjSSpOGUe+AYeyxDmcvo5NNvLM8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:from:content-type:x-mailer:message-id:date:to :content-transfer-encoding:mime-version:x-gm-message-state; bh=rX/kLOBTj0VORsryRiH88nN3Us6TjpSFx7THPBpFGGU=; b=SVdZ4BSyZ8cBmehsZlzZVzmTKfkDc2/SQAQmeTagm85HwzCzVQG1y9/or2iUqPRWeZ mdJ1xPTq+r0ZljwixTNFxdEsQ7hrBzgr/qERNBEOWVkMCTTQWgU+U30hFYYqeOPqfhj9 IK4gylEnaG6kqzSXQWXWDGOQUHLCS4fMjeYhE/UdYFNE1XomLxt351PQWUR6X50gIFLg wfiJ1C80t7HKYP1x9eNSr5lrkxYnUX8Yy+BPUnZiH5P4F2gaF9kKK9BlyB3z2fHK/Cyv 4AFKr/4XDBtDjtfX2EqNv/8JHMZ3qzyOpnJm5BXbiIRB+C/LIejPRJcTqil7PsWFxpUr FaEQ== X-Received: by 10.236.75.163 with SMTP id z23mr12080569yhd.163.1367652975323; Sat, 04 May 2013 00:36:15 -0700 (PDT) Received: from [192.168.30.77] (24-236-152-143.dhcp.aldl.mi.charter.com. [24.236.152.143]) by mx.google.com with ESMTPSA id j27sm27610233yhf.18.2013.05.04.00.36.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 May 2013 00:36:14 -0700 (PDT) Subject: IGMP with no matching rules From: Jason Hellenthal X-Mailer: iPhone Mail (10B329) Message-Id: <86C973B6-D12D-41AA-A1F9-D93E1C60856F@DataIX.net> Date: Sat, 4 May 2013 03:36:07 -0400 To: "freebsd-pf@FreeBSD.org" Mime-Version: 1.0 (1.0) X-Gm-Message-State: ALoCoQlmvCSOPpdIFaFujCNcjqyIsz7rteTdL0XvWOgSs/hN1pZbb9NT6XTesezbtKZwszK7nBFd X-Mailman-Approved-At: Sat, 04 May 2013 11:36:44 +0000 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 11:31:26 -0000 Hey Everyone, Has anyone seen IGMP traffic hit there pflog interface even if there are no r= ules matching that tell it to log ? Anyone that has a pointer to eliminate the logging of the IGMP traffic would= be extremely helpful. This has been fairly frustrating up to this point try= ing to either create a rule to catch it that does not specify logging or eli= minate rules that shouldn't be matching but do. Interfaces involved... if_lagg if_bridge if_dc if_ath pflog Forwarding enabled No skipped interfaces in pf FreeBSD STABLE 8.3 as of yesterday. Please keep me CC'd Thanks & Top posting is eminent... --=20 Jason Hellenthal JJH48-ARIN -(2^(N-1)) From owner-freebsd-net@FreeBSD.ORG Sat May 4 11:58:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8AC43888 for ; Sat, 4 May 2013 11:58:19 +0000 (UTC) (envelope-from prvs=1836316fb7=agave.spring@cadamericas.com) Received: from cadamericas.com (mail02.amotive.com [173.164.153.20]) by mx1.freebsd.org (Postfix) with ESMTP id 65AFD161F for ; Sat, 4 May 2013 11:58:16 +0000 (UTC) Received: from agave.cadamericas.com ([64.183.139.162]) by amotive.com (mail02.amotive.com) (MDaemon PRO v13.0.2) with ESMTP id md50001858327.msg; Sat, 04 May 2013 02:34:48 -0700 X-Spam-Processed: mail02.amotive.com, Sat, 04 May 2013 02:34:48 -0700 (not processed: message from trusted or authenticated source) X-MDRemoteIP: 64.183.139.162 X-Return-Path: prvs=1836316fb7=agave.spring@cadamericas.com X-Envelope-From: agave.spring@cadamericas.com X-MDaemon-Deliver-To: freebsd-net@freebsd.org Date: Sat, 4 May 2013 02:34:03 -0700 To: freebsd-net From: CAD Americas Subject: CAD Americas Early Bird Registration Ends May 7 Message-ID: <584ff324a74bf5b1c9e23c45ee9845bc@agave.cadamericas.com> X-Priority: 3 X-Mailer: PHPMailer 5.2.1 (http://code.google.com/a/apache-extras.org/p/phpmailer/) X-CampTrackID: 8d8f5dcd-cb42-1fde-147d-5184d6eb5d81 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: CAD Americas List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 11:58:19 -0000 TIME IS RUNNING OUT! Register for CAD Americas Training Days by MAY 7 and S= AVE!=0ACAD AMERICAS TRAINING DAYS ARRIVE IN YOUR AREA SOON Join us for this= one-day training event in your area. Whether your focus is Mechanical Desi= gn, Construction, BIM, Electrical Design or Plant Design, there will be ses= sions that will improve your productivity immediately!=0AJune 4June 6June 7= June 12June 13June 26 June 27=0A Cleveland Cincinnati Detroit Atlanta D= allas San Jose San_Bernardino =0ATAKE HOME NEW TOOLS AND TECHNIQUES THAT = WILL IMPROVE YOUR PERFORMANCE IMMEDIATELY=0A=0A=0A=0A=0ALynn AllenTechnical= Evangelist More =0ARobert GreenCAD Mgmt Expert More =0ASteve SchainAutoCAD= Expert More =0ATod StephensRevit Expert More =0AClick here to see current = session descriptions.Please note that sessions will vary by location =0ALea= rn from well-known industry instructors who will share best practices and t= rends, product tips and tricks, new features =E2=80=A6 and more.=0AImprove = your productivity with new techniques that you can put to work right away.= =0AMeet your peers and exchange ideas on how to best use the CAD tools you = have to meet the demands of your job.=0ATake a closer look at services and = technologies offered by resellers in your area.=0AREGISTER BY MAY 7TH AND S= AVERegister for=C2=A0a CAD Americas Training Day by May 7th and save.=0AEar= ly Bird Rate: $150 (Until May 7th)=0AStandard Rate: $195 (AFTER May 7th)=0A= Student/Faculty Rate: $95 (must present current student ID upon check-in at= registration)=0AREGISTER FOR CAD AMERICAS TRAINING TODAY!=0A=0A=0A=0A=0AJo= in us at=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=C2=A0INTERNATIONAL SPONSORS=0A= =0A=0A=C2=A0=C2=A0 =C2=A0=C2=A0 =0A=0AEDUCATION SPONSOR=0A=0A=0AMEDIA SPONS= ORS=0A=0A=C2=A0=0AThis email was sent to email address: freebsd-net@freebsd= .org Click here to Opt-Out=0A From owner-freebsd-net@FreeBSD.ORG Sat May 4 13:52:18 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C23A9BCF for ; Sat, 4 May 2013 13:52:18 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 603511C00 for ; Sat, 4 May 2013 13:52:18 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id z12so2362788wgg.35 for ; Sat, 04 May 2013 06:52:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=WyaEqzCWwWVuZ9vOJIaA50JPr2M0yLRorkcVKma7jD4=; b=zbi4Zsy4qBQOvvQzHRobpBy4xjSHV+LBNR8fcw1VIPd04JNSaRCCfw7XyL2G+OKt1R eTb96yKwB/W9m8czyqfnIKhuWpvefsCDjG1lHfiuSmOHlfKmBDYyE+ISMjrFKHSKZrKS O5zsHssjk4IfKIqlYN96hz3UaHpXkbO+D3c/fu7WZNkdGmP8/ezZK0LDsNnL4RMneD0K 7uqmgz8Y23oBFn5Wl8ZPSkw8wqfMGIKfdq6AhknOjA9cqXLWDFUrK4p0iZSCsHEQx0y4 VnvBuk7k6uVpUsbiPmCboHktiJNTQ/TPULQ3h4nDbqr67hI48pUGa4G2wo/qtS7gGnng 2kEQ== MIME-Version: 1.0 X-Received: by 10.180.182.110 with SMTP id ed14mr2156240wic.6.1367675537432; Sat, 04 May 2013 06:52:17 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Sat, 4 May 2013 06:52:17 -0700 (PDT) Date: Sat, 4 May 2013 06:52:17 -0700 Message-ID: Subject: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Richard Sharpe To: freebsd-net@freebsd.org Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 13:52:18 -0000 Hi folks, I understand better why I am seeing EINVAL intermittently when sending data from Samba via SMB2. The ixgbe driver, for TSO reasons, limits the amount of data that can be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger than that. The SO_SNDBUF for that socket is set to 131972. Mostly there is less than 64kiB of space available, so that is all TCP etc can put into the socket in one chain of mbufs. However, every now and then there is more than 65535 bytes available in the socket buffers, and we have an SMB packet that is larger than 65535 bytes, and we get hit. To confirm this I am going to set SO_SNDBUF back to the default of 65536 and test again. My repros are very reliable. However, I wondered if my only way around this if I want to continue to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf chains in the driver? --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Sat May 4 17:39:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D10FF885 for ; Sat, 4 May 2013 17:39:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 6D6857CC for ; Sat, 4 May 2013 17:39:19 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id e11so2412616wgh.26 for ; Sat, 04 May 2013 10:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=hA/LkLXebx9X8ljUIeQpoiWcV6Lit/5OxJgbgz2ymaU=; b=ssSaupNn26eI8FE2UV+5g3Ama0TEYOIef9NFQOGMLwa8blB1oMlOn2kLLHCLWZCwRy ds5yYPbZJFN5JWUaDdBp5bbgHiUo9FDH/eslJVbCZGh8dOzGkN+6lWMW8slgqMIiDdKA jMR4BuWzNWeQnQUC5a3KwIXQALiR6NDjErZigJOIuWoCfXfnb+6oGpJ69lYn96isADaz CnkYnerdwdv5t9gP9biSheBSgYqudg5qlaukCyRzR86awElISiUZSXszUQqx9M4bmT8I pAO9Boppc78NV7987dEtttsNh0IdYvEP1cU+sexf9eEq9hULQxzcfzE73jEi1iuNTy08 4Vaw== MIME-Version: 1.0 X-Received: by 10.180.149.200 with SMTP id uc8mr2646873wib.3.1367689158081; Sat, 04 May 2013 10:39:18 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Sat, 4 May 2013 10:39:17 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 10:39:17 -0700 X-Google-Sender-Auth: vEznjvlbMgXa2LjS5-Ki0hYdsdY Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Adrian Chadd To: Richard Sharpe Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 17:39:19 -0000 On 4 May 2013 06:52, Richard Sharpe wrote: > Hi folks, > > I understand better why I am seeing EINVAL intermittently when sending > data from Samba via SMB2. > > The ixgbe driver, for TSO reasons, limits the amount of data that can > be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger > than that. > > The SO_SNDBUF for that socket is set to 131972. Mostly there is less > than 64kiB of space available, so that is all TCP etc can put into the > socket in one chain of mbufs. However, every now and then there is > more than 65535 bytes available in the socket buffers, and we have an > SMB packet that is larger than 65535 bytes, and we get hit. > > To confirm this I am going to set SO_SNDBUF back to the default of > 65536 and test again. My repros are very reliable. > > However, I wondered if my only way around this if I want to continue > to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf > chains in the driver? Hm, is this is a problem without TSO? Is the problem that the NIC can't handle a frame that big, or a buffer that big? Ie - if you handed the hardware two descriptors of 64k each, for the same IP datagram, will it complain? Or do you need to break it up into two separate IP datagrams, facing the driver, with a maximum size of 64k each? Adrian From owner-freebsd-net@FreeBSD.ORG Sat May 4 17:39:44 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 658B5916 for ; Sat, 4 May 2013 17:39:44 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) by mx1.freebsd.org (Postfix) with ESMTP id 016147D7 for ; Sat, 4 May 2013 17:39:43 +0000 (UTC) Received: by mail-wg0-f54.google.com with SMTP id x12so2416891wgg.21 for ; Sat, 04 May 2013 10:39:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Zq3o+LHVPkw7bsteGuY32BJBzY65qUy+YV9w6/p3UHs=; b=X/z4tCcDxNmqXzv8GIrmV+h3ZWUpBIMkV621D381lTBDaMHmKvmVgQdJOwXCmPx++w PXiu9TKOcrKfklwhkzgyznzKAa/jMdByEASIz2UiWgfrjfnXOf/o8CkP62WTjOm5XjjH s/AFLkolNJ+UpWVawJVcrn7xrXY1hW7DUlWvWmTH8+Tr9mu41Tn7fuxNPolxSlkUUZhn 6sh0/WIr0qVOkoFlFjmGUTzAmKV5lPPaDZjp/Ai3OF2MOswCbeSoSl0Y/MsYQfeTsb5d /reVIPEUz6jJFR4H34oaD4Ypdk6otGshu4f78eAmQ4L2VhdvfAxghnjUIVAlXqUs+hM/ uVNQ== MIME-Version: 1.0 X-Received: by 10.180.89.140 with SMTP id bo12mr2602695wib.22.1367689183141; Sat, 04 May 2013 10:39:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Sat, 4 May 2013 10:39:43 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 10:39:43 -0700 X-Google-Sender-Auth: B6PcLIgsw-lPfNoQmC-L__Qoc1Y Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Adrian Chadd To: Richard Sharpe Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 17:39:44 -0000 .. and please file a PR. I'm sure Jack will love this kind of feedback. :) Thanks for doing this debugging! I'm glad to see others getting dirty in the network stack. Adrian On 4 May 2013 06:52, Richard Sharpe wrote: > Hi folks, > > I understand better why I am seeing EINVAL intermittently when sending > data from Samba via SMB2. > > The ixgbe driver, for TSO reasons, limits the amount of data that can > be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger > than that. > > The SO_SNDBUF for that socket is set to 131972. Mostly there is less > than 64kiB of space available, so that is all TCP etc can put into the > socket in one chain of mbufs. However, every now and then there is > more than 65535 bytes available in the socket buffers, and we have an > SMB packet that is larger than 65535 bytes, and we get hit. > > To confirm this I am going to set SO_SNDBUF back to the default of > 65536 and test again. My repros are very reliable. > > However, I wondered if my only way around this if I want to continue > to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf > chains in the driver? > > -- > Regards, > Richard Sharpe > (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat May 4 20:30:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 25EE215A; Sat, 4 May 2013 20:30:36 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 8E882E84; Sat, 4 May 2013 20:30:35 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id t9so2132498wey.10 for ; Sat, 04 May 2013 13:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=HttCSscRMIXITaNGFKnLc9cjxU5DaNwfHF337vdAvqs=; b=FrylsITfTcBgZJCnNNxCoY0nvhh2233MA6idktxAiybT8EFQXNJDD2sTAoW6Lj42Ok 4QwjpJ022qjk/rEYvdwqJ/jFaruxHK4aQU5NIyYL2HZy4UWTVY0FiqWeZ7rZtx582Nk1 c0qWJs6gPdicWcbAoNuyDqAUWPZA8w3cfwMDrWHrxgH0p4ZLiLJjif632u5dAh0VNr2s QY1O4hYTk11yFMNh11bJhpBFvCVfAbizqSQyHeqPhMThZBAvswI8nhMvNhZsL16jtolb ak1ILFBqlpUtTctNQPo4l54pqD+WRgijSCFDhhkyZvaFwkVXAY1sSBnZ1mJiL9A5g2bq vqzw== MIME-Version: 1.0 X-Received: by 10.180.90.203 with SMTP id by11mr2937227wib.10.1367699434709; Sat, 04 May 2013 13:30:34 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Sat, 4 May 2013 13:30:34 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 13:30:34 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Richard Sharpe To: Adrian Chadd Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 20:30:36 -0000 On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote: > On 4 May 2013 06:52, Richard Sharpe wrote: >> Hi folks, >> >> I understand better why I am seeing EINVAL intermittently when sending >> data from Samba via SMB2. >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >> than that. >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >> than 64kiB of space available, so that is all TCP etc can put into the >> socket in one chain of mbufs. However, every now and then there is >> more than 65535 bytes available in the socket buffers, and we have an >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> 65536 and test again. My repros are very reliable. >> >> However, I wondered if my only way around this if I want to continue >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> chains in the driver? > > Hm, is this is a problem without TSO? We are using the card without TSO, so I am thinking of changing that limit to 131072 and retesting. I am currently testing with SO_SNDBUF=3D32768 and have not hit the problem. > Is the problem that the NIC can't handle a frame that big, or a buffer th= at big? > Ie - if you handed the hardware two descriptors of 64k each, for the > same IP datagram, will it complain? I can't find any documentation, but it seems that with TSO it cannot handle a frame that big. Actually, since we are not using TSO, there really should not be a problem with larger frames. > Or do you need to break it up into two separate IP datagrams, facing > the driver, with a maximum size of 64k each? Not sure, but it looks like we need to do that. --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Sat May 4 20:41:18 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1F61E326; Sat, 4 May 2013 20:41:18 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vc0-f170.google.com (mail-vc0-f170.google.com [209.85.220.170]) by mx1.freebsd.org (Postfix) with ESMTP id C2E48EC3; Sat, 4 May 2013 20:41:17 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id gf12so2276344vcb.15 for ; Sat, 04 May 2013 13:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=4162ot7BsDr4AF7L8qoxyOlRf0uUNusCzBe9iGv8PR4=; b=vN73CQ4war7mGaZ2hQBJlVsQo04GjK4W6d9NncujWRW+ya5BYlFgmiX9PdHK6B7oEG XCPfWla831uqzbuMpoMVDFjtB0FVHtfyO24daS+Hb5EqErdXB1lY9BnMOmtlZ8oDZOcO c3Njrxr5dG9Y7Hw7DMLjghD/d3u9K+5pDxLCp1NbIOoMcHUqJgYc/vd2HvNvXqSoeHHW knTcqxaz9IQImEx1LT/vdEjeiIUO1CagR9sl+Vqw/a5tG/P42dzhUh92QKtXiMbGPjLL i34x2x3uYqkPC2JZhU+opTHtn9rXB2Lp9eLctZe6gqbXiEgU7DhVXG0s6Nzxmko+bU5G Tbgg== MIME-Version: 1.0 X-Received: by 10.59.0.226 with SMTP id bb2mr5154405ved.1.1367700070905; Sat, 04 May 2013 13:41:10 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Sat, 4 May 2013 13:41:10 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 13:41:10 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Jack Vogel To: Richard Sharpe Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 20:41:18 -0000 If you don't use TSO you will hurt your TX performance significantly from the tests that I've run. What exactly is the device you are using, I don't have the source in front of me now, but I'm almost sure that the limit is not 64K but 256K, or are you using some ancient version of the driver? Jack On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe wrote: > On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote: > > On 4 May 2013 06:52, Richard Sharpe wrote= : > >> Hi folks, > >> > >> I understand better why I am seeing EINVAL intermittently when sending > >> data from Samba via SMB2. > >> > >> The ixgbe driver, for TSO reasons, limits the amount of data that can > >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger > >> than that. > >> > >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less > >> than 64kiB of space available, so that is all TCP etc can put into the > >> socket in one chain of mbufs. However, every now and then there is > >> more than 65535 bytes available in the socket buffers, and we have an > >> SMB packet that is larger than 65535 bytes, and we get hit. > >> > >> To confirm this I am going to set SO_SNDBUF back to the default of > >> 65536 and test again. My repros are very reliable. > >> > >> However, I wondered if my only way around this if I want to continue > >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf > >> chains in the driver? > > > > Hm, is this is a problem without TSO? > > We are using the card without TSO, so I am thinking of changing that > limit to 131072 and retesting. > > I am currently testing with SO_SNDBUF=3D32768 and have not hit the proble= m. > > > Is the problem that the NIC can't handle a frame that big, or a buffer > that big? > > Ie - if you handed the hardware two descriptors of 64k each, for the > > same IP datagram, will it complain? > > I can't find any documentation, but it seems that with TSO it cannot > handle a frame that big. Actually, since we are not using TSO, there > really should not be a problem with larger frames. > > > Or do you need to break it up into two separate IP datagrams, facing > > the driver, with a maximum size of 64k each? > > Not sure, but it looks like we need to do that. > > > -- > Regards, > Richard Sharpe > (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat May 4 20:47:27 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2A7C63D7; Sat, 4 May 2013 20:47:27 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vc0-f173.google.com (mail-vc0-f173.google.com [209.85.220.173]) by mx1.freebsd.org (Postfix) with ESMTP id CCBABEE0; Sat, 4 May 2013 20:47:26 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id ht11so2291487vcb.32 for ; Sat, 04 May 2013 13:47:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=TOtLyy4iE7FjUnVSzy6OPY4Vx0uEVZe/R5FZO9iZNjQ=; b=lIsvCR1ROVcSZKOx+FMLSp+01b+2qibuqwSVDOidH+XmL9KrsCFQdpwIf0OYZyqlcY gU1kextqtkx4YlF6Kx2AY78vljfUm74gjp/w5PPgtmL6K4X/fQsuFCzaYuZiYP2VZmtj FVcc67vdhD1q5c3wkoApMQkjmW1tQnHRdI+3h6FNfAzl+IrGXMSqE3ArSsyfpmcL7j6y pVnMOTVbdDIim402jyP8FWodCIKeGyz4H+mgFtBFRG+M0/bNZNE10H55J5BY4N0cAY2f N/8lKQDt5LolyRwSXh1xgE+EGbWbIE3u8wtiZKEFtsnDmaO7mg6Z4C/FxsT3IUL/E+rJ nGyw== MIME-Version: 1.0 X-Received: by 10.52.122.18 with SMTP id lo18mr4377295vdb.32.1367700440537; Sat, 04 May 2013 13:47:20 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Sat, 4 May 2013 13:47:20 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 13:47:20 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Jack Vogel To: Richard Sharpe Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 20:47:27 -0000 Yes, I checked: #define IXGBE_TSO_SIZE 262140 So, the driver is not limiting you to 64K assuming you are using a version of recent vintage. Jack On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: > If you don't use TSO you will hurt your TX performance significantly from > the tests that I've run. What exactly is the device you are using, I don'= t > have the source in front of me now, but I'm almost sure that the limit is > not 64K but 256K, or are you using some ancient version of the driver? > > Jack > > > > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe < > realrichardsharpe@gmail.com> wrote: > >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote= : >> > On 4 May 2013 06:52, Richard Sharpe >> wrote: >> >> Hi folks, >> >> >> >> I understand better why I am seeing EINVAL intermittently when sendin= g >> >> data from Samba via SMB2. >> >> >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >> >> than that. >> >> >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >> >> than 64kiB of space available, so that is all TCP etc can put into th= e >> >> socket in one chain of mbufs. However, every now and then there is >> >> more than 65535 bytes available in the socket buffers, and we have an >> >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> >> 65536 and test again. My repros are very reliable. >> >> >> >> However, I wondered if my only way around this if I want to continue >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> >> chains in the driver? >> > >> > Hm, is this is a problem without TSO? >> >> We are using the card without TSO, so I am thinking of changing that >> limit to 131072 and retesting. >> >> I am currently testing with SO_SNDBUF=3D32768 and have not hit the probl= em. >> >> > Is the problem that the NIC can't handle a frame that big, or a buffer >> that big? >> > Ie - if you handed the hardware two descriptors of 64k each, for the >> > same IP datagram, will it complain? >> >> I can't find any documentation, but it seems that with TSO it cannot >> handle a frame that big. Actually, since we are not using TSO, there >> really should not be a problem with larger frames. >> >> > Or do you need to break it up into two separate IP datagrams, facing >> > the driver, with a maximum size of 64k each? >> >> Not sure, but it looks like we need to do that. >> >> >> -- >> Regards, >> Richard Sharpe >> (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Sat May 4 20:54:58 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 06F324F3; Sat, 4 May 2013 20:54:58 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 6C59FF08; Sat, 4 May 2013 20:54:57 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id h11so1550919wiv.8 for ; Sat, 04 May 2013 13:54:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=YQYaUt1bTxGcVvH8m5QZm7OxUi329QmQXKZUQlNxooE=; b=aEI99YJbEEhw0Cbdk/kS1uuqJz6e/wK9/q1iwK9If5tHtgv8Yb/6EilAUQDgv6Q/nj SUeRPWX/3sjiYR1/s/kdhT3igBrJBXVfP2qBO1nHEQr1/7x549T1cxXWgs4knjsMkhq2 /9+adR8+7NefuRAk1mWieBgUeFDIht7ceh9MaMhV+qbiGaALLEcK4rkShSC6BFMaf36Z vJYCJEDNNuYuZDA36+J0CGEO3bQsFUv0FbvoreAPEyY62VWQzq5lGsxsPznGncoyURPW +M7Scyy3NCcApenKOIsP/NrUzl9MGvXPcPas0i2NBGU0pWxHXzFIgX5CIVKtsX0vsW86 CB/w== MIME-Version: 1.0 X-Received: by 10.195.13.75 with SMTP id ew11mr6644949wjd.25.1367700896076; Sat, 04 May 2013 13:54:56 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Sat, 4 May 2013 13:54:56 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 13:54:56 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Richard Sharpe To: Jack Vogel Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 20:54:58 -0000 On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: > If you don't use TSO you will hurt your TX performance significantly from > the tests that I've run. What exactly is the device you are using, I don'= t > have the source in front of me now, but I'm almost sure that the limit is > not 64K but 256K, or are you using some ancient version of the driver? ix0 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x8086 subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D0 ix1 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x8086 subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D1 The version calls itself ixgbe-2.4.4 ... Hmmm, copyright is 2001-2010 ... so perhaps a bit old. > Jack > > > > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe > wrote: >> >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote= : >> > On 4 May 2013 06:52, Richard Sharpe wrot= e: >> >> Hi folks, >> >> >> >> I understand better why I am seeing EINVAL intermittently when sendin= g >> >> data from Samba via SMB2. >> >> >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >> >> than that. >> >> >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >> >> than 64kiB of space available, so that is all TCP etc can put into th= e >> >> socket in one chain of mbufs. However, every now and then there is >> >> more than 65535 bytes available in the socket buffers, and we have an >> >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> >> 65536 and test again. My repros are very reliable. >> >> >> >> However, I wondered if my only way around this if I want to continue >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> >> chains in the driver? >> > >> > Hm, is this is a problem without TSO? >> >> We are using the card without TSO, so I am thinking of changing that >> limit to 131072 and retesting. >> >> I am currently testing with SO_SNDBUF=3D32768 and have not hit the probl= em. >> >> > Is the problem that the NIC can't handle a frame that big, or a buffer >> > that big? >> > Ie - if you handed the hardware two descriptors of 64k each, for the >> > same IP datagram, will it complain? >> >> I can't find any documentation, but it seems that with TSO it cannot >> handle a frame that big. Actually, since we are not using TSO, there >> really should not be a problem with larger frames. >> >> > Or do you need to break it up into two separate IP datagrams, facing >> > the driver, with a maximum size of 64k each? >> >> Not sure, but it looks like we need to do that. >> >> >> -- >> Regards, >> Richard Sharpe >> (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D= =9C=E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > --=20 Regards, Richard Sharpe (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D=9C= =E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) From owner-freebsd-net@FreeBSD.ORG Sat May 4 21:00:46 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B6F2877A for ; Sat, 4 May 2013 21:00:46 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 39AACF3B for ; Sat, 4 May 2013 21:00:45 +0000 (UTC) Received: (qmail 26492 invoked from network); 4 May 2013 22:03:31 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 4 May 2013 22:03:31 -0000 Message-ID: <518576EF.90706@freebsd.org> Date: Sat, 04 May 2013 23:00:31 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jack Vogel Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net , Adrian Chadd , Richard Sharpe X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:00:46 -0000 On 04.05.2013 22:47, Jack Vogel wrote: > Yes, I checked: #define IXGBE_TSO_SIZE 262140 > > So, the driver is not limiting you to 64K assuming you are using a > version of recent vintage. The stack won't generate TCP and IP packets larger than 64K. However the ethernet header gets prepended to it and bumps the overall packet size as seen by the driver to 64K + sizeof(ether_hdr) and possibly additional VLAN headers. If the bus_dma_tag size is set to 64K then bus_dma mapping will fail for such maximum size packets. -- Andre > Jack > > > > On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: > >> If you don't use TSO you will hurt your TX performance significantly from >> the tests that I've run. What exactly is the device you are using, I don't >> have the source in front of me now, but I'm almost sure that the limit is >> not 64K but 256K, or are you using some ancient version of the driver? >> >> Jack >> >> >> >> On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe < >> realrichardsharpe@gmail.com> wrote: >> >>> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd wrote: >>>> On 4 May 2013 06:52, Richard Sharpe >>> wrote: >>>>> Hi folks, >>>>> >>>>> I understand better why I am seeing EINVAL intermittently when sending >>>>> data from Samba via SMB2. >>>>> >>>>> The ixgbe driver, for TSO reasons, limits the amount of data that can >>>>> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >>>>> than that. >>>>> >>>>> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >>>>> than 64kiB of space available, so that is all TCP etc can put into the >>>>> socket in one chain of mbufs. However, every now and then there is >>>>> more than 65535 bytes available in the socket buffers, and we have an >>>>> SMB packet that is larger than 65535 bytes, and we get hit. >>>>> >>>>> To confirm this I am going to set SO_SNDBUF back to the default of >>>>> 65536 and test again. My repros are very reliable. >>>>> >>>>> However, I wondered if my only way around this if I want to continue >>>>> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >>>>> chains in the driver? >>>> >>>> Hm, is this is a problem without TSO? >>> >>> We are using the card without TSO, so I am thinking of changing that >>> limit to 131072 and retesting. >>> >>> I am currently testing with SO_SNDBUF=32768 and have not hit the problem. >>> >>>> Is the problem that the NIC can't handle a frame that big, or a buffer >>> that big? >>>> Ie - if you handed the hardware two descriptors of 64k each, for the >>>> same IP datagram, will it complain? >>> >>> I can't find any documentation, but it seems that with TSO it cannot >>> handle a frame that big. Actually, since we are not using TSO, there >>> really should not be a problem with larger frames. >>> >>>> Or do you need to break it up into two separate IP datagrams, facing >>>> the driver, with a maximum size of 64k each? >>> >>> Not sure, but it looks like we need to do that. >>> >>> >>> -- >>> Regards, >>> Richard Sharpe >>> (何以解憂？唯有杜康。--曹操) >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat May 4 21:12:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9EC9DB12; Sat, 4 May 2013 21:12:56 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by mx1.freebsd.org (Postfix) with ESMTP id 39232F81; Sat, 4 May 2013 21:12:55 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id gd11so2302142vcb.28 for ; Sat, 04 May 2013 14:12:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=ztkBxKtzdogpxb9VKwuQHREujAbStymC6HDtGM4lS4A=; b=cy3FgHIEQR5qqSScqzhOzzM17apFLX60ryqKmAvf36ZWSdtx9tAA6WP4549vid3K3n RK2/fpZLYrp5Mpt8pcr5EsRJKhbQ64qU5SXTKruZNqafhwusb15m3zCTbVFjpu5hp+E5 qir1J4HyNZB4WH7APfJgg+gEnzGdF4u+IC1OfkLxDkQRWw4xb+2MqdB19bvRcbH+6TvQ 7txEjk1aXOJi1r7q9Bdbw40dmnudwu+wkB+TOKJq3iKgsnQ9+6sm1on8aa0bHrb3bayg aHUW1FHJ8bYc/tXud+fV0kmugqKmkg2av1Qj1FufuPRe50PBrri/ANN3JLLSv50a2gEU B6jw== MIME-Version: 1.0 X-Received: by 10.220.137.145 with SMTP id w17mr5165082vct.73.1367701975262; Sat, 04 May 2013 14:12:55 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Sat, 4 May 2013 14:12:55 -0700 (PDT) In-Reply-To: <518576EF.90706@freebsd.org> References: <518576EF.90706@freebsd.org> Date: Sat, 4 May 2013 14:12:55 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Jack Vogel To: Andre Oppermann Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , Adrian Chadd , Richard Sharpe X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:12:56 -0000 Hmmm, so its the stack, can that be easily increased Andre? Regards, Jack On Sat, May 4, 2013 at 2:00 PM, Andre Oppermann wrote: > On 04.05.2013 22:47, Jack Vogel wrote: > >> Yes, I checked: #define IXGBE_TSO_SIZE 262140 >> >> So, the driver is not limiting you to 64K assuming you are using a >> version of recent vintage. >> > > The stack won't generate TCP and IP packets larger than 64K. However > the ethernet header gets prepended to it and bumps the overall packet > size as seen by the driver to 64K + sizeof(ether_hdr) and possibly > additional VLAN headers. If the bus_dma_tag size is set to 64K then > bus_dma mapping will fail for such maximum size packets. > > -- > Andre > > > Jack >> >> >> >> On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: >> >> If you don't use TSO you will hurt your TX performance significantly fr= om >>> the tests that I've run. What exactly is the device you are using, I >>> don't >>> have the source in front of me now, but I'm almost sure that the limit = is >>> not 64K but 256K, or are you using some ancient version of the driver? >>> >>> Jack >>> >>> >>> >>> On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe < >>> realrichardsharpe@gmail.com> wrote: >>> >>> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd >>>> wrote: >>>> >>>>> On 4 May 2013 06:52, Richard Sharpe >>>>> >>>> wrote: >>>> >>>>> Hi folks, >>>>>> >>>>>> I understand better why I am seeing EINVAL intermittently when sendi= ng >>>>>> data from Samba via SMB2. >>>>>> >>>>>> The ixgbe driver, for TSO reasons, limits the amount of data that ca= n >>>>>> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >>>>>> than that. >>>>>> >>>>>> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >>>>>> than 64kiB of space available, so that is all TCP etc can put into t= he >>>>>> socket in one chain of mbufs. However, every now and then there is >>>>>> more than 65535 bytes available in the socket buffers, and we have a= n >>>>>> SMB packet that is larger than 65535 bytes, and we get hit. >>>>>> >>>>>> To confirm this I am going to set SO_SNDBUF back to the default of >>>>>> 65536 and test again. My repros are very reliable. >>>>>> >>>>>> However, I wondered if my only way around this if I want to continue >>>>>> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >>>>>> chains in the driver? >>>>>> >>>>> >>>>> Hm, is this is a problem without TSO? >>>>> >>>> >>>> We are using the card without TSO, so I am thinking of changing that >>>> limit to 131072 and retesting. >>>> >>>> I am currently testing with SO_SNDBUF=3D32768 and have not hit the >>>> problem. >>>> >>>> Is the problem that the NIC can't handle a frame that big, or a buffe= r >>>>> >>>> that big? >>>> >>>>> Ie - if you handed the hardware two descriptors of 64k each, for the >>>>> same IP datagram, will it complain? >>>>> >>>> >>>> I can't find any documentation, but it seems that with TSO it cannot >>>> handle a frame that big. Actually, since we are not using TSO, there >>>> really should not be a problem with larger frames. >>>> >>>> Or do you need to break it up into two separate IP datagrams, facing >>>>> the driver, with a maximum size of 64k each? >>>>> >>>> >>>> Not sure, but it looks like we need to do that. >>>> >>>> >>>> -- >>>> Regards, >>>> Richard Sharpe >>>> (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) >>>> ______________________________**_________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.or= g >>>> " >>>> >>> >>> >>> >>> ______________________________**_________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/**mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org<= freebsd-net-unsubscribe@freebsd.org> >> " >> >> > From owner-freebsd-net@FreeBSD.ORG Sat May 4 21:18:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7FD57CB5; Sat, 4 May 2013 21:18:21 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx1.freebsd.org (Postfix) with ESMTP id 2EF8FFA7; Sat, 4 May 2013 21:18:21 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id kw10so2265360vcb.37 for ; Sat, 04 May 2013 14:18:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=NjiBIsQ9rl6aLUu+yDp/VXeuMPye0sJDdJzKhu+XKOI=; b=cNKOqph3/isd9AAgVv/gAyRwsKyb8nIeUE4Dkl0VG2/hQWoLGTDvZwSP6Q0ldN4S4N SYAz/eUI5v7/Pssta4cbcnOgLAcG7UvWUgkfuLooB2yH98gq/SlsZLCxawkHNOvd7wco zkr2pS3lOLbuK3ap9s2QI9kWTSqPqD1BPtGH/L1Gww6SaKQUEXtqj5KG/gR8f8mLMASv w9KjMCImB3tVpCbJvVbmNtcdIXX8r/bOGIeIoHIYBEujzjUelTR+VgnH4soq9BOdOlGU ghG5sLV44hvEdDXk4qkxFvSCx5JgPAi+DhQgl3rD8N8Fjhl3Q3bWSGHv83MR1o+mIFhV csrg== MIME-Version: 1.0 X-Received: by 10.58.230.130 with SMTP id sy2mr5155893vec.22.1367702300388; Sat, 04 May 2013 14:18:20 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Sat, 4 May 2013 14:18:20 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 14:18:20 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Jack Vogel To: Richard Sharpe Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:18:21 -0000 Ahh, Twinville, new hardware :) The version at the tip is 2.5.8 and I am working on version 2.5.12 internally that I hope to commit next week... so your version is "a bit old" :) I would do some testing on newer code. Jack On Sat, May 4, 2013 at 1:54 PM, Richard Sharpe wrote: > On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: > > If you don't use TSO you will hurt your TX performance significantly fr= om > > the tests that I've run. What exactly is the device you are using, I > don't > > have the source in front of me now, but I'm almost sure that the limit = is > > not 64K but 256K, or are you using some ancient version of the driver? > > ix0 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x808= 6 > subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D0 > ix1 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x808= 6 > subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D1 > > The version calls itself ixgbe-2.4.4 ... > > Hmmm, copyright is 2001-2010 ... so perhaps a bit old. > > > Jack > > > > > > > > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe < > realrichardsharpe@gmail.com> > > wrote: > >> > >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd > wrote: > >> > On 4 May 2013 06:52, Richard Sharpe > wrote: > >> >> Hi folks, > >> >> > >> >> I understand better why I am seeing EINVAL intermittently when > sending > >> >> data from Samba via SMB2. > >> >> > >> >> The ixgbe driver, for TSO reasons, limits the amount of data that c= an > >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain large= r > >> >> than that. > >> >> > >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is les= s > >> >> than 64kiB of space available, so that is all TCP etc can put into > the > >> >> socket in one chain of mbufs. However, every now and then there is > >> >> more than 65535 bytes available in the socket buffers, and we have = an > >> >> SMB packet that is larger than 65535 bytes, and we get hit. > >> >> > >> >> To confirm this I am going to set SO_SNDBUF back to the default of > >> >> 65536 and test again. My repros are very reliable. > >> >> > >> >> However, I wondered if my only way around this if I want to continu= e > >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf > >> >> chains in the driver? > >> > > >> > Hm, is this is a problem without TSO? > >> > >> We are using the card without TSO, so I am thinking of changing that > >> limit to 131072 and retesting. > >> > >> I am currently testing with SO_SNDBUF=3D32768 and have not hit the > problem. > >> > >> > Is the problem that the NIC can't handle a frame that big, or a buff= er > >> > that big? > >> > Ie - if you handed the hardware two descriptors of 64k each, for the > >> > same IP datagram, will it complain? > >> > >> I can't find any documentation, but it seems that with TSO it cannot > >> handle a frame that big. Actually, since we are not using TSO, there > >> really should not be a problem with larger frames. > >> > >> > Or do you need to break it up into two separate IP datagrams, facing > >> > the driver, with a maximum size of 64k each? > >> > >> Not sure, but it looks like we need to do that. > >> > >> > >> -- > >> Regards, > >> Richard Sharpe > >> (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D= =9C=E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > > > > -- > Regards, > Richard Sharpe > (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D=9C= =E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) > From owner-freebsd-net@FreeBSD.ORG Sat May 4 21:19:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 334C2D50; Sat, 4 May 2013 21:19:53 +0000 (UTC) (envelope-from realrichardsharpe@gmail.com) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) by mx1.freebsd.org (Postfix) with ESMTP id 9890BFB7; Sat, 4 May 2013 21:19:52 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id m6so1559503wiv.9 for ; Sat, 04 May 2013 14:19:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=b2raqqmmhLHG13tyzwpDFJwq0wDi9liKipIBJRUTCVo=; b=D1lzqK1r67ugKCc6UW+l6N3UiIWmGlELjW7J5DgItr3jskY0fFTL0mOr3K249S1pep l1aVTfdjEVa9U1GYVKBdc2UcQFy0XYzRtVem9m4ZBejjDs7XEbnZjCKUcx0phtM4dCNk OCXnwB9nsE7FfyN85fvrTKWY6IUu42EU3Sz1ESRN1YzD2kMThhC8vdiz3Xlz/a/iheE3 6VoeCSg7FI110rPzUzEVYFcha0VYe8w5j3TGtVCijLm/LloBN9kOxNkG7LtVV6CfTk64 x8SLepYRJAQou7EjZf/o0KN3oiMJc7FrKdSWgQIbBoZyOtnvMdZU4Rnhap/v/iEsI1PC pZ3w== MIME-Version: 1.0 X-Received: by 10.180.90.203 with SMTP id by11mr3015897wib.10.1367702391647; Sat, 04 May 2013 14:19:51 -0700 (PDT) Received: by 10.194.179.194 with HTTP; Sat, 4 May 2013 14:19:51 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 May 2013 14:19:51 -0700 Message-ID: Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? From: Richard Sharpe To: Jack Vogel Content-Type: text/plain; charset=Big5 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:19:53 -0000 On Sat, May 4, 2013 at 2:18 PM, Jack Vogel wrote: > Ahh, Twinville, new hardware :) The version at the tip is 2.5.8 and I am > working on version 2.5.12 internally that I hope to commit next week... > so your version is "a bit old" :) I would do some testing on newer code. I would love to. Where is the repo. > Jack > > > > On Sat, May 4, 2013 at 1:54 PM, Richard Sharpe > wrote: >> >> On Sat, May 4, 2013 at 1:41 PM, Jack Vogel wrote: >> > If you don't use TSO you will hurt your TX performance significantly >> > from >> > the tests that I've run. What exactly is the device you are using, I >> > don't >> > have the source in front of me now, but I'm almost sure that the limit >> > is >> > not 64K but 256K, or are you using some ancient version of the driver? >> >> ix0 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x80= 86 >> subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D0 >> ix1 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x80= 86 >> subdevice=3D0x0001 class=3D0x020000 at slot=3D0 function=3D1 >> >> The version calls itself ixgbe-2.4.4 ... >> >> Hmmm, copyright is 2001-2010 ... so perhaps a bit old. >> >> > Jack >> > >> > >> > >> > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe >> > >> > wrote: >> >> >> >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd >> >> wrote: >> >> > On 4 May 2013 06:52, Richard Sharpe >> >> > wrote: >> >> >> Hi folks, >> >> >> >> >> >> I understand better why I am seeing EINVAL intermittently when >> >> >> sending >> >> >> data from Samba via SMB2. >> >> >> >> >> >> The ixgbe driver, for TSO reasons, limits the amount of data that >> >> >> can >> >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larg= er >> >> >> than that. >> >> >> >> >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is le= ss >> >> >> than 64kiB of space available, so that is all TCP etc can put into >> >> >> the >> >> >> socket in one chain of mbufs. However, every now and then there is >> >> >> more than 65535 bytes available in the socket buffers, and we have >> >> >> an >> >> >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> >> >> >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> >> >> 65536 and test again. My repros are very reliable. >> >> >> >> >> >> However, I wondered if my only way around this if I want to contin= ue >> >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> >> >> chains in the driver? >> >> > >> >> > Hm, is this is a problem without TSO? >> >> >> >> We are using the card without TSO, so I am thinking of changing that >> >> limit to 131072 and retesting. >> >> >> >> I am currently testing with SO_SNDBUF=3D32768 and have not hit the >> >> problem. >> >> >> >> > Is the problem that the NIC can't handle a frame that big, or a >> >> > buffer >> >> > that big? >> >> > Ie - if you handed the hardware two descriptors of 64k each, for th= e >> >> > same IP datagram, will it complain? >> >> >> >> I can't find any documentation, but it seems that with TSO it cannot >> >> handle a frame that big. Actually, since we are not using TSO, there >> >> really should not be a problem with larger frames. >> >> >> >> > Or do you need to break it up into two separate IP datagrams, facin= g >> >> > the driver, with a maximum size of 64k each? >> >> >> >> Not sure, but it looks like we need to do that. >> >> >> >> >> >> -- >> >> Regards, >> >> Richard Sharpe >> >> (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) >> >> _______________________________________________ >> >> freebsd-net@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >> > >> > >> >> >> >> -- >> Regards, >> Richard Sharpe >> (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) > > --=20 Regards, Richard Sharpe (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) From owner-freebsd-net@FreeBSD.ORG Sat May 4 21:30:49 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D196D79C; Sat, 4 May 2013 21:30:49 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id ABBA6D7; Sat, 4 May 2013 21:30:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r44LUnfv078567; Sat, 4 May 2013 21:30:49 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r44LUnAF078566; Sat, 4 May 2013 21:30:49 GMT (envelope-from linimon) Date: Sat, 4 May 2013 21:30:49 GMT Message-Id: <201305042130.r44LUnAF078566@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-i386@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178288: [fxp] fxp network driver broken in 7.4 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:30:49 -0000 Old Synopsis: fxp network driver broken in 7.4 New Synopsis: [fxp] fxp network driver broken in 7.4 Responsible-Changed-From-To: freebsd-i386->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sat May 4 21:30:24 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178288