Date: Fri, 22 Mar 2013 11:41:44 +0100 From: Michio Honda <micchie@sfc.wide.ad.jp> To: Walter de Donato <walter.dedonato@unina.it> Cc: freebsd-net@freebsd.org Subject: Re: Netmap ixgbe driver problem on 10Gbps X540-AT2 adapters (Linux only?) Message-ID: <D9E78ACE-2E40-4EE8-AC0F-2BCA8DA6D158@sfc.wide.ad.jp> In-Reply-To: <CAPRxaNQVkkb1nX-4ziL6BYp_pLqNAvZruYUEPQs_4=Xvinc2gg@mail.gmail.com> References: <CAPRxaNQVkkb1nX-4ziL6BYp_pLqNAvZruYUEPQs_4=Xvinc2gg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
You need to specify "-w 4" on running pkt-gen to wait for that the link = resets. Cheers, - Michio On Mar 22, 2013, at 11:16 AM, Walter de Donato wrote: > Dear Tahir, >=20 > I've already tried to disable Rx/Tx pause and autonegotiation but the > result in transmission is always the same. >=20 > I did the following: > $ sudo ifconfig eth2 up > $ sudo ethtool -A eth2 autoneg off > $ sudo ethtool -A eth2 rx off > $ sudo ethtool -A eth2 tx off >=20 > These are the resulting dmesg output lines: > [253131.445124] ixgbe 0000:0c:00.0: eth2: NIC Link is Up 10 Gbps, Flow > Control: RX/TX > [253147.935426] ixgbe 0000:0c:00.0: eth2: NIC Link is Up 10 Gbps, Flow > Control: TX > [253163.930776] ixgbe 0000:0c:00.0: eth2: NIC Link is Up 10 Gbps, Flow > Control: None >=20 > Here is what I obtain: > $ sudo ./pkt-gen -i eth2 -n 100000 -f tx > extract_ip_range [136] extract IP range from 10.0.0.1 > extract_ip_range [171] range is 10.0.0.1 0 to 10.0.0.1 0 > extract_ip_range [136] extract IP range from 10.1.0.1 > extract_ip_range [171] range is 10.1.0.1 0 to 10.1.0.1 0 > extract_mac_range [177] extract MAC range from a0:36:9f:0d:f0:98 > extract_mac_range [192] a0:36:9f:0d:f0:98 starts at a0:36:9f:d:f0:98 > extract_mac_range [177] extract MAC range from ff:ff:ff:ff:ff:ff > extract_mac_range [192] ff:ff:ff:ff:ff:ff starts at ff:ff:ff:ff:ff:ff > main [1401] map size is 334980 Kb > main [1423] mapping 334980 Kbytes > Sending on eth2: 4 queues, 1 threads and 1 cpus. > 10.0.0.1 -> 10.1.0.1 (a0:36:9f:0d:f0:98 -> ff:ff:ff:ff:ff:ff) > main [1477] Wait 2 secs for phy reset > main [1479] Ready... > sender_body [683] start > main_thread [1078] 7155 pps (7162 pkts in 1001020 usec) > main_thread [1078] 3484 pps (3488 pkts in 1001030 usec) > main_thread [1078] 3487 pps (3491 pkts in 1001024 usec) > main_thread [1078] 3466 pps (3469 pkts in 1000847 usec) > main_thread [1078] 3485 pps (3490 pkts in 1001521 usec) > main_thread [1078] 3322 pps (3328 pkts in 1001765 usec) > main_thread [1078] 3293 pps (3297 pkts in 1001322 usec) > main_thread [1078] 3492 pps (3498 pkts in 1001763 usec) > main_thread [1078] 3485 pps (3489 pkts in 1001168 usec) > main_thread [1078] 3351 pps (3356 pkts in 1001402 usec) > main_thread [1078] 3297 pps (3301 pkts in 1001314 usec) > main_thread [1078] 3933 pps (3938 pkts in 1001390 usec) > main_thread [1078] 3481 pps (3486 pkts in 1001528 usec) > main_thread [1078] 3493 pps (3497 pkts in 1001177 usec) > main_thread [1078] 3310 pps (3314 pkts in 1001292 usec) > main_thread [1078] 3485 pps (3491 pkts in 1001812 usec) > main_thread [1078] 3326 pps (3331 pkts in 1001547 usec) > main_thread [1078] 3480 pps (3485 pkts in 1001456 usec) > main_thread [1078] 3489 pps (3493 pkts in 1001209 usec) > main_thread [1078] 3377 pps (3381 pkts in 1001040 usec) > main_thread [1078] 3450 pps (3454 pkts in 1001048 usec) > main_thread [1078] 3378 pps (3382 pkts in 1001050 usec) > main_thread [1078] 3470 pps (3474 pkts in 1001052 usec) > main_thread [1078] 3323 pps (3327 pkts in 1001056 usec) > main_thread [1078] 3482 pps (3486 pkts in 1001050 usec) > main_thread [1078] 3472 pps (3476 pkts in 1001066 usec) > main_thread [1078] 3497 pps (3501 pkts in 1001044 usec) > main_thread [1078] 3115 pps (3115 pkts in 1000035 usec) > main_thread [1078] 0 pps (0 pkts in 1001057 usec) > Sent 100000 packets, 60 bytes each, in 28.42 seconds. > Speed: 3.52 Kpps Bandwidth: 1.69 Mbps (raw 2.36 Mbps) >=20 > While executing pkt-gen I see the following dmesg output lines: > [253741.892031] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253742.874686] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253743.864635] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253744.854513] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253745.844527] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253746.830323] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253747.821223] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253748.813229] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253749.804317] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253750.794286] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253751.784302] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253752.774324] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253753.764258] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253754.753735] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253755.748011] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253756.734927] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253757.724876] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253758.713267] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253759.704793] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253760.694759] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253761.684743] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253762.674699] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253763.664642] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253764.654488] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253765.644462] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253766.634425] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253767.624392] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253768.614364] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253769.611107] ixgbe 0000:0c:00.0: eth2: Reset adapter > [253769.715298] 893.960705 netmap_ring_reinit [966] called for eth2 > [253769.716646] 893.962054 netmap_ring_reinit [966] called for eth2 > [253769.725844] 893.971252 netmap_ioctl [1236] deprecated, data is = ded4df20 > [253776.141936] ixgbe 0000:0c:00.0: eth2: NIC Link is Up 10 Gbps, Flow > Control: None >=20 > It seems that the adapter is reset every second, but no packet goes on = the > wire. >=20 > I'm starting to look into the code to investigate this issue. > If you have any suggestion on where to look at it would save me a lot = of > time. >=20 > I also rechecked the rx mode. > It is working at least up to 50Kpps (as I'm generation on the other = side > with nping and the original driver). > Pkg-gen/netmap is configured to receive only the frames directed to = the > physical MAC address of the interface. > I don't know if and how a promiscuous mode can be enabled with netmap. >=20 > Thanks, > -Walter > 2013/3/22 Tahir Rauf <tahir.rauf1@gmail.com> >=20 >> Dear Walter, >>=20 >> Can you please disable Rx/Tx pause frames on your NIC. You can use >> "ethtool" to do so. Please run the example again after disabling = Rx/Tx >> frames and let me know about the results. >>=20 >> Regards >>=20 >>=20 >> On Thu, Mar 21, 2013 at 8:20 PM, Walter de Donato < >> walter.dedonato@gmail.com> wrote: >>=20 >>> Not that lucky since my first goal is to transmit packets... >>> I also noticed that the same car d is configured with 4 rings on = 32bit >>> kernels and 16 rings on 64bit kernels. >>> Do you have any idea to explain this difference? >>>=20 >>> -Walter >>>=20 >>> 2013/3/21 Tahir Rauf <tahir.rauf1@gmail.com> >>>=20 >>>> Dear Walter, >>>>=20 >>>> You are lucky to have* X540-AT2 :). *Mine is different (82598EB) = and >>>> can't receive packets properly. >>>> Yes, my tx is working and is able to send packets on line rate. >>>>=20 >>>> Regards >>>>=20 >>>>=20 >>>> On Thu, Mar 21, 2013 at 8:04 PM, Walter de Donato < >>>> walter.dedonato@gmail.com> wrote: >>>>=20 >>>>> Here is the lshw output: >>>>>=20 >>>>> *-network:0 >>>>> description: Ethernet interface >>>>> product: Ethernet Controller 10 Gigabit X540-AT2 >>>>> vendor: Intel Corporation >>>>> physical id: 0 >>>>> bus info: pci@0000:0c:00.0 >>>>> logical name: eth2 >>>>> version: 01 >>>>> serial: a0:36:9f:0d:f0:98 >>>>> capacity: 1Gbit/s >>>>> width: 64 bits >>>>> clock: 33MHz >>>>> capabilities: pm msi msix pciexpress vpd bus_master cap_list = rom >>>>> ethernet physical tp 100bt-fd 1000bt-fd autonegotiation >>>>> configuration: autonegotiation=3Don broadcast=3Dyes = driver=3Dixgbe >>>>> driverversion=3D3.6.7-k duplex=3Dfull firmware=3D0x800002ef = latency=3D0 link=3Dyes >>>>> multicast=3Dyes port=3Dtwisted pair >>>>> resources: irq:17 memory:d8400000-d85fffff >>>>> memory:d83fc000-d83fffff memory:fc500000-fc57ffff = memory:fc400000-fc4fffff >>>>> *-network:1 >>>>> description: Ethernet interface >>>>> product: Ethernet Controller 10 Gigabit X540-AT2 >>>>> vendor: Intel Corporation >>>>> physical id: 0.1 >>>>> bus info: pci@0000:0c:00.1 >>>>> logical name: eth3 >>>>> version: 01 >>>>> serial: a0:36:9f:0d:f0:9a >>>>> capacity: 1Gbit/s >>>>> width: 64 bits >>>>> clock: 33MHz >>>>> capabilities: pm msi msix pciexpress vpd bus_master cap_list = rom >>>>> ethernet physical tp 100bt-fd 1000bt-fd autonegotiation >>>>> configuration: autonegotiation=3Don broadcast=3Dyes = driver=3Dixgbe >>>>> driverversion=3D3.6.7-k duplex=3Dfull firmware=3D0x800002ef = ip=3D192.168.78.2 >>>>> latency=3D0 link=3Dyes multicast=3Dyes port=3Dtwisted pair >>>>> resources: irq:16 memory:d8000000-d81fffff >>>>> memory:d83f8000-d83fbfff memory:d8200000-d827ffff >>>>>=20 >>>>> Did you check if in tx mode your pkt-gen effectively sends the = packets >>>>> on the wire? >>>>> What rates are you observing with it? >>>>>=20 >>>>> -Walter >>>>>=20 >>>>> 2013/3/21 Tahir Rauf <tahir.rauf1@gmail.com> >>>>>=20 >>>>>> Dear walter, >>>>>>=20 >>>>>> Can you please run following command to get your NIC information. >>>>>> sudo lshw -class network >>>>>>=20 >>>>>> I have *82598EB *10GB AF dual port network connection NIC and my = rx >>>>>> is not working. >>>>>>=20 >>>>>> Regards >>>>>>=20 >>>>>>=20 >>>>>> On Thu, Mar 21, 2013 at 7:46 PM, Walter de Donato < >>>>>> walter.dedonato@gmail.com> wrote: >>>>>>=20 >>>>>>> I tried the rx command and it seems to work: >>>>>>> $ sudo ./pkt-gen -i eth3 -f rx -n 100000 -l 60 -w 1 >>>>>>> extract_ip_range [136] extract IP range from 10.0.0.1 >>>>>>> extract_ip_range [171] range is 10.0.0.1 0 to 10.0.0.1 0 >>>>>>> extract_ip_range [136] extract IP range from 10.1.0.1 >>>>>>> extract_ip_range [171] range is 10.1.0.1 0 to 10.1.0.1 0 >>>>>>> extract_mac_range [177] extract MAC range from a0:36:9f:0d:f0:9a >>>>>>> extract_mac_range [192] a0:36:9f:0d:f0:9a starts at = a0:36:9f:d:f0:9a >>>>>>> extract_mac_range [177] extract MAC range from ff:ff:ff:ff:ff:ff >>>>>>> extract_mac_range [192] ff:ff:ff:ff:ff:ff starts at = ff:ff:ff:ff:ff:ff >>>>>>> main [1401] map size is 334980 Kb >>>>>>> main [1423] mapping 334980 Kbytes >>>>>>> Receiving from eth3: 4 queues, 1 threads and 1 cpus. >>>>>>> main [1477] Wait 1 secs for phy reset >>>>>>> main [1479] Ready... >>>>>>> main_thread [1078] 0 pps (0 pkts in 1001020 usec) >>>>>>> receiver_body [833] waiting for initial packets, poll returns 0 = 0 >>>>>>> main_thread [1078] 0 pps (0 pkts in 1001059 usec) >>>>>>> receiver_body [833] waiting for initial packets, poll returns 0 = 0 >>>>>>> main_thread [1078] 0 pps (0 pkts in 1001047 usec) >>>>>>> receiver_body [833] waiting for initial packets, poll returns 0 = 0 >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001118 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001040 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001038 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001030 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001028 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001051 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001068 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001040 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001035 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001042 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001083 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001050 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001073 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001052 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001041 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001051 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001058 usec) >>>>>>> main_thread [1078] 1 pps (1 pkts in 1001020 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001021 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001022 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001072 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001020 usec) >>>>>>> main_thread [1078] 3 pps (3 pkts in 1001041 usec) >>>>>>> main_thread [1078] 3 pps (3 pkts in 1001023 usec) >>>>>>> main_thread [1078] 4 pps (4 pkts in 1001025 usec) >>>>>>> main_thread [1078] 3 pps (3 pkts in 1001032 usec) >>>>>>> main_thread [1078] 3 pps (3 pkts in 1001058 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001042 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001018 usec) >>>>>>> main_thread [1078] 2 pps (2 pkts in 1001024 usec) >>>>>>> main_thread [1078] 0 pps (0 pkts in 1001058 usec) >>>>>>> Received 58 packets, in 29.16 seconds. >>>>>>> Speed: 1.99 pps >>>>>>>=20 >>>>>>>=20 >>>>>>> The proble is that on the other side I had to use ping instead = of >>>>>>> pkt-gen. >>>>>>>=20 >>>>>>> If I try to use pkt-gen I don't receive any packet. >>>>>>> I just checked that pkt-gen is not effectively sending the = packets on >>>>>>> the wire, >>>>>>> on the other side using tcpdump I don't receive any packet. >>>>>>>=20 >>>>>>> So the problem on the tx side has not been completely solved in = my >>>>>>> case. >>>>>>> Can you do the same test please? >>>>>>>=20 >>>>>>> -Walter >>>>>>>=20 >>>>>>> 2013/3/21 Tahir Rauf <tahir.rauf1@gmail.com> >>>>>>>=20 >>>>>>>> Dear Walter, >>>>>>>>=20 >>>>>>>> What about the Rx, Can you please try and inform me. >>>>>>>> sudo ./pkt-gen -i eth2 -f rx -n 100000 -l 60 -w 1 >>>>>>>>=20 >>>>>>>> Regards >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> On Thu, Mar 21, 2013 at 7:06 PM, Walter de Donato < >>>>>>>> walter.dedonato@gmail.com> wrote: >>>>>>>>=20 >>>>>>>>> Thanks Thair, >>>>>>>>>=20 >>>>>>>>> with the fixed version the interface correctly attaches to the >>>>>>>>> netmap driver and it is able to transmit packets. >>>>>>>>>=20 >>>>>>>>> Anyway, I'm observing really poor performance: >>>>>>>>> $ sudo ./pkt-gen -i eth2 -f tx -n 100000 -l 60 -w 1 >>>>>>>>> extract_ip_range [136] extract IP range from 10.0.0.1 >>>>>>>>> extract_ip_range [171] range is 10.0.0.1 0 to 10.0.0.1 0 >>>>>>>>> extract_ip_range [136] extract IP range from 10.1.0.1 >>>>>>>>> extract_ip_range [171] range is 10.1.0.1 0 to 10.1.0.1 0 >>>>>>>>> extract_mac_range [177] extract MAC range from = a0:36:9f:0d:f0:98 >>>>>>>>> extract_mac_range [192] a0:36:9f:0d:f0:98 starts at = a0:36:9f:d:f0:98 >>>>>>>>> extract_mac_range [177] extract MAC range from = ff:ff:ff:ff:ff:ff >>>>>>>>> extract_mac_range [192] ff:ff:ff:ff:ff:ff starts at >>>>>>>>> ff:ff:ff:ff:ff:ff >>>>>>>>> main [1401] map size is 334980 Kb >>>>>>>>> main [1423] mapping 334980 Kbytes >>>>>>>>> Sending on eth2: 4 queues, 1 threads and 1 cpus. >>>>>>>>> 10.0.0.1 -> 10.1.0.1 (a0:36:9f:0d:f0:98 -> ff:ff:ff:ff:ff:ff) >>>>>>>>> main [1477] Wait 1 secs for phy reset >>>>>>>>> main [1479] Ready... >>>>>>>>> sender_body [683] start >>>>>>>>> main_thread [1078] 7101 pps (7108 pkts in 1001019 usec) >>>>>>>>> main_thread [1078] 3482 pps (3486 pkts in 1001031 usec) >>>>>>>>> main_thread [1078] 3490 pps (3494 pkts in 1001026 usec) >>>>>>>>> main_thread [1078] 3483 pps (3487 pkts in 1001026 usec) >>>>>>>>> main_thread [1078] 3340 pps (3343 pkts in 1001019 usec) >>>>>>>>> main_thread [1078] 3455 pps (3459 pkts in 1001025 usec) >>>>>>>>> main_thread [1078] 3493 pps (3497 pkts in 1001024 usec) >>>>>>>>> main_thread [1078] 3494 pps (3498 pkts in 1001022 usec) >>>>>>>>> main_thread [1078] 3353 pps (3357 pkts in 1001056 usec) >>>>>>>>> main_thread [1078] 3488 pps (3492 pkts in 1001093 usec) >>>>>>>>> main_thread [1078] 3474 pps (3478 pkts in 1001064 usec) >>>>>>>>> main_thread [1078] 3485 pps (3489 pkts in 1001102 usec) >>>>>>>>> main_thread [1078] 3479 pps (3483 pkts in 1001064 usec) >>>>>>>>> main_thread [1078] 3335 pps (3339 pkts in 1001111 usec) >>>>>>>>> main_thread [1078] 3484 pps (3488 pkts in 1001074 usec) >>>>>>>>> main_thread [1078] 3484 pps (3488 pkts in 1001030 usec) >>>>>>>>> main_thread [1078] 3470 pps (3474 pkts in 1001046 usec) >>>>>>>>> main_thread [1078] 3486 pps (3490 pkts in 1001040 usec) >>>>>>>>> main_thread [1078] 3482 pps (3486 pkts in 1001035 usec) >>>>>>>>> main_thread [1078] 3278 pps (3281 pkts in 1001045 usec) >>>>>>>>> main_thread [1078] 3476 pps (3480 pkts in 1001021 usec) >>>>>>>>> main_thread [1078] 3490 pps (3494 pkts in 1001036 usec) >>>>>>>>> main_thread [1078] 3477 pps (3481 pkts in 1001083 usec) >>>>>>>>> main_thread [1078] 3493 pps (3497 pkts in 1001048 usec) >>>>>>>>> main_thread [1078] 3353 pps (3357 pkts in 1001082 usec) >>>>>>>>> main_thread [1078] 3490 pps (3494 pkts in 1001051 usec) >>>>>>>>> main_thread [1078] 3465 pps (3469 pkts in 1001060 usec) >>>>>>>>> main_thread [1078] 3008 pps (3011 pkts in 1001030 usec) >>>>>>>>> main_thread [1078] 0 pps (0 pkts in 1001039 usec) >>>>>>>>> Sent 100000 packets, 60 bytes each, in 28.58 seconds. >>>>>>>>> Speed: 3.50 Kpps Bandwidth: 1.68 Mbps (raw 2.35 Mbps) >>>>>>>>>=20 >>>>>>>>> With the rtl8169 1G driver (which they say to have the worst >>>>>>>>> performance) >>>>>>>>> I easily obtained Speed: 359.24Kpps. Bandwidth: 172.44Mbps = with 64 >>>>>>>>> bytes packets. >>>>>>>>>=20 >>>>>>>>> Also, if I increase the packet size this is what happens: >>>>>>>>> $ sudo ./pkt-gen -i eth2 -f tx -t 1000000 -l 508 >>>>>>>>> main [1270] -t deprecated, please use -f tx -n 1000000 >>>>>>>>> extract_ip_range [136] extract IP range from 10.0.0.1 >>>>>>>>> extract_ip_range [171] range is 10.0.0.1 0 to 10.0.0.1 0 >>>>>>>>> extract_ip_range [136] extract IP range from 10.1.0.1 >>>>>>>>> extract_ip_range [171] range is 10.1.0.1 0 to 10.1.0.1 0 >>>>>>>>> extract_mac_range [177] extract MAC range from = a0:36:9f:0d:f0:98 >>>>>>>>> extract_mac_range [192] a0:36:9f:0d:f0:98 starts at = a0:36:9f:d:f0:98 >>>>>>>>> extract_mac_range [177] extract MAC range from = ff:ff:ff:ff:ff:ff >>>>>>>>> extract_mac_range [192] ff:ff:ff:ff:ff:ff starts at >>>>>>>>> ff:ff:ff:ff:ff:ff >>>>>>>>> main [1401] map size is 334980 Kb >>>>>>>>> main [1423] mapping 334980 Kbytes >>>>>>>>> Sending on eth2: 4 queues, 1 threads and 1 cpus. >>>>>>>>> 10.0.0.1 -> 10.1.0.1 (a0:36:9f:0d:f0:98 -> ff:ff:ff:ff:ff:ff) >>>>>>>>> main [1477] Wait 2 secs for phy reset >>>>>>>>> main [1479] Ready... >>>>>>>>> sender_body [683] start >>>>>>>>> main_thread [1078] 2042 pps (2044 pkts in 1001016 usec) >>>>>>>>> main_thread [1078] 0 pps (0 pkts in 1001049 usec) >>>>>>>>> sender_body [729] poll error/timeout on queue 0 >>>>>>>>> main_thread [1078] 0 pps (0 pkts in 1001035 usec) >>>>>>>>> main_thread [1096] ouch, thread 0 exited with error >>>>>>>>> Sent 2044 packets, 508 bytes each, in -1363874673.38 seconds. >>>>>>>>> Speed: -0.00 pps Bandwidth: -0.01 bps (raw -0.01 bps) >>>>>>>>>=20 >>>>>>>>> Don't you experience the same issues? >>>>>>>>>=20 >>>>>>>>> -Walter >>>>>>>>>=20 >>>>>>>>> 2013/3/21 Tahir Rauf <tahir.rauf1@gmail.com> >>>>>>>>>=20 >>>>>>>>>> Dear Walter, >>>>>>>>>>=20 >>>>>>>>>> Please try following code. >>>>>>>>>> http://info.iet.unipi.it/~luigi/doc/20130217-netmap.tgz >>>>>>>>>> It will fix your problem with Ixgbe tx function. >>>>>>>>>>=20 >>>>>>>>>> Please let me know, if you encounter any problem. >>>>>>>>>>=20 >>>>>>>>>> Regards >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> On Thu, Mar 21, 2013 at 6:28 PM, Walter de Donato < >>>>>>>>>> walter.dedonato@gmail.com> wrote: >>>>>>>>>>=20 >>>>>>>>>>> Dear Tahir, >>>>>>>>>>>=20 >>>>>>>>>>> thank you for the prompt reply. >>>>>>>>>>> I downloaded the source code from the official website: >>>>>>>>>>> http://info.iet.unipi.it/~luigi/doc/20120813-netmap.tgz >>>>>>>>>>> Can you please share with me the fix about the tx function? >>>>>>>>>>> I was just starting to look at the patched code of the = drivers. >>>>>>>>>>>=20 >>>>>>>>>>> Thanks in advance, >>>>>>>>>>> -Walter >>>>>>>>>>>=20 >>>>>>>>>>> 2013/3/21 Tahir Rauf <tahir.rauf1@gmail.com> >>>>>>>>>>>=20 >>>>>>>>>>>> Hi dedonato, >>>>>>>>>>>>=20 >>>>>>>>>>>> I just observed that you are having trouble with tx = function of >>>>>>>>>>>> pkt-gen. Whereas, we already have fixed that problem. We = are facing issues >>>>>>>>>>>> with rx only now. >>>>>>>>>>>> Please tell me that from where you get the code of netmap? >>>>>>>>>>>>=20 >>>>>>>>>>>> Regards >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> On Thu, Mar 21, 2013 at 5:49 PM, Tahir Rauf < >>>>>>>>>>>> tahir.rauf1@gmail.com> wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> Dear Walter, >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I tried to fix the problem but i don't have any success = yet. >>>>>>>>>>>>> Also if you are trying to fix this problem, please share = with >>>>>>>>>>>>> me that which path you are going to take for fixing. It = might be possible >>>>>>>>>>>>> that I already have tried that path. In that case, I can = share my results >>>>>>>>>>>>> with you, thus saving your time. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Regards >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On Thu, Mar 21, 2013 at 5:08 PM, = <walter.dedonato@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Dear Tahir and Luigi, >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I have exactly the same problem with a similar = hardware/os >>>>>>>>>>>>>> configuration reported in the following. >>>>>>>>>>>>>> Do you hav any uptade on this issue? >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> -Walter >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Netmap version: 20120813 >>>>>>>>>>>>>> OS: Ubuntu 12.04.2 LTS >>>>>>>>>>>>>> Kernel: 3.2.0-38-generic-pae #61-Ubuntu SMP Tue Feb 19 >>>>>>>>>>>>>> 12:39:51 UTC 2013 i686 i686 i386 GNU/Linux >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Network interfaces: >>>>>>>>>>>>>> eth2 >>>>>>>>>>>>>> 0c:00.0 Ethernet controller: Intel Corporation Ethernet >>>>>>>>>>>>>> Controller 10 Gigabit X540-AT2 (rev 01) >>>>>>>>>>>>>> Subsystem: Intel Corporation Ethernet 10G 2P = X540-t >>>>>>>>>>>>>> Adapter >>>>>>>>>>>>>> Flags: bus master, fast devsel, latency 0, IRQ 17 >>>>>>>>>>>>>> Memory at d8400000 (64-bit, prefetchable) = [size=3D2M] >>>>>>>>>>>>>> Memory at d83fc000 (64-bit, prefetchable) = [size=3D16K] >>>>>>>>>>>>>> Expansion ROM at fc500000 [disabled] [size=3D512K] >>>>>>>>>>>>>> Capabilities: [40] Power Management version 3 >>>>>>>>>>>>>> Capabilities: [50] MSI: Enable- Count=3D1/1 = Maskable+ >>>>>>>>>>>>>> 64bit+ >>>>>>>>>>>>>> Capabilities: [70] MSI-X: Enable+ Count=3D64 = Masked- >>>>>>>>>>>>>> Capabilities: [a0] Express Endpoint, MSI 00 >>>>>>>>>>>>>> Capabilities: [e0] Vital Product Data >>>>>>>>>>>>>> Capabilities: [100] Advanced Error Reporting >>>>>>>>>>>>>> Capabilities: [140] Device Serial Number >>>>>>>>>>>>>> a0-36-9f-ff-ff-0d-f0-98 >>>>>>>>>>>>>> Capabilities: [150] Alternative Routing-ID >>>>>>>>>>>>>> Interpretation (ARI) >>>>>>>>>>>>>> Capabilities: [160] Single Root I/O Virtualization >>>>>>>>>>>>>> (SR-IOV) >>>>>>>>>>>>>> Capabilities: [1d0] Access Control Services >>>>>>>>>>>>>> Kernel driver in use: ixgbe >>>>>>>>>>>>>> Kernel modules: ixgbe >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> eth3 >>>>>>>>>>>>>> 0c:00.1 Ethernet controller: Intel Corporation Ethernet >>>>>>>>>>>>>> Controller 10 Gigabit X540-AT2 (rev 01) >>>>>>>>>>>>>> Subsystem: Intel Corporation Ethernet 10G 2P = X540-t >>>>>>>>>>>>>> Adapter >>>>>>>>>>>>>> Flags: bus master, fast devsel, latency 0, IRQ 16 >>>>>>>>>>>>>> Memory at d8000000 (64-bit, prefetchable) = [size=3D2M] >>>>>>>>>>>>>> Memory at d83f8000 (64-bit, prefetchable) = [size=3D16K] >>>>>>>>>>>>>> Expansion ROM at d8200000 [disabled] [size=3D512K] >>>>>>>>>>>>>> Capabilities: [40] Power Management version 3 >>>>>>>>>>>>>> Capabilities: [50] MSI: Enable- Count=3D1/1 = Maskable+ >>>>>>>>>>>>>> 64bit+ >>>>>>>>>>>>>> Capabilities: [70] MSI-X: Enable+ Count=3D64 = Masked- >>>>>>>>>>>>>> Capabilities: [a0] Express Endpoint, MSI 00 >>>>>>>>>>>>>> Capabilities: [e0] Vital Product Data >>>>>>>>>>>>>> Capabilities: [100] Advanced Error Reporting >>>>>>>>>>>>>> Capabilities: [140] Device Serial Number >>>>>>>>>>>>>> a0-36-9f-ff-ff-0d-f0-98 >>>>>>>>>>>>>> Capabilities: [150] Alternative Routing-ID >>>>>>>>>>>>>> Interpretation (ARI) >>>>>>>>>>>>>> Capabilities: [160] Single Root I/O Virtualization >>>>>>>>>>>>>> (SR-IOV) >>>>>>>>>>>>>> Capabilities: [1d0] Access Control Services >>>>>>>>>>>>>> Kernel driver in use: ixgbe >>>>>>>>>>>>>> Kernel modules: ixgbe >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> After loading the modules this is the (interesting) lsmod >>>>>>>>>>>>>> output: >>>>>>>>>>>>>> $ lsmod >>>>>>>>>>>>>> Module Size Used by >>>>>>>>>>>>>> ixgbe 160150 0 >>>>>>>>>>>>>> netmap_lin 102855 1 ixgbe >>>>>>>>>>>>>> dca 14728 1 ixgbe >>>>>>>>>>>>>> mdio 13559 1 ixgbe >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> And the following is the (interesting) dmesg output: >>>>>>>>>>>>>> [175397.625200] 521.870604 netmap_new_obj_allocator [426] >>>>>>>>>>>>>> objsize 1024 clustsize 4096 objects 4 >>>>>>>>>>>>>> [175397.625501] 521.870912 netmap_new_obj_allocator [504] >>>>>>>>>>>>>> Pre-allocated 128 clusters (4/512KB) for 'netmap_if' >>>>>>>>>>>>>> [175397.625653] 521.871065 netmap_new_obj_allocator [426] >>>>>>>>>>>>>> objsize 36864 clustsize 36864 objects 1 >>>>>>>>>>>>>> [175397.629906] 521.875312 netmap_new_obj_allocator [504] >>>>>>>>>>>>>> Pre-allocated 200 clusters (36/7200KB) for 'netmap_ring' >>>>>>>>>>>>>> [175397.630064] 521.875477 netmap_new_obj_allocator [426] >>>>>>>>>>>>>> objsize 2048 clustsize 4096 objects 2 >>>>>>>>>>>>>> [175397.708572] 521.953976 netmap_new_obj_allocator [504] >>>>>>>>>>>>>> Pre-allocated 50000 clusters (4/200000KB) for = 'netmap_buf' >>>>>>>>>>>>>> [175397.708737] 521.954147 netmap_memory_init [554] Have = 512 >>>>>>>>>>>>>> KB for interfaces, 7200 KB for rings and 195 MB for = buffers >>>>>>>>>>>>>> [175397.708885] netmap: loaded module with 202 Mbytes >>>>>>>>>>>>>> [175418.013476] ixgbe: Intel(R) 10 Gigabit PCI Express = Network >>>>>>>>>>>>>> Driver - version 3.6.7-k >>>>>>>>>>>>>> [175418.013479] ixgbe: Copyright (c) 1999-2011 Intel >>>>>>>>>>>>>> Corporation. >>>>>>>>>>>>>> [175418.013531] ixgbe 0000:0c:00.0: PCI INT B -> GSI 17 >>>>>>>>>>>>>> (level, low) -> IRQ 17 >>>>>>>>>>>>>> [175418.013543] ixgbe 0000:0c:00.0: setting latency timer = to 64 >>>>>>>>>>>>>> [175418.425782] ixgbe 0000:0c:00.0: irq 271 for MSI/MSI-X >>>>>>>>>>>>>> [175418.425791] ixgbe 0000:0c:00.0: irq 272 for MSI/MSI-X >>>>>>>>>>>>>> [175418.425798] ixgbe 0000:0c:00.0: irq 273 for MSI/MSI-X >>>>>>>>>>>>>> [175418.425806] ixgbe 0000:0c:00.0: irq 274 for MSI/MSI-X >>>>>>>>>>>>>> [175418.425813] ixgbe 0000:0c:00.0: irq 275 for MSI/MSI-X >>>>>>>>>>>>>> [175418.425829] ixgbe 0000:0c:00.0: Multiqueue Enabled: = Rx >>>>>>>>>>>>>> Queue count =3D 4, Tx Queue count =3D 4 >>>>>>>>>>>>>> [175418.485972] ixgbe 0000:0c:00.0: (PCI = Express:2.5GT/s:Width >>>>>>>>>>>>>> x8) a0:36:9f:0d:f0:98 >>>>>>>>>>>>>> [175418.646372] ixgbe 0000:0c:00.0: MAC: 3, PHY: 3, PBA = No: >>>>>>>>>>>>>> G44743-006 >>>>>>>>>>>>>> [175418.788262] ixgbe 0000:0c:00.0: Intel(R) 10 Gigabit >>>>>>>>>>>>>> Network Connection >>>>>>>>>>>>>> [175418.788285] ixgbe 0000:0c:00.1: PCI INT A -> GSI 16 >>>>>>>>>>>>>> (level, low) -> IRQ 16 >>>>>>>>>>>>>> [175418.788296] ixgbe 0000:0c:00.1: setting latency timer = to 64 >>>>>>>>>>>>>> [175419.213734] ixgbe 0000:0c:00.1: irq 276 for MSI/MSI-X >>>>>>>>>>>>>> [175419.213742] ixgbe 0000:0c:00.1: irq 277 for MSI/MSI-X >>>>>>>>>>>>>> [175419.213750] ixgbe 0000:0c:00.1: irq 278 for MSI/MSI-X >>>>>>>>>>>>>> [175419.213758] ixgbe 0000:0c:00.1: irq 279 for MSI/MSI-X >>>>>>>>>>>>>> [175419.213766] ixgbe 0000:0c:00.1: irq 280 for MSI/MSI-X >>>>>>>>>>>>>> [175419.213782] ixgbe 0000:0c:00.1: Multiqueue Enabled: = Rx >>>>>>>>>>>>>> Queue count =3D 4, Tx Queue count =3D 4 >>>>>>>>>>>>>> [175419.273937] ixgbe 0000:0c:00.1: (PCI = Express:2.5GT/s:Width >>>>>>>>>>>>>> x8) a0:36:9f:0d:f0:9a >>>>>>>>>>>>>> [175419.434299] ixgbe 0000:0c:00.1: MAC: 3, PHY: 3, PBA = No: >>>>>>>>>>>>>> G44743-006 >>>>>>>>>>>>>> [175419.576197] ixgbe 0000:0c:00.1: Intel(R) 10 Gigabit >>>>>>>>>>>>>> Network Connection >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> This is the pkt-gen output: >>>>>>>>>>>>>> $ sudo ./pkt-gen -i eth2 tx -n 10000 >>>>>>>>>>>>>> extract_ip_range [119] extract IP range from 10.0.0.1 >>>>>>>>>>>>>> extract_ip_range [129] 10.0.0.1 starts at 10.0.0.1 >>>>>>>>>>>>>> extract_ip_range [119] extract IP range from 10.1.0.1 >>>>>>>>>>>>>> extract_ip_range [129] 10.1.0.1 starts at 10.1.0.1 >>>>>>>>>>>>>> extract_mac_range [135] extract MAC range from >>>>>>>>>>>>>> a0:36:9f:0d:f0:98 >>>>>>>>>>>>>> extract_mac_range [150] a0:36:9f:0d:f0:98 starts at >>>>>>>>>>>>>> a0:36:9f:d:f0:98 >>>>>>>>>>>>>> extract_mac_range [135] extract MAC range from >>>>>>>>>>>>>> ff:ff:ff:ff:ff:ff >>>>>>>>>>>>>> extract_mac_range [150] ff:ff:ff:ff:ff:ff starts at >>>>>>>>>>>>>> ff:ff:ff:ff:ff:ff >>>>>>>>>>>>>> main [1053] map size is 207712 Kb >>>>>>>>>>>>>> main [1059] Unable to get if info for eth2 >>>>>>>>>>>>>> main [1066] bad nthreads 1, have 0 queues >>>>>>>>>>>>>> main [1075] mmapping 207712 Kbytes >>>>>>>>>>>>>> main [1094] Unable to register interface eth2 >>>>>>>>>>>>>> Receiving from eth2: 0 queues, 1 threads and 1 cpus. >>>>>>>>>>>>>> main [1128] Wait 2 secs for phy reset >>>>>>>>>>>>>> main [1130] Ready... >>>>>>>>>>>>>> main [1181] Unable to register eth2 >>>>>>>>>>>>>> main [1242] 0 pps >>>>>>>>>>>>>> Received 0 packets, in 0.00 seconds. >>>>>>>>>>>>>> Speed: -nanpps. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Dmesg after trying to use pkt-gen on eth2: >>>>>>>>>>>>>> [175537.838503] ADDRCONF(NETDEV_UP): eth2: link is not = ready >>>>>>>>>>>>>> [175542.802288] ixgbe 0000:0c:00.0: eth2: NIC Link is Up = 10 >>>>>>>>>>>>>> Gbps, Flow Control: RX/TX >>>>>>>>>>>>>> [175542.802607] ADDRCONF(NETDEV_CHANGE): eth2: link = becomes >>>>>>>>>>>>>> ready >>>>>>>>>>>>>> [175553.256029] eth2: no IPv6 routers present >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> -- >>>>>>>>>>>>> Tahir Rauf >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> -- >>>>>>>>>>>> Tahir Rauf >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> -- >>>>>>>>>> Tahir Rauf >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> -- >>>>>>>> Tahir Rauf >>>>>>>>=20 >>>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Tahir Rauf >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> Tahir Rauf >>>>=20 >>>>=20 >>>=20 >>=20 >>=20 >> -- >> Tahir Rauf >>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D9E78ACE-2E40-4EE8-AC0F-2BCA8DA6D158>