Date: Wed, 2 Oct 2013 13:18:38 -0700 From: hiren panchasara <hiren.panchasara@gmail.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: netmap: understanding pkg-gen.c Message-ID: <CALCpEUG1dM9Z6OG1BqRAPAfHciANXxaYVpezDEOUNug22QN6HQ@mail.gmail.com> In-Reply-To: <CA%2BhQ2%2BjYyoxpLEoz0kHm8Hikfgf8Fke-o_ctkO3fKBdGqEz7wQ@mail.gmail.com> References: <CALCpEUHj_Mke_njM0NwzCUa1f_pXZdCy3_aaqJtKz5NUJUU8Mg@mail.gmail.com> <CA%2BhQ2%2BjYyoxpLEoz0kHm8Hikfgf8Fke-o_ctkO3fKBdGqEz7wQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks Luigi. Coming back to this thread to actually understand what's going on. On Tue, Sep 24, 2013 at 8:37 PM, Luigi Rizzo <rizzo@iet.unipi.it> wrote: > > > > On this box, I have 2 interfaces igb0 (10.73.149.28) and ix1 > (10.73.149.17) > > and this is how I am using this binary: > > > > -bash-4.2$ sudo ./pkt-gen -i ix1 -f tx -n 100000000 -c 8 -p 8 -d > > 10.73.149.17 -s 10.73.149.28 > So, my intention is to *send* 10gbps data to ix1 and see the card use all of its 8 queues. Above command is the correct one? I kldunloaded/loaded ixbge to clear out all the stats and tested it again: -bash-4.2$ sudo ./pkt-gen -i ix1 -f tx -n 100000000 -c 8 -p 8 -d 10.73.149.17 -s 10.73.149.28 extract_ip_range [143] extract IP range from 10.73.149.28 extract_ip_range [178] range is 10.73.149.28 0 to 10.73.149.28 0 extract_ip_range [143] extract IP range from 10.73.149.17 extract_ip_range [178] range is 10.73.149.17 0 to 10.73.149.17 0 extract_mac_range [184] extract MAC range from 90:e2:ba:30:68:c5 extract_mac_range [199] 90:e2:ba:30:68:c5 starts at 90:e2:ba:30:68:c5 extract_mac_range [184] extract MAC range from ff:ff:ff:ff:ff:ff extract_mac_range [199] ff:ff:ff:ff:ff:ff starts at ff:ff:ff:ff:ff:ff main [1530] map size is 334980 Kb main [1552] mapping 334980 Kbytes Sending on ix1: 8 queues, 8 threads and 8 cpus. 10.73.149.28 -> 10.73.149.17 (90:e2:ba:30:68:c5 -> ff:ff:ff:ff:ff:ff) main [1622] Sending 512 packets every 0.000000000 ns main [1624] Wait 2 secs for phy reset main [1626] Ready... sender_body [775] start sender_body [775] start sender_body [775] start sender_body [775] start sender_body [775] start sender_body [775] start sender_body [775] start sender_body [848] drop copy sender_body [775] start sender_body [848] drop copy sender_body [848] drop copy sender_body [848] drop copy sender_body [848] drop copy sender_body [848] drop copy sender_body [848] drop copy main_thread [1192] 15122963 pps (15130434 pkts in 1000494 usec) main_thread [1192] 14881444 pps (14896266 pkts in 1000996 usec) sender_body [841] poll error/timeout on queue 1 main_thread [1192] 14880708 pps (15659371 pkts in 1052327 usec) main_thread [1192] 14878611 pps (14888684 pkts in 1000677 usec) main_thread [1192] 14882655 pps (14897538 pkts in 1001000 usec) main_thread [1192] 11900044 pps (12029754 pkts in 1010900 usec) main_thread [1212] ouch, thread 1 exited with error Sent 87502047 packets, 60 bytes each, in 5.86 seconds. Speed: 14.92 Mpps Bandwidth: 7.16 Gbps (raw 10.03 Gbps) -bash-4.2$ But looking at the queue stats: -bash-4.2$ sysctl -a | grep ix.1 | grep queue | grep rx_packets dev.ix.1.queue0.rx_packets: 171 dev.ix.1.queue1.rx_packets: 0 dev.ix.1.queue2.rx_packets: 0 dev.ix.1.queue3.rx_packets: 0 dev.ix.1.queue4.rx_packets: 0 dev.ix.1.queue5.rx_packets: 0 dev.ix.1.queue6.rx_packets: 0 dev.ix.1.queue7.rx_packets: 0 And after a few seconds: -bash-4.2$ sysctl -a | grep ix.1 | grep queue | grep rx_packets dev.ix.1.queue0.rx_packets: 310 dev.ix.1.queue1.rx_packets: 0 dev.ix.1.queue2.rx_packets: 0 dev.ix.1.queue3.rx_packets: 0 dev.ix.1.queue4.rx_packets: 8 dev.ix.1.queue5.rx_packets: 0 dev.ix.1.queue6.rx_packets: 0 dev.ix.1.queue7.rx_packets: 0 -bash-4.2$ What is going on here? Should I be seeing more pkts in rx_packets? Should I see more queues being used? I am using stock ixgbe at this point. I believe RSS is enabled by default? I apologize if I am asking obvious/answered questions here. cheers, Hiren
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALCpEUG1dM9Z6OG1BqRAPAfHciANXxaYVpezDEOUNug22QN6HQ>