Date: Mon, 2 Feb 2015 17:43:44 -0600 From: Xiaoye Sun <sunxiaoye07@gmail.com> To: net@freebsd.org, Luigi Rizzo <rizzo@iet.unipi.it> Cc: "yx15@rice.edu" <yx15@rice.edu> Subject: netmap nic to host stack forwarding Message-ID: <CAJnByzgUC2u8x-Ua3XYr_PfHQL4wZRcRm_qktG=yx_U6A5fmWA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi, I am new to netmap. What I want to do with netmap is to implement file transfer protocol so that netmap can help achieve fast network IO. In the netmap client program, I want that the unrelated packets are forwarded to the linux host stack and all the packets from the host stack are forwarded to the NIC also. That is, besides the netmap client achieving our file transfer protocol, all the other netowrk applications running on top of the host stack are not affected. Our machine configuration is as follows: Linux 3.11 + Myricom 10G nic + original Myricom driver coming in the linux kernel, not the modified one. We start to play with the bridge.c example in the netmap google code repository. We want to use it in a way so that the netmap client forwards packets from nic to the host stack and vice versa. So we use the command as follows sudo ./bridge -i netmap:eth3 to run the bridge netmap client. We did not change anything in the bridge.c code and the netmap source code. We test the functionality with the ping command line in linux. In our setting, another machine (not running netmap) sends ping request message to the netmap client machine and waits for the reply message. By looking at the tcpdump, we found that all the packets got a copy. That is, in the tcpdump results, there is a duplicate for each packet. Here is the output of tcpdump -i eth3: 17:40:32.708585 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 1, length 64 17:40:32.708647 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 1, length 64 17:40:32.708721 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 1, length 64 17:40:32.708760 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 1, length 64 17:40:33.708410 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 2, length 64 17:40:33.708454 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 2, length 64 17:40:33.708507 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 2, length 64 17:40:33.708540 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 2, length 64 17:40:34.708400 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 3, length 64 17:40:34.708441 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910, seq 3, length 64 17:40:34.708488 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 3, length 64 17:40:34.708516 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910, seq 3, length 64 Here is the output of the bridge.c in verbose mode 420.122815 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 422.625336 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 425.127857 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 427.630354 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 430.132872 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 432.635393 main [749] poll timeout [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 0 rx 0@0 tx 1023 432.708611 main [749] poll ok [0] ev 1 0 rx 0@0 tx 1023, [1] ev 1 1 rx 1@0 tx 1023 432.708635 main [749] poll ok [0] ev 5 4 rx 0@0 tx 1023, [1] ev 0 0 rx 1@0 tx 1023 432.708641 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000 432.708743 main [749] poll ok [0] ev 1 1 rx 1@0 tx 1023, [1] ev 1 0 rx 0@1 tx 1023 432.708751 main [749] poll ok [0] ev 0 0 rx 1@0 tx 1023, [1] ev 5 4 rx 0@1 tx 1023 432.708755 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000 433.708430 main [749] poll ok [0] ev 1 0 rx 0@1 tx 1023, [1] ev 1 1 rx 1@1 tx 1022 433.708445 main [749] poll ok [0] ev 5 4 rx 0@1 tx 1023, [1] ev 0 0 rx 1@1 tx 1022 433.708449 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000 433.708525 main [749] poll ok [0] ev 1 1 rx 1@1 tx 1023, [1] ev 1 0 rx 0@2 tx 1022 433.708532 main [749] poll ok [0] ev 0 0 rx 1@1 tx 1023, [1] ev 5 4 rx 0@2 tx 1022 433.708536 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000 434.708416 main [749] poll ok [0] ev 1 0 rx 0@2 tx 1023, [1] ev 1 1 rx 1@2 tx 1022 434.708432 main [749] poll ok [0] ev 5 4 rx 0@2 tx 1023, [1] ev 0 0 rx 1@2 tx 1022 434.708437 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000 434.708497 main [749] poll ok [0] ev 1 1 rx 1@2 tx 1023, [1] ev 1 0 rx 0@3 tx 1022 434.708505 main [749] poll ok [0] ev 0 0 rx 1@2 tx 1023, [1] ev 5 4 rx 0@3 tx 1022 434.708511 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000 437.211025 main [749] poll timeout [0] ev 1 0 rx 0@3 tx 1023, [1] ev 1 0 rx 0@3 tx 1022 437.709106 main [749] poll ok [0] ev 1 1 rx 1@3 tx 1023, [1] ev 1 0 rx 0@3 tx 1022 437.709116 main [749] poll ok [0] ev 0 0 rx 1@3 tx 1023, [1] ev 5 4 rx 0@3 tx 1022 437.709121 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000 437.709226 main [749] poll ok [0] ev 1 0 rx 0@4 tx 1023, [1] ev 1 1 rx 1@3 tx 1022 437.709235 main [749] poll ok [0] ev 5 4 rx 0@4 tx 1023, [1] ev 0 0 rx 1@3 tx 1022 437.709240 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000 440.211745 main [749] poll timeout [0] ev 1 0 rx 0@4 tx 1023, [1] ev 1 0 rx 0@4 tx 1022 442.714271 main [749] poll timeout [0] ev 1 0 rx 0@4 tx 1023, [1] ev 1 0 rx 0@4 tx 1022 444.026300 main [749] poll timeout [0] ev 1 0 rx 0@4 tx 1023, [1] ev 1 0 rx 0@4 tx 1022 I am wondering if the output looks normal. In particular, is this the expected behavior of the bridge.c example? In addition, how to make the netmap client only send one copy of the packets to the network host stack and how to drop the packets in the netmap client? Do you think the problem is due to the unmodified Myricom driver? Thanks! Best, Xiaoye -- Xiaoye Steven Sun, Ph.D. Student 6100 Main St. MS-366 Department of Electrical and Computer Engineering (ECE) & Department of Computer Science (CS) George R. Brown School of Engineering Rice University, Houston, Texas, USA
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJnByzgUC2u8x-Ua3XYr_PfHQL4wZRcRm_qktG=yx_U6A5fmWA>