Date: Tue, 26 Jan 2016 10:53:11 +0300 From: Pavel Odintsov <pavel.odintsov@gmail.com> To: Xiaoye Sun <Xiaoye.Sun@rice.edu> Cc: freebsd-net@freebsd.org Subject: Re: swaping ring slots between NIC ring and Host ring does not always success Message-ID: <CALgsdbd3XuE3wMYp4ey%2B1aer%2BHSVNojLYoVqwqTBPAXXdf9i%2BQ@mail.gmail.com> In-Reply-To: <CAJnByzj6Dj3vouZ2NbxqvCV-2-7TVtTR4FaWKuCFaaRN2X%2ByAA@mail.gmail.com> References: <CAJnByzj6Dj3vouZ2NbxqvCV-2-7TVtTR4FaWKuCFaaRN2X%2ByAA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello! Really useful and detailed research. Thanks a lot! Could you share your validation code? It could be useful for consistency tests for netmap. So I could not help with your original question. Let's wait answer from developers of awesome Netmap :) On Tue, Jan 26, 2016 at 2:55 AM, Xiaoye Sun <Xiaoye.Sun@rice.edu> wrote: > Hi all, > > I wrote a netmap application that is very similar to bridge.c in master/example > directory of the netmap code. The major difference between my program and > bridge.c is that my application also creates custom packets that are > directly put into the tx ring of the NIC (like a packet generator with > NIC-Host packet forwarding). Each generated packet has an unique sequence > number so that the receiver can tell if a packet is lost. > > Like bridge.c, the packet forwarding between the nic and the host uses > 'zerocopy', so that the program only swaps the buf_idx of the slots to > virtually forward the packet to the other end. NS_BUF_CHANGED is set for > the slots (both rx and tx) whose buf_idx has been changed due to zerocopy. > > I set the number of rings on nic to 1 using the ethtool command, i.e., one > pair of netmap rings for the nic (one for host tx and one for host rx) and > another pair of rings for the host (one for host tx and one for host rx). > > However, at the receiver side, I found that there is a chance (very little > chance, less than 1% of the swaps) where swapping the buf_idx does not > success. The receiver might get the packet in the buffer swapped out. For > example, the netmap program wants to swap host slot *SH* with NIC slot *SN* > in order to send the packet in *SH* from host to the receiver; the receiver > might get the packet in *SN* instead. This usually happens to the very end > of the available slots. > > Our experiment is done on *Linux* machine (Linux 3.16.0-4-amd64 #1 SMP > Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux) with* intel NIC* > (Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection) using > the *ixgbe* driver. > > To understand this problem, I look into the patch code to the ixgbe driver. > I found that the flag NS_BUF_CHANGED is not used in the linux driver for > ixgbe. please look the header file /LINUX/ixgbe_netmap_linux.h. In funtions > ixgbe_netmap_txsync and ixgbe_netmap_rxsync, the function call > netmap_reload_map is commented out. However, the ixgbe's driver patch for > FreeBSD calls the reload function. > > So, I am wondering if this is a known problem when using netmap on LINUX. > Has anybody found the same problem? > Is netmap_reload_map required in Linux? why it is not called in the driver > patch? > What is the recommended solution for this problem? > > Thanks! > > Best, > Xiaoye > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Sincerely yours, Pavel Odintsov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALgsdbd3XuE3wMYp4ey%2B1aer%2BHSVNojLYoVqwqTBPAXXdf9i%2BQ>