Date: Mon, 18 Nov 2002 14:54:56 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: David Gilbert <dgilbert@velocet.ca>, dolemite@wuli.nu, freebsd-hackers@freebsd.org, freebsd-net@freebsd.org Subject: Small initial LRP processing patch vs. -current Message-ID: <3DD96FC0.B77331A1@mindspring.com> References: <20021109180321.GA559@unknown.nycap.rr.com> <3DCD8761.5763AAB2@mindspring.com> <15823.51640.68022.555852@canoe.velocet.net> <3DD1865E.B9C72DF5@mindspring.com> <15826.24074.605709.966155@canoe.velocet.net> <3DD2F33E.BE136568@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> Well... I have all those stats, but I wasn't wanting to type that > much. IIRC, we normally test with 80 byte packets ... they can be UDP > or TCP ... we're testing the routing. The box has two interfaces and > we measure the number of PPS that get to the box on the other side. > > Without polling patches, the single processor box definately > experiences live lock. Interestingly, the degree of livelock is > fairly motherboard dependant. We have tested many cards and so far > fxp's are our best performers. Please try the attached patch, with and without DEVICE_POLLING. The patch gets rid of the NETISR calls to ipintr(), by calling the ip_input() routine directly from the ether_input() called at input interrupt time by the ethernet controller. The effect of this should be that the input processing will run to completion at interrupt time, and therefore avoid input livelock, all the way up to the top of the TCP stack, but not past the bottom of the sockets layer (I have patches to take it all the way to the system call layer). What this will basically do is get rid of the latency in the NETISR delay for processing input IP packets via ipintr, and it should avoid locking out IP and TCP processing, if the interrupt load is very high. I've tested this locally, with no ill effects (so far). This should drastically increase your packets per second number for performance under load. It will also increase your connections per second number on the peak before falloff, by about a factor of 1.5 (I have been unable to load it sufficiently in the DEVICE_POLLING case to guess a number in the non-falloff case). The connections per second number will be better when connection requests run to completion from the TCP stack, up through the sockets layer (I will provide patches for this, after you have tested this one). The basic theory here is that ipintr processing can be delayed indefinitely, if interrupt load is high enough, and there will be a maximum latency of 10ms for IP processing after ether_input(), in the normal stack case, without the patches. Let me know how this works. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3DD96FC0.B77331A1>