Date: Thu, 15 Jan 2004 02:41:13 +0200 From: Vlad Galu <dudu@diaspar.rdsnet.ro> To: freebsd-net@freebsd.org Subject: Re: Handling 100.000 packets/sec or more Message-ID: <20040115024113.7dbff80f.dudu@diaspar.rdsnet.ro> In-Reply-To: <20040114221531.74b20ebe.dudu@diaspar.rdsnet.ro> References: <Pine.WNT.4.58.0401141048001.2804@ady-home> <20040114221531.74b20ebe.dudu@diaspar.rdsnet.ro>
next in thread | previous in thread | raw e-mail | index | archive | help
--Signature=_Thu__15_Jan_2004_02_41_13_+0200_SI5/SJM9qzwoEsKF Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: 7bit Vlad Galu <dudu@diaspar.rdsnet.ro> writes: |Adrian Penisoara <ady@freebsd.ady.ro> writes: | ||Hi, || || At one site that I administer we have a gateway server which |services|a large SOHO LAN (more than 300 stations) and I'm facing a |serious|issue: very often we see strong spoofed floods (variable source |IP and|port, variable destination IP, destination port 80) which can go |as far|as 100 000 packets/sec! || || Of course, the server (FreeBSD 5.2-REL, PIII 733Mhz, 256Mb RAM, 3COM ||3C905B-TX aka xl0 with checksum offloading support) has a hard time ||swallowing this kind of traffic. The main issue are the IRQ |interrupts:|over 15000 interrupts/sec which consume more than 90% of |the CPU time.|We got ingress filtering so the packets go no further |than the firewall|(which, BTW, is not the issue, even disabling it it's |the same|problem). The system is still responsive but the load average |goes as|high as 10 and the interface is losing packets (input errors) |which|dramatically affects legitimate traffic, besides mbuf(9) |starvation. We|are taking down the culprit clients, but this takes time |and we need|the other clients not to be affected by it. || || What can I do to make the system better handle this kind of traffic |?|Could device polling(8) or just increasing the kernel frequency clock ||to 1000Hz or more improve the situation ? || What kind of network cards could face a lot better this burden ? Are ||there any other solutions ? I have just had an idea. It might be stupid, but at least it's something :) Why not try to use splnet() & co for lowering the priority of the NIC irq's a little bit ? I mean, just at least to a state where the system is able to respond to other tasks than forwarding packets, without loosing them. Anyone, please slap me if this is bullshit. || | Try fxp. It has better polling support, and there's the advantage of |the link0 flag. When it's set, the interface won't send interrupts to |the kernel for each packet it catches from the wire, but instead will |wait until its own buffer is full, and generate an interrupt |afterwards. It should be a great deal of improvement when asociated |with device polling. As you surely know, when the kernel receives an |interrupt from an interface, it masks all further interrupts and |schedules a polling task instead. | In matters of hardware improvements, I can only think of upgrading to | a |system board with a higher bandwidth between the south and the north |bridges, and of course, using a CPU that has a higher clock rate. | || On a side note: what would be a adequate formula to calculate the ||NMBCLUSTERS and MBUFS we should set on this server (via boot-time ||kern.ipc.nmbclusters and kern.ipc.nmbufs) ? || | | I'm still thinking about that ... | || Thank you. || ||-- ||Adrian Penisoara ||Ady (@freebsd.ady.ro) || ||_______________________________________________ ||freebsd-net@freebsd.org mailing list ||http://lists.freebsd.org/mailman/listinfo/freebsd-net ||To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" || | | |---- |If it's there, and you can see it, it's real. |If it's not there, and you can see it, it's virtual. |If it's there, and you can't see it, it's transparent. |If it's not there, and you can't see it, you erased it. | ---- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. --Signature=_Thu__15_Jan_2004_02_41_13_+0200_SI5/SJM9qzwoEsKF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFABeGrP5WtpVOrzpcRApu5AJ97aq3uFauL7XEK2Uz308JSjeoGsACgg/hZ WWXNAwZ7bb7GMdkNCfDe2L8= =SqnS -----END PGP SIGNATURE----- --Signature=_Thu__15_Jan_2004_02_41_13_+0200_SI5/SJM9qzwoEsKF--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040115024113.7dbff80f.dudu>