Date: Fri, 10 Sep 2004 08:50:36 +0300 From: Vlad GALU <dudu@diaspar.rdsnet.ro> To: freebsd-net@freebsd.org Subject: Re: [TEST/REVIEW] Netflow implementation Message-ID: <20040910085036.11a5d3b3.dudu@diaspar.rdsnet.ro> In-Reply-To: <20040910053619.GB14470@cell.sick.ru> References: <20040905121111.GA78276@cell.sick.ru> <4140834C.3000306@freebsd.org> <20040909171018.GA11540@cell.sick.ru> <414093DE.A6DC6E67@freebsd.org> <Pine.BSF.4.53.0409091743120.51837@e0-0.zab2.int.zabbadoz.net> <41409CB5.836DE816@freebsd.org> <20040909193507.GA12168@cell.sick.ru> <4140B603.8E979D72@freebsd.org> <20040909200052.GD12168@cell.sick.ru> <20040909234126.3f1c7cf3.dudu@diaspar.rdsnet.ro> <20040910053619.GB14470@cell.sick.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 10 Sep 2004 09:36:19 +0400 Gleb Smirnoff <glebius@freebsd.org> wrote: > On Thu, Sep 09, 2004 at 11:41:26PM +0300, Vlad GALU wrote: > V> This made me raise my eyebrow. I wrote a small tool that we use > V> in > V> production at RDS: http://freshmeat.net/projects/glflow. The way I > V> designed it, it is supposed to clean up the flow tree once in a > V> while and remove 'old' flows (that haven't had any packet matching > V> them in the last X seconds). The problem is that I currently have > V> about 400-500k active flows on a 700Mbps link. Every 10 seconds the > V> software removes about 100-200k of them in no more than 0.2-0.3 > V> seconds. Of course, I couldn't possibly send them over a socket > V> somewhere else at that speed,, and chose to open a tempfile, mmap() > V> it, write the expired flows to the buffer. When the buffer exceeds > V> a programatically chosen number of packets, it is msync()-ed, > V> munmap()-ed and a new file is open. > > If you remove 100-200k of flows every 10 seconds, this means you have > 10 - 20 kpps of worm traffic or more! Impressive. That, and the fact that we get a DoS attack every minute :( > 200k flows expired in 10 seconds is 666 export dgrams per second, 7 > times more than I usually have in my testbed. Not sure, that > ng_netflow + ng_ksocket can do this. > > Do you collect all this traffic on a single router? > For now, yes. I've planned into developing this to a distributed solution, using the spread toolkit (www.spread.org). My goal is to have an instance of glflow running in every branch of our national backbone and filter the destination from the very source of the flood, so we can save traffic. Right now there's only one instance running at the entrypoint of our national backbone ring. However, since I'm going to leave RDS in a week or so, I'll probably find another testbed. I hope it will be one with at least half the traffic we have here. > V> Do you accidentally have a better storage model ? I've been > V> trying to > V> dump these binary files to SQL but for a 42 meg binary log the > V> necessary SQL storage went to about 150 megs, which is a bit over > V> reasonable, considering the fact that the software dumps a binary > V> file every 5 to 10 seconds. > > Dumping to SQL is a bad idea. I have tried it, too :) > > V> P.S. I haven't yet tried to aggregate the flows between reading > V> them from the binary file and inserting the data into SQL. I > V> thought it would take too much time to be able to keep up with the > V> newly created dumps. > > This is how many people do. However I don't know details. > > -- > Totus tuus, Glebius. > GLEBIUS-RIPN GLEB-RIPE > ---- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040910085036.11a5d3b3.dudu>