Date: Tue, 4 Sep 2007 13:35:49 -0700 (PDT) From: Weiguang Shi <wgshizz@yahoo.com> To: Gleb Smirnoff <glebius@FreeBSD.org> Cc: maxim@FreeBSD.org, freebsd-net@FreeBSD.org Subject: Re: questions wrt ng_netflow Message-ID: <396207.74117.qm@web43144.mail.sp1.yahoo.com>
next in thread | raw e-mail | index | archive | help
Thanks! That all make sense.=0A=0AWei=0A=0A----- Original Message ----=0AFr=
om: Gleb Smirnoff <glebius@FreeBSD.org>=0ATo: Weiguang Shi <wgshizz@yahoo.c=
om>=0ACc: maxim@FreeBSD.org; freebsd-net@FreeBSD.org=0ASent: Saturday, Sept=
ember 1, 2007 1:51:38 AM=0ASubject: Re: questions wrt ng_netflow=0A=0A Wei=
guang,=0A=0A sorry for late answer, I'm too loaded with daytime job.=0A=0A=
On Thu, Aug 23, 2007 at 09:40:30AM -0700, Weiguang Shi wrote:=0AW> I've bee=
n reading netlfow.c in FreeBSD-6.2 and this piece of code confuses me.=0AW=
> 484 /*=0AW> 485 * Go through hash and fi=
nd our entry. If we encounter an=0AW> 486 * entry, that sh=
ould be expired, purge it. We do a reverse=0AW> 487 * sear=
ch since most active entries are first, and most=0AW> 488 =
* searches are done on most active entries.=0AW> 489 */=0A=
W> 490 TAILQ_FOREACH_REVERSE_SAFE(fle, &hsh->head, fhead, f=
le_hash, fle1) {=0AW> 491 if (bcmp(&r, &fle->f.r, s=
izeof(struct flow_rec)) =3D=3D 0)=0AW> 492 =
break;=0AW> 493 if ((INACTIVE(fle) && SMALL(fle)) |=
| AGED(fle)) {=0AW> 494 TAILQ_REMOVE(&hsh->=
head, fle, fle_hash);=0AW> 495 expire_flow(=
priv, &item, fle, NG_QUEUE);=0AW> 496 atomi=
c_add_32(&priv->info.nfinfo_act_exp, 1);=0AW> 497 }=
=0AW> 498 }=0AW> =0AW> +-------------+ +--------+ =
+--------+ +--------+ +--------+=0AW> | Bucket Head |---=
-->| RecA |----->| RecB |----->| RecC |----->| RecD |=0AW> +---=
----------+ +--------+ +--------+ +--------+ +--------+=
=0AW> =0AW> In the figure above, let's say our packet matches RecC. So befo=
re the=0AW> match, RecD is examined to see if it's AGED, i.e., it's lasted =
for too=0AW> long, or if it's too small and inactive. As the match is found=
, the=0AW> code stops searching.=0AW> =0AW> First, isn't INACTIVE alone eno=
ugh to expire a flow? Why must INACTIVE=0AW> _and_ SMALL?=0A=0ANo. Netflow =
engine tries to minimise number of export records sent, and=0Aavoid splitti=
ng one long flow into several records. Thus, if we have enough=0Aspace in t=
he cache, we keep inactive flows, because they can become active=0Aagain.=
=0A=0AFor example, a TCP ssh session, where you have stopped typing and are=
=0Areading the text becomes inactive after some time passes. However, it wi=
ll=0Acontinue, when you start typeing again.=0A=0AWe make an exclusion for =
SMALL flows, to avoid blowing the cache due to=0Acontinuous internet scanni=
ng by worms:=0A=0A/*=0A * 4 is a magical number: statistically number of 4-=
packet flows is=0A * bigger than 5,6,7...-packet flows by an order of magni=
tude. Most UDP/ICMP=0A * scans are 1 packet (~ 90% of flow cache). TCP scan=
s are 2-packet in case=0A * of reachable host and 4-packet otherwise.=0A */=
=0A#define SMALL(fle) (fle->f.packets <=3D 4)=0A=0AW> RecA and RecB wo=
uld not be examined for expiration but since they are=0AW> to the beginning=
of the queue and therefore actually less recently=0AW> accessed, they are =
more likely to be INACTIVE and could be more AGED.=0AW> I must be missing s=
omething, but what justifies examining RecD but not =0AW> RecA and RecB?=0A=
=0ABecause we are in the interrupt thread. Our aim is to finish processing=
=0Aof one IP packet as fast as possible and return. Our aim is not to expir=
e=0Aas much as possible. However we examine the flows that we have just bcm=
p()'ed.=0AThese entires are in the CPU's cache, so we can quickly check the=
m.=0A=0AThe periodic expiry routine goes through the TAILQ in opposite orde=
r,=0Astarting from head, so it accesses the oldest flows earlier.=0A=0A-- =
=0ATotus tuus, Glebius.=0AGLEBIUS-RIPN GLEB-RIPE=0A=0A=0A=0A=0A=0A =
=0A________________________________________________________________________=
____________=0ATake the Internet to Go: Yahoo!Go puts the Internet in your =
pocket: mail, news, photos & more. =0Ahttp://mobile.yahoo.com/go?refer=3D1G=
NXIC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?396207.74117.qm>