From owner-freebsd-net@FreeBSD.ORG Tue Sep 4 20:35:56 2007 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1203216A41B for ; Tue, 4 Sep 2007 20:35:56 +0000 (UTC) (envelope-from wgshizz@yahoo.com) Received: from web43144.mail.sp1.yahoo.com (web43144.mail.sp1.yahoo.com [216.252.121.74]) by mx1.freebsd.org (Postfix) with SMTP id F0ECA13C4B7 for ; Tue, 4 Sep 2007 20:35:55 +0000 (UTC) (envelope-from wgshizz@yahoo.com) Received: (qmail 74293 invoked by uid 60001); 4 Sep 2007 20:35:55 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=sCPVIudNn9TreBZfkp8GJeVjn+CToCi5JtOZ3GbDPXcP3bBfyHjnjlWDFv6Ao5D07cMszQcss7E1ZVNOfVFEgY/fvYFBEN0rF8IrUzbZeb/mlw+XuWh3MDDfRGmKJeVsSng3SPEbs4HbH2ntYMLQqgkCdCewjARqDlYLF6nePz4=; Received: from [69.147.84.253] by web43144.mail.sp1.yahoo.com via HTTP; Tue, 04 Sep 2007 13:35:49 PDT X-Mailer: YahooMailRC/651.50 YahooMailWebService/0.7.134 Date: Tue, 4 Sep 2007 13:35:49 -0700 (PDT) From: Weiguang Shi To: Gleb Smirnoff MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-ID: <396207.74117.qm@web43144.mail.sp1.yahoo.com> Cc: maxim@FreeBSD.org, freebsd-net@FreeBSD.org Subject: Re: questions wrt ng_netflow X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2007 20:35:56 -0000 Thanks! That all make sense.=0A=0AWei=0A=0A----- Original Message ----=0AFr= om: Gleb Smirnoff =0ATo: Weiguang Shi =0ACc: maxim@FreeBSD.org; freebsd-net@FreeBSD.org=0ASent: Saturday, Sept= ember 1, 2007 1:51:38 AM=0ASubject: Re: questions wrt ng_netflow=0A=0A Wei= guang,=0A=0A sorry for late answer, I'm too loaded with daytime job.=0A=0A= On Thu, Aug 23, 2007 at 09:40:30AM -0700, Weiguang Shi wrote:=0AW> I've bee= n reading netlfow.c in FreeBSD-6.2 and this piece of code confuses me.=0AW= > 484 /*=0AW> 485 * Go through hash and fi= nd our entry. If we encounter an=0AW> 486 * entry, that sh= ould be expired, purge it. We do a reverse=0AW> 487 * sear= ch since most active entries are first, and most=0AW> 488 = * searches are done on most active entries.=0AW> 489 */=0A= W> 490 TAILQ_FOREACH_REVERSE_SAFE(fle, &hsh->head, fhead, f= le_hash, fle1) {=0AW> 491 if (bcmp(&r, &fle->f.r, s= izeof(struct flow_rec)) =3D=3D 0)=0AW> 492 = break;=0AW> 493 if ((INACTIVE(fle) && SMALL(fle)) |= | AGED(fle)) {=0AW> 494 TAILQ_REMOVE(&hsh->= head, fle, fle_hash);=0AW> 495 expire_flow(= priv, &item, fle, NG_QUEUE);=0AW> 496 atomi= c_add_32(&priv->info.nfinfo_act_exp, 1);=0AW> 497 }= =0AW> 498 }=0AW> =0AW> +-------------+ +--------+ = +--------+ +--------+ +--------+=0AW> | Bucket Head |---= -->| RecA |----->| RecB |----->| RecC |----->| RecD |=0AW> +---= ----------+ +--------+ +--------+ +--------+ +--------+= =0AW> =0AW> In the figure above, let's say our packet matches RecC. So befo= re the=0AW> match, RecD is examined to see if it's AGED, i.e., it's lasted = for too=0AW> long, or if it's too small and inactive. As the match is found= , the=0AW> code stops searching.=0AW> =0AW> First, isn't INACTIVE alone eno= ugh to expire a flow? Why must INACTIVE=0AW> _and_ SMALL?=0A=0ANo. Netflow = engine tries to minimise number of export records sent, and=0Aavoid splitti= ng one long flow into several records. Thus, if we have enough=0Aspace in t= he cache, we keep inactive flows, because they can become active=0Aagain.= =0A=0AFor example, a TCP ssh session, where you have stopped typing and are= =0Areading the text becomes inactive after some time passes. However, it wi= ll=0Acontinue, when you start typeing again.=0A=0AWe make an exclusion for = SMALL flows, to avoid blowing the cache due to=0Acontinuous internet scanni= ng by worms:=0A=0A/*=0A * 4 is a magical number: statistically number of 4-= packet flows is=0A * bigger than 5,6,7...-packet flows by an order of magni= tude. Most UDP/ICMP=0A * scans are 1 packet (~ 90% of flow cache). TCP scan= s are 2-packet in case=0A * of reachable host and 4-packet otherwise.=0A */= =0A#define SMALL(fle) (fle->f.packets <=3D 4)=0A=0AW> RecA and RecB wo= uld not be examined for expiration but since they are=0AW> to the beginning= of the queue and therefore actually less recently=0AW> accessed, they are = more likely to be INACTIVE and could be more AGED.=0AW> I must be missing s= omething, but what justifies examining RecD but not =0AW> RecA and RecB?=0A= =0ABecause we are in the interrupt thread. Our aim is to finish processing= =0Aof one IP packet as fast as possible and return. Our aim is not to expir= e=0Aas much as possible. However we examine the flows that we have just bcm= p()'ed.=0AThese entires are in the CPU's cache, so we can quickly check the= m.=0A=0AThe periodic expiry routine goes through the TAILQ in opposite orde= r,=0Astarting from head, so it accesses the oldest flows earlier.=0A=0A-- = =0ATotus tuus, Glebius.=0AGLEBIUS-RIPN GLEB-RIPE=0A=0A=0A=0A=0A=0A = =0A________________________________________________________________________= ____________=0ATake the Internet to Go: Yahoo!Go puts the Internet in your = pocket: mail, news, photos & more. =0Ahttp://mobile.yahoo.com/go?refer=3D1G= NXIC