From owner-freebsd-ipfw@FreeBSD.ORG  Tue Nov 13 20:15:43 2012
Return-Path: <owner-freebsd-ipfw@FreeBSD.ORG>
Delivered-To: freebsd-ipfw@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id 7817DF56;
 Tue, 13 Nov 2012 20:15:43 +0000 (UTC) (envelope-from bright@mu.org)
Received: from elvis.mu.org (elvis.mu.org [192.203.228.196])
 by mx1.freebsd.org (Postfix) with ESMTP id 571AB8FC0C;
 Tue, 13 Nov 2012 20:15:43 +0000 (UTC)
Received: from kruse-124.4.ixsystems.com (drawbridge.ixsystems.com
 [206.40.55.65])
 by elvis.mu.org (Postfix) with ESMTPSA id 8056C1A3C1A;
 Tue, 13 Nov 2012 12:15:37 -0800 (PST)
Message-ID: <50A2AA89.9060309@mu.org>
Date: Tue, 13 Nov 2012 12:16:09 -0800
From: Alfred Perlstein <bright@mu.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:16.0) Gecko/20121026 Thunderbird/16.0.2
MIME-Version: 1.0
To: "Alexander V. Chernikov" <melifaro@yandex-team.ru>
Subject: Re: [CFT] ipfw SMP-ready dynamic states
References: <50A29F57.6090701@yandex-team.ru>
In-Reply-To: <50A29F57.6090701@yandex-team.ru>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.14
Cc: freebsd-ipfw@freebsd.org, Luigi Rizzo <rizzo@iet.unipi.it>,
 "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
X-BeenThere: freebsd-ipfw@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: IPFW Technical Discussions <freebsd-ipfw.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-ipfw>,
 <mailto:freebsd-ipfw-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ipfw>
List-Post: <mailto:freebsd-ipfw@freebsd.org>
List-Help: <mailto:freebsd-ipfw-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw>,
 <mailto:freebsd-ipfw-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Nov 2012 20:15:43 -0000

Alexander, this is awesome.

On 11/13/12 11:28 AM, Alexander V. Chernikov wrote:
> Hello list!
>
> Currently most ipfw operations with dynamic states (keep-state, 
> check-state, limit) are serialized via IPFW_DYN_LOCK() which is 
> per-vnet mutex lock.
>
> As a result, performance is limited to the same ~650kpps as in routing
> (in several cases).
>
> Patch changes the following:
> * global lock is changed to per-bucket mutex
> * state expiration is done in ipfw_tick every 1s. No expiration is 
> done on forwarding path
> * hash table resize is done automatically and does not cause all 
> states to be lost
>
> The only (architectural) problem I see is unlocked V_dyn_count 
> increments.
> So, we can do the following:
> 1) lock increments/decrements via some separate mutex
> 2) do nothing
> 3) take some combined approach:
>
> Generally, we don't need value to be _exact_.
> As a result, we count total number of states in every ipfw_tick run 
> and set V_dyn_count to new value. New states still increment 
> V_dyn_count unlocked.
>
What about using per-cpu PCPU counters, and then collecting them for 
display/reporting?

-Alfred


>
> Performance:
>
> Synthetic traffic, ipfw with single allow ip from any to any rule: 2.4M.
> single keep-state ip from any to any: 2.2M.
>
> Some more tests should be taken (with large number of states, 
> different types of traffic, etc), maybe I can do some next week.
>
>
> You need to run recent -current or merge r242631 and r242834 before 
> applying this patch.
>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"