From owner-freebsd-hackers@freebsd.org Thu Apr 28 17:20:05 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E846B1F8AD for ; Thu, 28 Apr 2016 17:20:05 +0000 (UTC) (envelope-from zclaudio@bsd.com.br) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB75F11D5 for ; Thu, 28 Apr 2016 17:20:04 +0000 (UTC) (envelope-from zclaudio@bsd.com.br) Received: by mail-wm0-x231.google.com with SMTP id e201so84944256wme.0 for ; Thu, 28 Apr 2016 10:20:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=o6WpFxMkpMTB0ydlNtOhUZS15v0+YuQZ+jihMtMpncI=; b=gA2JbyWhZ6WYNW1w8SjMRJ4aQAFwIZoIgfHp0xuEwnVYSgaXL1UWCX7UneoxY10xNs VQSWtPUuOzh6+1vOxTApGM5u4dQRFJfxVJt6KNyESrMQDKowMv0KAJt/Z/aFR4Z/447k 1v1aJ+lxQ0NBDry710y6KbARB/rDO4Yal9t3M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=o6WpFxMkpMTB0ydlNtOhUZS15v0+YuQZ+jihMtMpncI=; b=GSutG1lgsMpryhnGHET0s3A3DgY4lFG5dzLfs+Gf+FRmEQbCMqMopuUoQGPcfipYdb aDZk818zLbkgmtJcvNLHPSxqClrbY1Bef+2OM/dXvXRtTKClbsrDZNp8GLQATuJWJ0wy JmaSxHNH8/Ew04AYjkxh1qXz/YdnIOyNfvqV9pzY4AT4Ft66gFGKdCiv0FZyMf4b8XMu dQ/0Hace5LsKYStvCBCFv81KQf7n1qBvzIIwmjJT22Qt2VL9j9DyHhY6RRj9PD15pUQW cmeXQHEpZv0sm4kxkQj4PMfs4QgA3coDMCL3yxzzCnGR5QwS98yN5/NFX+NuTKCD3oBl 5vDw== X-Gm-Message-State: AOPr4FXURhbczp8g/EoAqxwOgCLOpOUYgyyX7SJAKTe27xXJIXRdVElorv4Pk5YGJtVBAimmvIJGkFiZ5G5rIg== MIME-Version: 1.0 X-Received: by 10.28.157.143 with SMTP id g137mr17698929wme.29.1461864002779; Thu, 28 Apr 2016 10:20:02 -0700 (PDT) Received: by 10.28.183.131 with HTTP; Thu, 28 Apr 2016 10:20:02 -0700 (PDT) In-Reply-To: References: Date: Thu, 28 Apr 2016 14:20:02 -0300 Message-ID: Subject: Re: Best option to process packet ACL From: Ze Claudio Pastore To: Alan Somers Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 17:20:05 -0000 Because actually, this is ot a packet firewall. When I mentioned pf/ipfw is only to reffer to ideas on how to best match each acl criteria. But my userland application is a proxy, ACL will handle L7 requests within the packets. I will filter based on the mentioned criteria but it will be processed at a different moment unrelated to packet in kernel. It's also DPDK enabled so it's mostly skipping the whole kernel. 2016-04-28 11:50 GMT-03:00 Alan Somers : > On Wed, Apr 27, 2016 at 1:21 PM, Z=C3=A9 Claudio Pastore > wrote: > >> Hello everyone, >> >> I would like to hear your suggestion regarding the best approach to >> process >> IP packets for filtering, in such a way I can avoid lowering my pps rate= . >> >> Today a have a simple application proxies http application. It's dual >> threaded on a 4 core system with low CPU power. The current application >> uses two threads, one for control and one for data flow processing. >> >> I need to implement a simple set of stateless filtering, I will process >> only: >> >> - src-ip >> - dst-ip >> - src-port >> - dst-port >> - iplen >> - proto (tcp/udp/other) >> >> My current rate of requests per second is high, around 200K. I have no >> idea >> how I can leverage the IDLE CPUs the best way to implement this ACL >> filtering trying not to impact on the pps rate I have today. >> >> I have implemented it serial today (not threaded) and I get 40% >> performance >> loss. I will handle max 128 filter rules, this is a decision which is >> made. >> This is going to be first match wins. >> >> My current plans are to test: >> >> 1) Create 6 threads, one to test each aspect of the ACL (src-ip, dst-ip, >> etc) the first thread that returns false to parent thread I stop >> processing >> that rule and go to the next, and tell all other threads to die/exit sin= ce >> they don't matter anymore. >> >> 2) Create one thread to process a batch of rules, say, 8 rules per threa= d >> per request. Don't know if I would limit total number of threads and loc= k >> requests while threads ar e busy. >> >> 3) Someone suggested "do as pf/ipfw do" but I have no idea how it's done= , >> how multithreaded it is and what is done on each thread. >> >> 4) Other suggestion? >> >> This is going to run FreeBSD 11, I use libevent2 on the current >> application >> so far. >> >> Thanks. >> _______________________________________________ >> >> > Is there some reason why you can't simply use pf or ipfw? ipfw can do > everything you described. pf can do most of it, but I'm not sure if pf c= an > filter on iplen. If I were you, I wouldn't attempt to write my own > userland firewall until I was absolutely sure that neither pf nor ipfw > would work. If that's the case, then I would try using diverter sockets. > With a diverter socket, pf or ipfw does most of the work, but when it > encounters a packet it can't process it pushes it up to a userland helper= . > The userland helper processes the packet and then tells pf or ipfw what t= o > do with it. In realistic applications, pf or ipfw also creates a tempora= ry > rule based on the userland helper's decision. Applying the temporary rul= e > in the future is far faster than invoking the userland helper. After a > certain amount of time, the temporary rule will expire again. > > > Here's an example in action: > http://daemonforums.org/showthread.php?t=3D8846 > > -Alan >