From owner-freebsd-pf@FreeBSD.ORG Sat Mar 9 13:37:57 2013 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 87A59BDC for ; Sat, 9 Mar 2013 13:37:57 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ee0-f46.google.com (mail-ee0-f46.google.com [74.125.83.46]) by mx1.freebsd.org (Postfix) with ESMTP id 10D801D7 for ; Sat, 9 Mar 2013 13:37:56 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id e49so1533689eek.33 for ; Sat, 09 Mar 2013 05:37:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=ZfG3s3NFULdgi+srDZR0jl6TBJ+5Egt1AF4+r9mpUUI=; b=lWYVvY+0Ave05yAkooAY1Nok75SddJQCKrs22IyKfPfcWgUS9CUX6OzRwL5CFihKrj ox+1WSUR0ClUoMSHYapy+ziqG5ghipNCUTpLpiKSV5hwyy9e0qw4Y0DZ8pFrhwe0vop4 4HJHL+rkG9KMd2/aKfgkaq7OOxkkiVvp3VTWRDYX5BVPi35R7bX7mpFksgTxzRchfUQK F1w4Th0qe1uZ6WDn2CAx9pkDCPgjn6NXyeo0i2jWIw4tKt8NgQGOHngiqewvpormHZk1 uMdjE0bsa7fMipzEiQipMOgtfPPz4foLblR1+w+IRj7FAuq8A0YNi06EUk6ACLr7/1eg Vy+Q== X-Received: by 10.14.183.198 with SMTP id q46mr16472183eem.1.1362836275730; Sat, 09 Mar 2013 05:37:55 -0800 (PST) Received: from zvezda.localnet ([37.81.64.97]) by mx.google.com with ESMTPS id 44sm13262429eek.5.2013.03.09.05.37.53 (version=TLSv1 cipher=RC4-SHA bits=128/128); Sat, 09 Mar 2013 05:37:54 -0800 (PST) From: Kajetan Staszkiewicz To: Ermal =?utf-8?q?Lu=C3=A7i?= Subject: Re: [patch] Source entries removing is awfully slow. Date: Sat, 9 Mar 2013 14:37:51 +0100 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <201303081419.17743.vegeta@tuxpowered.net> <201303082151.00895.vegeta@tuxpowered.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201303091437.51945.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQn3K1pnIJCORF7OizZyVAYJYUNeW/Jll7/VWOCTfuOqrLV7eHfpi4NidxqzskUHdvndK9nj Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 13:37:57 -0000 Dnia sobota, 9 marca 2013 o 13:14:16 Ermal Lu=C3=A7i napisa=C5=82(a): > On Fri, Mar 8, 2013 at 9:51 PM, Kajetan Staszkiewicz >=20 > wrote: > > Dnia pi=C4=85tek, 8 marca 2013 o 21:11:43 Ermal Lu=C3=A7i napisa=C5=82(= a): > > > Is this FreeBSD 9.x or HEAD? > >=20 > > I found the problem and developed the patch on 9.1. > >=20 > Can you please test this more 'beautiful' patch. Oh, somehow I did not notice an existing implementation for doubly linked l= ist.=20 I'm quite new to kernel programming. > Its similar to yours but also delays src state removal to the proper purge > thread. I'll try it right after the weekend. > Though the src node removal option through pfctl -K does a lot of job to > cleanup things > Still need to undertand why it takes so much time for you to loop through > 500K states. That is because the loop will not be called just once. `pfctl -K 0.0.0.0/0 -K ip.of.internal.server.behind.this.loadbalancer` will= =20 match multiple Source entries, up to a thousand of them in normal condition= s=20 ("normal" for my loadbalancers) and many many more when under a DDoS attack. > The purge thread does that every tick by partitioning it to a few per time > slot but still minutes is way loong. >=20 > Can you please try to give a top -SH view of the time when this happens a= nd > a pfctl -vvsa output? I'll try on Monday, although as far as I remember the machine was quite fro= zen=20 during this operation. =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------'