From owner-freebsd-pf@FreeBSD.ORG Wed May 18 20:13:32 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D624E106566B for ; Wed, 18 May 2011 20:13:32 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 80B098FC08 for ; Wed, 18 May 2011 20:13:32 +0000 (UTC) Received: by ywf7 with SMTP id 7so860248ywf.13 for ; Wed, 18 May 2011 13:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to:x-openpgp-key-id:x-openpgp-key-fingerprint :x-openpgp-key-url; bh=CIaOdOf+WG7Er6PlssInuU2kVVbpkzhW0TOgl4SOX6Q=; b=pWF06aYT1lFfe6W/YO2EGM0JVgXYYL7hqXmA9VyM2fmy9RzNjvQCEq47L5nQPK6Ag/ oJl5Lke0LxPIDYOXucm2VVibiDk+aNWKsiA2uS7cZoNW4aH1Le5nJFNN+etkMDIzB3xg pXZXZ4W8pgDUS+kpUoNfQtxYHAuLQ1RariZi8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:x-openpgp-key-id :x-openpgp-key-fingerprint:x-openpgp-key-url; b=xvPKXmgou0/vEbZeDHhC3qlCJCHcEAsxI9bsukBxseRZc7VkjPeUfUES5DuwYpm3kN 4Lw/lpA5FTSte9pMI5+x5ktDZwDs2X+d/ao9Jl67leUFT3znTEmym513S1BAPSQpZ9Ll RCtkmrsOfRk3M/428/G8fOFBG/WLkCIfucfIU= Received: by 10.150.9.29 with SMTP id 29mr1814707ybi.148.1305749611572; Wed, 18 May 2011 13:13:31 -0700 (PDT) Received: from DataIX.net (adsl-99-181-146-200.dsl.klmzmi.sbcglobal.net [99.181.146.200]) by mx.google.com with ESMTPS id f13sm878696ybi.18.2011.05.18.13.13.28 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 18 May 2011 13:13:29 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.4/8.14.4) with ESMTP id p4IKDPcp035922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 18 May 2011 16:13:26 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.4/8.14.4/Submit) id p4IKDOo5035921; Wed, 18 May 2011 16:13:25 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Wed, 18 May 2011 16:13:24 -0400 From: Jason Hellenthal To: "quentin.narvor" Message-ID: <20110518201324.GA35466@DataIX.net> References: <390946c3b25ae3d887574555a494cb42@ensi-bourges.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline In-Reply-To: X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E X-OpenPGP-Key-URL: http://bit.ly/0x89D8547E Cc: freebsd-pf@freebsd.org Subject: Re: Large table issue X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 20:13:32 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable quentin.narvor, On Wed, May 18, 2011 at 03:00:57PM +0200, quentin.narvor wrote: > On Wed, 18 May 2011 15:34:49 +0300, Richard Brend=F6rfer wrote: > > Hi, > > try with=A0_set limit table-entries number_ in pf.vonf=A0or split you > > table in 2 or 3 tables. > > > Hi, >=20 > I forgot to say that I have already set this option to 3000000 in my=20 > pf.conf. > I have tried to split the table in smaller pieces (~450000 entries in=20 > each table) but the command "pfctl -f /etc/pf.conf" gives me the same=20 > memory issue when loading the third table. > I don't know the precise number but it seems that there is a limit near= =20 > 1000000 entries for the sum of all tables, even with the limit=20 > table-entries set to 3000000. >=20 > > On Wed, May 18, 2011 at 2:03 PM, quentin.narvor wrote: > > > >> I am trying to detect problems on hosts in my network : I want to > >> detect when a communication occurs with a compromised host. > >> I have built a blacklist which holds near 2 millions ip (spam, > >> malware.... hosts). > >> > >> But I can't load it into pf, I get this when I try : > >> > >> =A0 =A0 /etc/pf.conf:6: cannot define table bl: Cannot allocate > >> memory > >> =A0 =A0 pfctl: Syntax error in config file: pf rules not loaded > >> > >> I suspect there is a memory limitation somewhere (in the kernel ??) > >> which prevent me from loading the table but I am not very > >> comfortable with kernel variables. > >> I have already try modifying kern.maxssiz and kern.dflsiz without > >> success. > >> > >> Any idea? If you are going to be dealing with tables this size it might be wise to write a filter to run your table file through and output the end result of multiple CIDR ranges that are going to take up a considerable less amount of space than what you have there. And if you hit a range where you dont want certain ip's blocked you can also use a !127.0.0.1/29 to cover a specfic range for example. Ive seen someone on the lists once post something about a script but don't remember off hand what that was so youll have to do some searching. Have fun! --=20 Regards, (jhell) Jason Hellenthal --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJN1ChkAAoJEJBXh4mJ2FR+yOcIAIovM0vfxolx4N+2jHByeOvq PtBXLHX/qK2QWwGQ3/ygVT4PN1zXbwISaNPq4zreMahckaWjrBn9cMozI46+Kvpm t0ig+Fn1zlRPd7xW4qO2qBNycQQ3ev0J5PS1bDnBzmxseM8FaY7wnKKOjLxdt61G xInK0HevMi7whwnzdV4XpG+gg6hLYhYN2Oo626Gp7VcESDL4qNn5JEoKdFu8NjeO gJiNFjNZxGBIGbVecZtLgkfUk0o0alpxts2P4QPhYHfG5w4Q/ahkwOTc3L5DCJpZ RYkUO2+zb2T68VEfDUn8vf1BzUzOEGLUuxkhcSJkMDO77jLIbCWFAsmQaN0ufos= =G3qO -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24--