From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 00:41:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 561E0319 for ; Sun, 21 Apr 2013 00:41:51 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 22208152A for ; Sun, 21 Apr 2013 00:41:50 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r3L0WjZX082422 for ; Sat, 20 Apr 2013 19:32:45 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sat Apr 20 19:32:45 2013 Message-ID: <517333A8.7020704@denninger.net> Date: Sat, 20 Apr 2013 19:32:40 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Odd NAT/IPSEC question -- help! :-) X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 00:41:51 -0000 Here's the situation. I have a FreeBSD-Stable 9.1 system that has been running through the various versions of FreeBSD for the last several years. It uses ipfw and NAT to protect and serve PC clients along with other devices inside, and has an outside connection as well. The topology looks like this: Clients[192.168.1.x/24) ---- [192.168.1.100-em0 -Server- em1-70.169.168.7] <---> Internet em1 has the following: em1: flags=8943 metric 0 mtu 1500 options=4219b ether 00:30:48:db:7b:a7 inet 70.169.168.7 netmask 0xffffff80 broadcast 70.169.168.127 inet6 fe80::230:48ff:fedb:7ba7%em1 prefixlen 64 scopeid 0x6 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active The client topology is a bit complex but from the server's perspective looks like a single LAN (everything is effectively bridged.) I've used LT2P/PTPP for a while to run VPN "road warrior" clients in. This works ok but is slow. Recently I obtained a BlackBerry Z-10, which only supports IKEv2 and other similar protocols. No big deal, I thought, so I recompiled the kernel with the appropriate IPSEC defines in it, downloaded StrongSwan and after much gnashing of teeth got a configuration that works. I can connect to the gateway and see anything on it, along with anything else on the client subnet (after a fair of screwing around that involved placing the VPN's offered "tunnel" addresses inside of the client subnet.) The problem is that ipfw NAT utterly refuses to translate this traffic outbound. What's even worse is that I can't find it anywhere with tcpdump! That is, if try to connect to an external web address and run a tcpdump -i em1 host whatever-I-went-to I see nothing being emitted for that address at all. My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", which works fine for ordinary "on the client" traffic; no problems with that. The IPSEC tunnel looks like this: [root@NewFS /usr/local/etc]# ipsec status Security Associations (1 up, 0 connecting): remote[1]: ESTABLISHED 25 minutes ago, 70.169.168.7[70.169.168.7]...208.54.35.133[karl@denninger.net] remote{1}: INSTALLED, TUNNEL, ESP in UDP SPIs: c717241a_i ad4563f9_o remote{1}: 192.168.1.0/24 === 192.168.1.71/32 And again, if I access something on the 192.168.1.x network, on or off the gateway host, or a service on the server endpoint (e.g. the IMAP mail server which is listening on 70.169.168.7), it works. It appears that once the packets come into the system via ipsec they wind up being omitted from everything _*other than*_ going either into a local listening socket or being forwarded out the local client interface. I can't find them otherwise -- it's as if they disappeared! I have logging turned on for all "deny" ipfw firewall lines and nothing is showing up in the log related to this. If I can't translate those packets then I can use the VPN to get INTO the network but I CANNOT use it to make the remote machine appears to be PART OF the network, and that sucks. Any ideas? -- -- Karl Denninger /The Market Ticker ®/ Cuda Systems LLC From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 00:59:16 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A11EB57A for ; Sun, 21 Apr 2013 00:59:16 +0000 (UTC) (envelope-from prvs=18233350e1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 3064615CA for ; Sun, 21 Apr 2013 00:59:15 +0000 (UTC) Received: from r2d2 ([46.65.172.4]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50003398509.msg for ; Sun, 21 Apr 2013 01:59:14 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 21 Apr 2013 01:59:14 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 46.65.172.4 X-Return-Path: prvs=18233350e1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-net@freebsd.org Message-ID: <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> From: "Steven Hartland" To: "Karl Denninger" , References: <517333A8.7020704@denninger.net> Subject: Re: Odd NAT/IPSEC question -- help! :-) Date: Sun, 21 Apr 2013 01:59:36 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 00:59:16 -0000 ----- Original Message ----- From: "Karl Denninger" ... > My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", > which works fine for ordinary "on the client" traffic; no problems with > that. ... Just a stab in the dark, as I vaguely remember something similar, do you also need to configure your nat for gre as well as ip? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 01:20:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8FEE37A4 for ; Sun, 21 Apr 2013 01:20:07 +0000 (UTC) (envelope-from laurie_jennings_1977@yahoo.com) Received: from nm36-vm3.bullet.mail.ne1.yahoo.com (nm36-vm3.bullet.mail.ne1.yahoo.com [98.138.229.115]) by mx1.freebsd.org (Postfix) with ESMTP id DDB871641 for ; Sun, 21 Apr 2013 01:20:06 +0000 (UTC) Received: from [98.138.90.51] by nm36.bullet.mail.ne1.yahoo.com with NNFMP; 21 Apr 2013 01:18:24 -0000 Received: from [98.138.88.239] by tm4.bullet.mail.ne1.yahoo.com with NNFMP; 21 Apr 2013 01:18:24 -0000 Received: from [127.0.0.1] by omp1039.mail.ne1.yahoo.com with NNFMP; 21 Apr 2013 01:18:24 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 549035.36514.bm@omp1039.mail.ne1.yahoo.com Received: (qmail 56725 invoked by uid 60001); 21 Apr 2013 01:18:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1366507104; bh=bb2KLYKRvaq26RTJZ6ZBWyGJBzuuD+f0mPHuYvR0dhg=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=NOiawse8xOisP6FtrOmhkCRAR11FQKd6m8XvuzOHpAymwHI/ZRisfTu2wfqZtoSBlFuRn2xhLiEg5NU1EygM8LPYy0BRjt7Rw+NUQnmI8kgjiZEEjr3ilQMt3BYjIRnWTTSWAcQkaYe1UU9W/8I8AOOBk8I0k+8VxcOhL6del1o= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=sQqZMSxnvsPKjbrK/l4JVxyu+5HJTDuV/4T780qFkLN3WgGAZv9InUxuXl74AIsTAfS9q0vo/WJX91Ww8JwB+vNsfGtE792Y7y74hvZ3uSY27lMdvAUySxDoYY3pI1zCXIwgbDHt8gEnzj6zuyebFSRp1XFSU/two9CS0wMlrYY=; X-YMail-OSG: XQntZX4VM1l6hh9vACLO.VVhZLbbSmsezpJIXVKB1rle64l tk4V.BPr.OsXZB8jn4Qctg2u6nuBsojte6ZVMrauMQONxuu0pUXZXxLusQYe i0B_lTO5HlL0z1eKIPIFMCFSNy1qOvxMQ28MZGmAXAp4EnlQhHb_Saj17TM1 q29uqjcuGDgxKJ2gxcfygez5icNl1DE11EA4euBmm6O5N9FEbVSZjiLkS1A4 dMSRrl1Ldv.t7ORd7vYoV5PLEbP2eOR0Wn0j7FVcUpMXDyRGs4S8PGV07bl3 TwUTvA5gnkeon9uExUsxwz2TbtjYhFCZRB9x.PmdjKshAlI_xNUm3brJcY8D 59uNptkd8GFh33p2AXfhzY7ngdbPXqrxIBuWW6TLWvHAMaoAGwPYrSrPFclz D_xHgfKcPn1Fa2eD.0wRkY8BcPAWHTtd3EMwhnaCRONS0mmYxomntrHA2Ioq A1B1ryUG_O.NEp5TPaZcrR9J4FJ.BWixhDI5y Received: from [98.203.118.124] by web125804.mail.ne1.yahoo.com via HTTP; Sat, 20 Apr 2013 18:18:24 PDT X-Rocket-MIMEInfo: 002.001, VGhhdCBkb2VzIGhlbHAuIElzIHRoZXJlIGEgd2F5IGZvciB0aGUga2VybmVsIHRvIGFjY2VzcyB0aGUgbWVtb3J5IG1hcCBkaXJlY3RseWJ5IHNlZ21lbnQgbmFtZT8NCkxhdXJpZQ0KDQotLS0gT24gVGh1LCA0LzE4LzEzLCBKb2huIEJhbGR3aW4gPGpoYkBmcmVlYnNkLm9yZz4gd3JvdGU6DQoNCkZyb206IEpvaG4gQmFsZHdpbiA8amhiQGZyZWVic2Qub3JnPg0KU3ViamVjdDogUmU6IHNobV9tYXAgcXVlc3Rpb25zDQpUbzogZnJlZWJzZC1uZXRAZnJlZWJzZC5vcmcNCkNjOiAiTGF1cmllIEplbm5pbmdzIiABMAEBAQE- X-Mailer: YahooMailClassic/15.1.7 YahooMailWebService/0.8.141.536 Message-ID: <1366507104.55455.YahooMailClassic@web125804.mail.ne1.yahoo.com> Date: Sat, 20 Apr 2013 18:18:24 -0700 (PDT) From: Laurie Jennings Subject: Re: shm_map questions To: freebsd-net@freebsd.org, John Baldwin In-Reply-To: <201304180950.18349.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 01:20:07 -0000 That does help. Is there a way for the kernel to access the memory map dire= ctlyby segment name? Laurie --- On Thu, 4/18/13, John Baldwin wrote: From: John Baldwin Subject: Re: shm_map questions To: freebsd-net@freebsd.org Cc: "Laurie Jennings" Date: Thursday, April 18, 2013, 6:50 AM On Thursday, April 11, 2013 10:58:14 am Laurie Jennings wrote: > Im working on a simple project that shares a memory segment between a use= r=20 processand a kernel module. I'm having some problems with shm_map and there= =20 doesn't seem to be much info on it. > Im not sure what happened to the memory when the user process that create= s=20 it terminates.=A0 I have some questions. > 1) Does the kernel mapping keep the segment from being garbage collected= =20 when the use process that creates it terminated? I've experienced shm_unmap= ()=20 panic when tryingto unmap a segment > scenario:=A0=20 > User process Maps SegmentKernel maps it=A0 with shm_map()User Process=20 TerminatesKernel tries to shm_unmap() and it panics. The kernel mapping bumps the refcount on the underlying vm object, so it wi= ll not go away.=A0 OTOH, you should be keeping your own reference count on the associated fd so that you can call shm_unmap().=A0 That is, the model shoul= d be something like: struct mydata *foo; foo->fp =3D fget(fd); shm_map(fp, &foo->p); /* Don't call fdrop */ and then when unmapping: struct mydata *foo; shm_unmap(foo->fp, foo->p); fdrop(foo->fp); > 2) Is there a way for the kernel process to know when the user process ha= s=20 goneaway? A ref count? You can install a process_exit EVENTHANDLER if you want to destroy this whe= n a process goes away.=A0 I have used shm_map/unmap for other objects that alre= ady had a reference count so I did my shm_unmap when that object was destroyed. > 3) Does a SHM_ANON segment persist as long as the kernel has it mapped, o= r=20 doesit get garbage collected when the creating user process terminates? It goes away when the backing 'struct file' goes away.=A0 If you follow the= =20 model above of keeping a reference count on the associated struct file then it won't go away until you fdrop() after the shm_unmap. > 4) When using a named segment, can the kernel "reuse" a mapping for a new= =20 userprocess? > Example: > User process creates shm segment with path /fooKernel Maps shm segment wi= th=20 shm_map()User process terminates.User process runs again, opening segment /= foo > Does the kernel need to re-map, or is the original mapping valid? The mapping is not per-process, so if you have mapped a shm for /foo and mapped it, it will stay mapped until you call shm_unmap.=A0 Multiple proces= ses can shm_open /foo and mmap it and they will all share the same memory. You could even share a SHM_ANON fd among multiple processes by passing it across a UNIX domain socket. Hope this helps. --=20 John Baldwin _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 03:41:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CAD06E36 for ; Sun, 21 Apr 2013 03:41:56 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 764B21DD for ; Sun, 21 Apr 2013 03:41:55 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r3L2aFX0084747 for ; Sat, 20 Apr 2013 21:36:16 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sat Apr 20 21:36:16 2013 Message-ID: <5173509A.905@denninger.net> Date: Sat, 20 Apr 2013 21:36:10 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Steven Hartland Subject: Re: Odd NAT/IPSEC question -- help! :-) References: <517333A8.7020704@denninger.net> <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> In-Reply-To: <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 03:41:56 -0000 I don't think so -- gre is not involved in the config. On 4/20/2013 7:59 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Karl Denninger" > ... >> My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", >> which works fine for ordinary "on the client" traffic; no problems with >> that. > ... > > Just a stab in the dark, as I vaguely remember something similar, do you > also need to configure your nat for gre as well as ip? > > Regards > Steve > > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. > and the person or entity to whom it is addressed. In the event of > misdirection, the recipient is prohibited from using, copying, > printing or otherwise disseminating it or any information contained in > it. > In the event of misdirection, illegible or incomplete transmission > please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > > > %SPAMBLOCK-SYS: Matched [+killing@multiplay.co.uk], message ok -- -- Karl Denninger /The Market Ticker ®/ Cuda Systems LLC From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 03:54:24 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B2ED8B7 for ; Sun, 21 Apr 2013 03:54:24 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) by mx1.freebsd.org (Postfix) with ESMTP id 91F642B3 for ; Sun, 21 Apr 2013 03:54:24 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id 3so2909719pdj.41 for ; Sat, 20 Apr 2013 20:54:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=hUluvbbXIP9SobIC7N8/8etTvXN6oIwfyw/+7P8UjP8=; b=Qw3y7iBfaiXCrlJxxz74MtR3ds+wdHX9Nt7s73Gy7IH8WP508kJZfwaEapw+F7yt9n xzClejYV0aOXemqgSA81H2n53FAl6YaZ414X9JYFEyZsw5RC0yr0pNwA53Zt+RwcTlr2 M3fqUimN7qDtoMA8D/z2fUvQkeTw/DUCqBpMtDzn6GMpJVu9NwMauJpZthde92wCaLqo qc0Tr19mFhh2fpUtuMS+Xn0y0xJDKGaB0SfDUea9BkWPJirn7KTyU/fo/uqseoLdKIrE iwueWcgiLKKspOHga+BfSqt51ZhrDvtHvOhHnoWgKrfnpQG38V+t+JahIcqJ2HgM+jBv M3bg== MIME-Version: 1.0 X-Received: by 10.68.223.167 with SMTP id qv7mr26480674pbc.68.1366516458541; Sat, 20 Apr 2013 20:54:18 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Sat, 20 Apr 2013 20:54:18 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Sat, 20 Apr 2013 20:54:18 -0700 (PDT) In-Reply-To: <5173509A.905@denninger.net> References: <517333A8.7020704@denninger.net> <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> <5173509A.905@denninger.net> Date: Sun, 21 Apr 2013 06:54:18 +0300 Message-ID: Subject: Re: Odd NAT/IPSEC question -- help! :-) From: Sami Halabi To: Karl Denninger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Steven Hartland X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 03:54:24 -0000 Be sure default gateway is properly cobfigured on the client not only the tunnel Sami On Apr 21, 2013 6:42 AM, "Karl Denninger" wrote: > I don't think so -- gre is not involved in the config. > > On 4/20/2013 7:59 PM, Steven Hartland wrote: > > ----- Original Message ----- From: "Karl Denninger" > > ... > >> My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", > >> which works fine for ordinary "on the client" traffic; no problems wit= h > >> that. > > ... > > > > Just a stab in the dark, as I vaguely remember something similar, do yo= u > > also need to configure your nat for gre as well as ip? > > > > Regards > > Steve > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > This e.mail is private and confidential between Multiplay (UK) Ltd. > > and the person or entity to whom it is addressed. In the event of > > misdirection, the recipient is prohibited from using, copying, > > printing or otherwise disseminating it or any information contained in > > it. > > In the event of misdirection, illegible or incomplete transmission > > please telephone +44 845 868 1337 > > or return the E.mail to postmaster@multiplay.co.uk. > > > > > > > > %SPAMBLOCK-SYS: Matched [+killing@multiplay.co.uk], message ok > > -- > -- Karl Denninger > /The Market Ticker =AE/ > Cuda Systems LLC > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 04:02:02 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 02FB61E1 for ; Sun, 21 Apr 2013 04:02:02 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id BE7C8338 for ; Sun, 21 Apr 2013 04:02:01 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r3L420sL086701 for ; Sat, 20 Apr 2013 23:02:00 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sat Apr 20 23:02:00 2013 Message-ID: <517364B3.9000403@denninger.net> Date: Sat, 20 Apr 2013 23:01:55 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Steven Hartland Subject: Re: Odd NAT/IPSEC question -- help! :-) References: <517333A8.7020704@denninger.net> <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> <5173509A.905@denninger.net> In-Reply-To: <5173509A.905@denninger.net> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 04:02:02 -0000 On 4/20/2013 9:36 PM, Karl Denninger wrote: > I don't think so -- gre is not involved in the config. > > On 4/20/2013 7:59 PM, Steven Hartland wrote: >> ----- Original Message ----- From: "Karl Denninger" >> ... >>> My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", >>> which works fine for ordinary "on the client" traffic; no problems with >>> that. >> ... >> >> Just a stab in the dark, as I vaguely remember something similar, do you >> also need to configure your nat for gre as well as ip? >> >> Regards >> Steve >> I traced this down a bit further -- it's more odd than I thought. The packets are coming from the OTHER END'S native IP number! That's why I couldn't find them. If I point the DNS server at the external gateway IP in the strongswan config file I immediately start getting complaints in the logs, but they're coming from the external IP Address -- NOT THE TUNNEL ADDRESS. It looks like contrary to my previous expectations the tunnel address is pretty-much irrelevant, so long as it's not an address in use elsewhere (which implies I should use something like 10.x.x.x for it.) If I'm reading the IPSEC documentation on how tunnel mode works correctly this is what's supposed to happen -- the tunnel is a pure encapsulation+encryption; it is stripped and the original packet (which was encrypted) then decoded, and voila -- the packet appears exactly as if it was presented "raw" from the other end. So it appears it's working, but this is VERY different than what I'm used to seeing from PPTP/LT2P. This does not explain why I thought I had full access to the internal LAN -- it is rather clear now that I do not. In other words: [root@NewFS /usr/local/etc]# ipsec status Security Associations (1 up, 0 connecting): remote[1]: ESTABLISHED 16 minutes ago, 70.169.168.7[70.169.168.7]..._*208.54.35.246*_[karl@denninger.net] remote{1}: INSTALLED, TUNNEL, ESP in UDP SPIs: c2ecb09f_i ae9d3747_o remote{1}: 192.168.1.0/24 === 192.168.1.71/32 The packets are coming from the underlined/bolded IP number once they're decoded and presented in the local system! What I'm getting in the log with the DNS IP defined as the external IP of the gateway machine is this: Apr 20 22:54:41 NewFS named[3775]: client 208.54.35.246#33784: query (cache) 'imap.gmail.com/A/IN' denied That won't work because the external address is configured to refuse to respond to anything other than known secondary DNS servers. But it explains where my packets are disappearing to -- they're being emitted from IPSEC on the gateway under the client's public IP number. I'm getting more confused rather than more enlightened here.... What I THOUGHT I should be seeing are packets, once decoded, coming from the tunnel IP. Nope. -- -- Karl Denninger /The Market Ticker ®/ Cuda Systems LLC From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 10:18:59 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5D56E648; Sun, 21 Apr 2013 10:18:59 +0000 (UTC) (envelope-from oleg@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 3619DE4D; Sun, 21 Apr 2013 10:18:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3LAIxvX096913; Sun, 21 Apr 2013 10:18:59 GMT (envelope-from oleg@freefall.freebsd.org) Received: (from oleg@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3LAIxgv096912; Sun, 21 Apr 2013 10:18:59 GMT (envelope-from oleg) Date: Sun, 21 Apr 2013 10:18:59 GMT Message-Id: <201304211018.r3LAIxgv096912@freefall.freebsd.org> To: oleg@FreeBSD.org, freebsd-net@FreeBSD.org, oleg@FreeBSD.org From: oleg@FreeBSD.org Subject: Re: kern/172985: [patch] [ip6] lltable leak when adding and removing IPv6 addresses X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 10:18:59 -0000 Synopsis: [patch] [ip6] lltable leak when adding and removing IPv6 addresses Responsible-Changed-From-To: freebsd-net->oleg Responsible-Changed-By: oleg Responsible-Changed-When: Sun Apr 21 10:17:44 UTC 2013 Responsible-Changed-Why: grab it. http://www.freebsd.org/cgi/query-pr.cgi?pr=172985 From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 19:30:44 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7E34BCFA; Sun, 21 Apr 2013 19:30:44 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 57A831023; Sun, 21 Apr 2013 19:30:44 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3LJUi1n003708; Sun, 21 Apr 2013 19:30:44 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3LJUiwa003707; Sun, 21 Apr 2013 19:30:44 GMT (envelope-from linimon) Date: Sun, 21 Apr 2013 19:30:44 GMT Message-Id: <201304211930.r3LJUiwa003707@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178017: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 19:30:44 -0000 Old Synopsis: Kernel panic: general protection fault in tcp_do_segment New Synopsis: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Responsible-Changed-From-To: freebsd-amd64->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 21 19:30:25 UTC 2013 Responsible-Changed-Why: reclassify http://www.freebsd.org/cgi/query-pr.cgi?pr=178017 From owner-freebsd-net@FreeBSD.ORG Sun Apr 21 23:49:14 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1E79AB16; Sun, 21 Apr 2013 23:49:14 +0000 (UTC) (envelope-from konrad.witaszczyk@uj.edu.pl) Received: from mail1.uj.edu.pl (mail1.uj.edu.pl [149.156.89.193]) by mx1.freebsd.org (Postfix) with ESMTP id DA32816DD; Sun, 21 Apr 2013 23:49:13 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from [10.0.0.33] ([90.184.203.55]) by mta.uoks.uj.edu.pl (Oracle Communications Messaging Server 7u4-27.01 (7.0.4.27.0) 64bit (built Aug 30 2012)) with ESMTPSA id <0MLM00MMKOIRO610@mta.uoks.uj.edu.pl>; Mon, 22 Apr 2013 01:13:40 +0200 (CEST) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.0 X-Antivirus-Code: 0x100000 Message-id: <517472A1.6020100@uj.edu.pl> Date: Mon, 22 Apr 2013 01:13:37 +0200 From: Konrad Witaszczyk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 To: net@freebsd.org, bms@FreeBSD.org Subject: Implementation of SCPS X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Apr 2013 23:49:14 -0000 Hi, I'm looking for information about an implementation of SCPS in FreeBSD (https://wiki.freebsd.org/IdeasPage#SCPS.2C_Space_Communication_Protocol_Standards). Could you tell me what's the current status of this project? Do you think that it's a good project for Google Summer of Code? Regards, Konrad From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 04:21:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 03A5380E for ; Mon, 22 Apr 2013 04:21:05 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id C393F1F2E for ; Mon, 22 Apr 2013 04:21:03 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r3M4Kf95010525 for ; Sun, 21 Apr 2013 23:20:53 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Sun Apr 21 23:20:53 2013 Message-ID: <5174BA94.7000907@denninger.net> Date: Sun, 21 Apr 2013 23:20:36 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Steven Hartland Subject: Re: Odd NAT/IPSEC question -- help! :-) References: <517333A8.7020704@denninger.net> <394C5F1CA58741F9965A5782A1117153@multiplay.co.uk> <5173509A.905@denninger.net> <517364B3.9000403@denninger.net> In-Reply-To: <517364B3.9000403@denninger.net> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 04:21:05 -0000 On 4/20/2013 11:01 PM, Karl Denninger wrote: > On 4/20/2013 9:36 PM, Karl Denninger wrote: >> I don't think so -- gre is not involved in the config. >> >> On 4/20/2013 7:59 PM, Steven Hartland wrote: >>> ----- Original Message ----- From: "Karl Denninger" >>> ... >>>> My "ordinary" NAT entry is simply "nat 1 ip from any to any via em1", >>>> which works fine for ordinary "on the client" traffic; no problems with >>>> that. >>> ... >>> >>> Just a stab in the dark, as I vaguely remember something similar, do you >>> also need to configure your nat for gre as well as ip? >>> >>> Regards >>> Steve >>> > I traced this down a bit further -- it's more odd than I thought. > > The packets are coming from the OTHER END'S native IP number! That's > why I couldn't find them. > > If I point the DNS server at the external gateway IP in the strongswan > config file I immediately start getting complaints in the logs, but > they're coming from the external IP Address -- NOT THE TUNNEL ADDRESS. > It looks like contrary to my previous expectations the tunnel address is > pretty-much irrelevant, so long as it's not an address in use elsewhere > (which implies I should use something like 10.x.x.x for it.) > > If I'm reading the IPSEC documentation on how tunnel mode works > correctly this is what's supposed to happen -- the tunnel is a pure > encapsulation+encryption; it is stripped and the original packet (which > was encrypted) then decoded, and voila -- the packet appears exactly as > if it was presented "raw" from the other end. So it appears it's > working, but this is VERY different than what I'm used to seeing from > PPTP/LT2P. This does not explain why I thought I had full access to > the internal LAN -- it is rather clear now that I do not. > > In other words: > [root@NewFS /usr/local/etc]# ipsec status > Security Associations (1 up, 0 connecting): > remote[1]: ESTABLISHED 16 minutes ago, > 70.169.168.7[70.169.168.7]..._*208.54.35.246*_[karl@denninger.net] > remote{1}: INSTALLED, TUNNEL, ESP in UDP SPIs: c2ecb09f_i ae9d3747_o > remote{1}: 192.168.1.0/24 === 192.168.1.71/32 > > The packets are coming from the underlined/bolded IP number once they're > decoded and presented in the local system! What I'm getting in the log > with the DNS IP defined as the external IP of the gateway machine is this: > > Apr 20 22:54:41 NewFS named[3775]: client 208.54.35.246#33784: query > (cache) 'imap.gmail.com/A/IN' denied > > That won't work because the external address is configured to refuse to > respond to anything other than known secondary DNS servers. But it > explains where my packets are disappearing to -- they're being emitted > from IPSEC on the gateway under the client's public IP number. > > I'm getting more confused rather than more enlightened here.... What I > THOUGHT I should be seeing are packets, once decoded, coming from the > tunnel IP. > > Nope. Further update on this -- I got it. StrongSwan is a bit confusing, but I got it figured out. I'll write it up in the next few days once I condense it all down. Everything is working in tunnel/transport mode and it's FAST. -- -- Karl Denninger /The Market Ticker ®/ Cuda Systems LLC From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 09:58:02 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 42DF35C8 for ; Mon, 22 Apr 2013 09:58:02 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id C31C51C39 for ; Mon, 22 Apr 2013 09:58:01 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3M9vxAD045504; Mon, 22 Apr 2013 13:57:59 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3M9vxsm045503; Mon, 22 Apr 2013 13:57:59 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 22 Apr 2013 13:57:59 +0400 From: Gleb Smirnoff To: Juan Mojica Subject: Re: ARP: Error Message in if_ether.c "arprequest: cannot find matching address" Message-ID: <20130422095759.GC76816@FreeBSD.org> References: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="45nhIriPRwzWsBkh" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 09:58:02 -0000 --45nhIriPRwzWsBkh Content-Type: text/plain; charset=koi8-r Content-Disposition: inline On Wed, Apr 17, 2013 at 08:46:42AM -0400, Juan Mojica wrote: J> We manage to hit the following message with some regularity. J> J> arprequest: cannot find matching address J> J> The code shows a printf: J> J> printf("%s: cannot find matching address\n", __func__); J> J> J> Any reason this is a printf and not a J> J> log(LOG_ERR, J> J> The only things I can come up with are: J> J> a) it is a really severe and should be printed out, which if that is the J> case why isn't there an assert there? J> b) whoops, that should probably be a log(LOG_ERR, J> On our end we need to figure out exactly why we're intermittently hitting J> this patch of code. Can you please try this patch? Let's see what's going on. printf()ing in kernel is especially unsafe when the event can be triggered remotely. The arprequest() is called on output path of a packet, however I'm not sure that it can't be triggered by incoming packet. -- Totus tuus, Glebius. --45nhIriPRwzWsBkh Content-Type: text/x-diff; charset=koi8-r Content-Disposition: attachment; filename="if_ether.c.diff" Index: if_ether.c =================================================================== --- if_ether.c (revision 249764) +++ if_ether.c (working copy) @@ -250,7 +250,9 @@ arprequest(struct ifnet *ifp, struct in_addr *sip, } IF_ADDR_RUNLOCK(ifp); if (sip == NULL) { - printf("%s: cannot find matching address\n", __func__); + log(LOG_ERR, "%s: cannot find matching address for " + "%s on %s\n", __func__, inet_ntoa(*tip), + if_name(ifp)); return; } } --45nhIriPRwzWsBkh-- From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 10:30:20 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 95A25BCC for ; Mon, 22 Apr 2013 10:30:20 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 4DBC71E05 for ; Mon, 22 Apr 2013 10:30:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3MAU1I3081078 for ; Mon, 22 Apr 2013 10:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3MAU1Qw081077; Mon, 22 Apr 2013 10:30:01 GMT (envelope-from gnats) Date: Mon, 22 Apr 2013 10:30:01 GMT Message-Id: <201304221030.r3MAU1Qw081077@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: Gleb Smirnoff Subject: Re: kern/178017: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Gleb Smirnoff List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 10:30:20 -0000 The following reply was made to PR kern/178017; it has been noted by GNATS. From: Gleb Smirnoff To: Nate Denning Cc: bug-followup@FreeBSD.org Subject: Re: kern/178017: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment Date: Mon, 22 Apr 2013 14:19:30 +0400 Nate, it looks like we are trying to free an mbuf that is already free. Can you please try FreeBSD 9.1-STABLE? The em(4) driver has underwent quite a lot of updates and bugfixes since 9.1-RELEASE. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 10:55:03 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7D4FFF24; Mon, 22 Apr 2013 10:55:03 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 46B021F3D; Mon, 22 Apr 2013 10:54:51 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3MAsn7a085910; Mon, 22 Apr 2013 10:54:49 GMT (envelope-from glebius@freefall.freebsd.org) Received: (from glebius@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3MAsmJj085909; Mon, 22 Apr 2013 10:54:48 GMT (envelope-from glebius) Date: Mon, 22 Apr 2013 10:54:48 GMT Message-Id: <201304221054.r3MAsmJj085909@freefall.freebsd.org> To: nate-freebsd@ooz.net, glebius@FreeBSD.org, freebsd-net@FreeBSD.org From: glebius@FreeBSD.org Subject: Re: kern/178017: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 10:55:03 -0000 Synopsis: [tcp] [panic] Kernel panic: general protection fault in tcp_do_segment State-Changed-From-To: open->closed State-Changed-By: glebius State-Changed-When: Mon Apr 22 10:54:18 UTC 2013 State-Changed-Why: Email to submitters address bounces. http://www.freebsd.org/cgi/query-pr.cgi?pr=178017 From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 11:06:48 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 562405F3 for ; Mon, 22 Apr 2013 11:06:48 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 479471082 for ; Mon, 22 Apr 2013 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3MB6mx9089180 for ; Mon, 22 Apr 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3MB6ljl089178 for freebsd-net@FreeBSD.org; Mon, 22 Apr 2013 11:06:47 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Apr 2013 11:06:47 GMT Message-Id: <201304221106.r3MB6ljl089178@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177417 net [ip6] Invalid protocol value in ipsec6_common_input_cb o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod o kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176667 net [libalias] [patch] libalias locks on uninitalized data o kern/176596 net [firewire] [ip6] Crash with IPv6 and Firewire o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176446 net [netinet] [patch] Concurrency in ixgbe driving out-of- o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176097 net [lagg] [patch] lagg/lacp broken when aggregated interf o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o bin/175974 net ppp(8): logic issue o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos o kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169664 net [bgp] Wrongful replacement of interface connected net o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/169438 net [ipsec] ipv4-in-ipv6 tunnel mode IPsec does not work p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167947 net [setfib] [patch] arpresolve checks only the default FI o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/167059 net [tcp] [panic] System does panic in in_pcbbind() and ha o kern/166940 net [ipfilter] [panic] Double fault in kern 8.2 o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166372 net [patch] [ipfilter] drops UDP packets with zero checksu o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165963 net [panic] [ipfilter] ipfilter/nat NULL pointer deference p kern/165903 net mbuf leak o kern/165643 net [net] [patch] Missing vnet restores in net/if_ethersub o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162926 net [ipfilter] Infinite loop in ipfilter with fragmented I o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel p kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipfilter] There is no ippool start script/ipfilter ma o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [ipfilter] [rc.d] [patch] No good way to set ipfilter o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipfilter] FreeBSD 6.1+VPN+ipnat+ipf: port mapping doe o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipfilter] [panic] Kernel Panic Trap No 12 Page Fault o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipfilter] [patch] ipfilter drops OOW packets under 6. o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipfilter] [patch] wrong behaviour with skip rule insi o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87521 net [ipfilter] [panic] using ipfilter "auth" keyword leads o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipfilter] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation s kern/77195 net [ipfilter] [patch] ipfilter ioctl SIOCGNATL does not m o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipfilter] ipfilter ipnat problem with h323 proxy supp o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipfilter] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipfilter] [ppp] Interactive use of user PPP and ipfil f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 462 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 16:07:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E57F0FF for ; Mon, 22 Apr 2013 16:07:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) by mx1.freebsd.org (Postfix) with ESMTP id BAD18134C for ; Mon, 22 Apr 2013 16:07:06 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 16B53B97C; Mon, 22 Apr 2013 12:07:06 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: shm_map questions Date: Mon, 22 Apr 2013 11:43:54 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <1366507104.55455.YahooMailClassic@web125804.mail.ne1.yahoo.com> In-Reply-To: <1366507104.55455.YahooMailClassic@web125804.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201304221143.54205.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 22 Apr 2013 12:07:06 -0400 (EDT) Cc: Laurie Jennings X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 16:07:07 -0000 On Saturday, April 20, 2013 9:18:24 pm Laurie Jennings wrote: > That does help. Is there a way for the kernel to access the memory map directlyby segment name? There is not, no. It wouldn't be hard to add, but the issue there is that the existing shm_map/unmap API assumes you have an open file descriptor and is tailored to having a userland process provide memory rather than having the kernel provide a SHM to userland, so even if you added a shm_open() that gave you a reference on the underlying shm object (struct shmfd *), you would need a slightly different shm_map/unmap that took that object directly rather than an fd. > Laurie > > --- On Thu, 4/18/13, John Baldwin wrote: > > From: John Baldwin > Subject: Re: shm_map questions > To: freebsd-net@freebsd.org > Cc: "Laurie Jennings" > Date: Thursday, April 18, 2013, 6:50 AM > > On Thursday, April 11, 2013 10:58:14 am Laurie Jennings wrote: > > Im working on a simple project that shares a memory segment between a user > processand a kernel module. I'm having some problems with shm_map and there > doesn't seem to be much info on it. > > Im not sure what happened to the memory when the user process that creates > it terminates. I have some questions. > > 1) Does the kernel mapping keep the segment from being garbage collected > when the use process that creates it terminated? I've experienced shm_unmap() > panic when tryingto unmap a segment > > scenario: > > User process Maps SegmentKernel maps it with shm_map()User Process > TerminatesKernel tries to shm_unmap() and it panics. > > The kernel mapping bumps the refcount on the underlying vm object, so it will > not go away. OTOH, you should be keeping your own reference count on the > associated fd so that you can call shm_unmap(). That is, the model should be > something like: > > struct mydata *foo; > > foo->fp = fget(fd); > shm_map(fp, &foo->p); > /* Don't call fdrop */ > > and then when unmapping: > > struct mydata *foo; > > shm_unmap(foo->fp, foo->p); > fdrop(foo->fp); > > > 2) Is there a way for the kernel process to know when the user process has > goneaway? A ref count? > > You can install a process_exit EVENTHANDLER if you want to destroy this when a > process goes away. I have used shm_map/unmap for other objects that already > had a reference count so I did my shm_unmap when that object was destroyed. > > > 3) Does a SHM_ANON segment persist as long as the kernel has it mapped, or > doesit get garbage collected when the creating user process terminates? > > It goes away when the backing 'struct file' goes away. If you follow the > model above of keeping a reference count on the associated struct file then > it won't go away until you fdrop() after the shm_unmap. > > > 4) When using a named segment, can the kernel "reuse" a mapping for a new > userprocess? > > Example: > > User process creates shm segment with path /fooKernel Maps shm segment with > shm_map()User process terminates.User process runs again, opening segment /foo > > Does the kernel need to re-map, or is the original mapping valid? > > The mapping is not per-process, so if you have mapped a shm for /foo and > mapped it, it will stay mapped until you call shm_unmap. Multiple processes > can shm_open /foo and mmap it and they will all share the same memory. > > You could even share a SHM_ANON fd among multiple processes by passing it > across a UNIX domain socket. > > Hope this helps. > > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 18:48:49 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 50405507; Mon, 22 Apr 2013 18:48:49 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) by mx1.freebsd.org (Postfix) with ESMTP id B8C751DEB; Mon, 22 Apr 2013 18:48:48 +0000 (UTC) Received: by mail-wi0-f176.google.com with SMTP id hj19so4987612wib.3 for ; Mon, 22 Apr 2013 11:48:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=Tt+pwKm68o1oasOqlSFkSVJzE1mitstJjg9rhp0HEus=; b=zq1ppOk74wZ7RDUC+OKz+3nEa28kD9YPMA2mG7IrjqhqykwAU+9rmWXqS9PVeneYLB mFte0S7xlLfTMVfrLYpurm0p61TBepaGD4KxyF+rAgNu8dh/Vz4Yzey3fOYJqBiM+Hgu xfnwxtOCJRYxJzjnLOI3rPIioDJrL5ynVhz+MK0hxZrLhIR1QWzQJP4UO6/6CyzqI54A 4JsxkA4jJ2V/xuFWLBV41Uc+QMuY2o5f2REgg5q8bV/68YG19I33nw5ohnc/Exue0HcC jvVh7zFv1AI/LmHm1oPvnpyi44c8+fnoQKedA45gZfkTI0r3fni+DZypLsZoO4gw3xb9 h8kg== MIME-Version: 1.0 X-Received: by 10.194.81.71 with SMTP id y7mr55150760wjx.19.1366656527810; Mon, 22 Apr 2013 11:48:47 -0700 (PDT) Received: by 10.194.122.73 with HTTP; Mon, 22 Apr 2013 11:48:47 -0700 (PDT) In-Reply-To: <20130422095759.GC76816@FreeBSD.org> References: <20130422095759.GC76816@FreeBSD.org> Date: Mon, 22 Apr 2013 14:48:47 -0400 Message-ID: Subject: Re: ARP: Error Message in if_ether.c "arprequest: cannot find matching address" From: Juan Mojica To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 18:48:49 -0000 Thanks Gleb. I do not believe this path can triggered by an incoming frame. I am planning on adding a KASSERT here to get a core when we exercise this code in our debug images. -Juan On Mon, Apr 22, 2013 at 5:57 AM, Gleb Smirnoff wrote: > On Wed, Apr 17, 2013 at 08:46:42AM -0400, Juan Mojica wrote: > J> We manage to hit the following message with some regularity. > J> > J> arprequest: cannot find matching address > J> > J> The code shows a printf: > J> > J> printf("%s: cannot find matching address\n", __func__); > J> > J> > J> Any reason this is a printf and not a > J> > J> log(LOG_ERR, > J> > J> The only things I can come up with are: > J> > J> a) it is a really severe and should be printed out, which if that is the > J> case why isn't there an assert there? > J> b) whoops, that should probably be a log(LOG_ERR, > J> On our end we need to figure out exactly why we're intermittently > hitting > J> this patch of code. > > Can you please try this patch? Let's see what's going on. > > printf()ing in kernel is especially unsafe when the event can be triggered > remotely. The arprequest() is called on output path of a packet, however > I'm not sure that it can't be triggered by incoming packet. > > -- > Totus tuus, Glebius. > -- Juan Mojica Email: jmojica@gmail.com From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 19:16:18 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 630A3466; Mon, 22 Apr 2013 19:16:18 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) by mx1.freebsd.org (Postfix) with ESMTP id C91631F66; Mon, 22 Apr 2013 19:16:17 +0000 (UTC) Received: by mail-we0-f175.google.com with SMTP id t11so6909185wey.20 for ; Mon, 22 Apr 2013 12:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=YYYWirGMnoukTQWDPOM3wJbARNPff8Xey8eKft7SWVw=; b=DmBklR4QC4CC2nFkLmPGieQkduc4BWaHaaTseQjnxj0Z0m/gH3O9yD7InYphIAZwGP ELwzbEISTX2BmfrJ8DwZgRnP91/U7P9OhWCxvAt4tMv52ONkK3ZDRRBvkv6Pe7qi/9jy AxGuvxfJPrqRCK9/WWVCO0Bc05T9th7fM1QDiS9SNOs/icl6mRLic78y8ZVIUA2gy/mK 7Oo/Bt9DdEuDC+vsgOS82h3uP6tWO2zsXaJn24zUF0bY3pGK6zSHz0kVMMHtxbymxu2i ivK0H6tAZprwV47s7iEefwW/XivhKR0yCvN//lT1j0XMVtgfUzxmiDg3AvvddYqgreay Xjew== MIME-Version: 1.0 X-Received: by 10.194.92.197 with SMTP id co5mr25429918wjb.41.1366658176464; Mon, 22 Apr 2013 12:16:16 -0700 (PDT) Received: by 10.194.122.73 with HTTP; Mon, 22 Apr 2013 12:16:16 -0700 (PDT) In-Reply-To: References: <20130422095759.GC76816@FreeBSD.org> Date: Mon, 22 Apr 2013 15:16:16 -0400 Message-ID: Subject: Re: ARP: Error Message in if_ether.c "arprequest: cannot find matching address" From: Juan Mojica To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 19:16:18 -0000 I'll provide my revision shortly. On Mon, Apr 22, 2013 at 2:48 PM, Juan Mojica wrote: > Thanks Gleb. I do not believe this path can triggered by an incoming > frame. I am planning on adding a KASSERT here to get a core when we > exercise this code in our debug images. > > -Juan > > > On Mon, Apr 22, 2013 at 5:57 AM, Gleb Smirnoff wrote: > >> On Wed, Apr 17, 2013 at 08:46:42AM -0400, Juan Mojica wrote: >> J> We manage to hit the following message with some regularity. >> J> >> J> arprequest: cannot find matching address >> J> >> J> The code shows a printf: >> J> >> J> printf("%s: cannot find matching address\n", __func__); >> J> >> J> >> J> Any reason this is a printf and not a >> J> >> J> log(LOG_ERR, >> J> >> J> The only things I can come up with are: >> J> >> J> a) it is a really severe and should be printed out, which if that is >> the >> J> case why isn't there an assert there? >> J> b) whoops, that should probably be a log(LOG_ERR, >> J> On our end we need to figure out exactly why we're intermittently >> hitting >> J> this patch of code. >> >> Can you please try this patch? Let's see what's going on. >> >> printf()ing in kernel is especially unsafe when the event can be triggered >> remotely. The arprequest() is called on output path of a packet, however >> I'm not sure that it can't be triggered by incoming packet. >> >> -- >> Totus tuus, Glebius. >> > > > > -- > Juan Mojica > Email: jmojica@gmail.com > -- Juan Mojica Email: jmojica@gmail.com From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 19:39:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 20DCB193 for ; Mon, 22 Apr 2013 19:39:13 +0000 (UTC) (envelope-from ddesimone@verio.net) Received: from relay1-bcrtfl2.verio.net (relay1-bcrtfl2.verio.net [131.103.218.142]) by mx1.freebsd.org (Postfix) with ESMTP id D609F108B for ; Mon, 22 Apr 2013 19:39:12 +0000 (UTC) Received: from iad-wprd-xchw01.corp.verio.net (iad-wprd-xchw01.corp.verio.net [198.87.7.164]) by relay1-bcrtfl2.verio.net (Postfix) with ESMTP id 81428B0384A5; Mon, 22 Apr 2013 15:21:30 -0400 (EDT) Thread-Index: Ac4/jpfPy16LNZnXTryrFSlXHnUfag== Received: from hometx-733b1p1.corp.verio.net ([10.144.2.53]) by iad-wprd-xchw01.corp.verio.net over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Mon, 22 Apr 2013 15:21:29 -0400 Received: by hometx-733b1p1.corp.verio.net (sSMTP sendmail emulation); Mon, 22 Apr 2013 14:21:28 -0500 Date: Mon, 22 Apr 2013 14:21:28 -0500 Content-Transfer-Encoding: 7bit From: "David DeSimone" To: "Juan Mojica" Content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4913 Importance: normal Priority: normal Subject: Re: ARP: Error Message in if_ether.c "arprequest: cannot find matching address" Message-ID: <20130422192127.GJ1620@verio.net> Mail-Followup-To: Juan Mojica , FreeBSD Net References: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Disposition: inline In-Reply-To: Precedence: bulk User-Agent: Mutt/1.5.20 (2009-12-10) X-OriginalArrivalTime: 22 Apr 2013 19:21:29.0459 (UTC) FILETIME=[972DCC30:01CE3F8E] Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 19:39:13 -0000 Juan Mojica wrote: > > We manage to hit the following message with some regularity. > > arprequest: cannot find matching address > > The code shows a printf: > > printf("%s: cannot find matching address\n", __func__); > > > Any reason this is a printf and not a > > log(LOG_ERR, > > The only things I can come up with are: > > a) it is a really severe and should be printed out, which if that is the > case why isn't there an assert there? > b) whoops, that should probably be a log(LOG_ERR, > > On our end we need to figure out exactly why we're intermittently hitting > this patch of code. We see this regularly on our network, because we have multiple overlaid subnets on the same VLAN. The server sees an ARP request come in, but it does not match any of its confgired subnets, so it generates this error when trying to formulate an ARP reply. For example, the server might be configured to use 192.168.1.10/24 on an interface, but if it receives an ARP request for 172.31.250.249 on that interface, it won't know how to reply, and will generate this error. You might see this error if someone is inserting hosts with wrong IP's on your network, and they start trying to ARP for one another. -- David DeSimone == Network Admin == fox@verio.net "I don't like spinach, and I'm glad I don't, because if I liked it I'd eat it, and I just hate it." -- Clarence Darrow This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you. From owner-freebsd-net@FreeBSD.ORG Mon Apr 22 20:01:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 15B8681F for ; Mon, 22 Apr 2013 20:01:12 +0000 (UTC) (envelope-from jmojica@gmail.com) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) by mx1.freebsd.org (Postfix) with ESMTP id A2BAA11AC for ; Mon, 22 Apr 2013 20:01:11 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id s43so1109172wey.13 for ; Mon, 22 Apr 2013 13:01:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=vG2bAieQTcmpHWoYHoJWMC0pD7a6xnsQ9g8UZNOAJPc=; b=g/m1hlj1RJ5OGYUrQy3B+8EHiQTRfz61zesvVkqAsR6rBUMkCRL+h506pJEpIAI5fN V6FZ7QnYGrGDGoQrLdTAilqyOsncyg4iY4K24LjI6TWeRDjRS/f9RA+85aVAmcbJrzR0 UvDqy7eMS7bVChSb74/z1ti7NB4zzrPCgbNRuAcgZBrG9TOTEdaVMtW57LEreXkf3Iiz 4KJasJ93sduCmOAjw5nyNx6Feu7g2iseW8YYmeAdchGTPpmLK+v451zHg7juQPf4y+Bj B933JPnWWdWnH4eeb+oEiSSOQciRh5jAUhVi0knpBh44BufrJoHFfJzOCt8s1tXquLz8 BOOg== MIME-Version: 1.0 X-Received: by 10.180.24.69 with SMTP id s5mr22448544wif.34.1366660870827; Mon, 22 Apr 2013 13:01:10 -0700 (PDT) Received: by 10.194.122.73 with HTTP; Mon, 22 Apr 2013 13:01:10 -0700 (PDT) In-Reply-To: <20130422192127.GJ1620@verio.net> References: <20130422192127.GJ1620@verio.net> Date: Mon, 22 Apr 2013 16:01:10 -0400 Message-ID: Subject: Re: ARP: Error Message in if_ether.c "arprequest: cannot find matching address" From: Juan Mojica To: Juan Mojica , FreeBSD Net Content-Type: multipart/mixed; boundary=f46d043be18cbbf6b604daf887b3 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 20:01:12 -0000 --f46d043be18cbbf6b604daf887b3 Content-Type: text/plain; charset=ISO-8859-1 Hey David, That's not quite the scenario we're dealing with, but it is disconcerting to here that it looks like this path can be triggered from an incoming frame. We had been seen it frequently when quickly unconfiguring/configuring interfaces from a host. An application would receive the configure message, start trying to send out traffic, but not get the unconfigure in time. That was the working theory. We've not hit it recently - could just be a dormant race condition. I have attach my proposed solution, which takes what Gleb provided, adds a KASSERT, and wraps it all up with a sysctl so that it can be turned off. Thanks, Juan On Mon, Apr 22, 2013 at 3:21 PM, David DeSimone wrote: > Juan Mojica wrote: > > > > We manage to hit the following message with some regularity. > > > > arprequest: cannot find matching address > > > > The code shows a printf: > > > > printf("%s: cannot find matching address\n", __func__); > > > > > > Any reason this is a printf and not a > > > > log(LOG_ERR, > > > > The only things I can come up with are: > > > > a) it is a really severe and should be printed out, which if that is the > > case why isn't there an assert there? > > b) whoops, that should probably be a log(LOG_ERR, > > > > On our end we need to figure out exactly why we're intermittently hitting > > this patch of code. > > > We see this regularly on our network, because we have multiple overlaid > subnets on the same VLAN. The server sees an ARP request come in, but > it does not match any of its confgired subnets, so it generates this > error when trying to formulate an ARP reply. > > For example, the server might be configured to use 192.168.1.10/24 on an > interface, but if it receives an ARP request for 172.31.250.249 on that > interface, it won't know how to reply, and will generate this error. > > > You might see this error if someone is inserting hosts with wrong IP's > on your network, and they start trying to ARP for one another. > > -- > David DeSimone == Network Admin == fox@verio.net > "I don't like spinach, and I'm glad I don't, because if I > liked it I'd eat it, and I just hate it." -- Clarence Darrow > > > This email message is intended for the use of the person to whom it has > been sent, and may contain information that is confidential or legally > protected. If you are not the intended recipient or have received this > message in error, you are not authorized to copy, distribute, or otherwise > use this message or its attachments. Please notify the sender immediately > by return e-mail and permanently delete this message and any attachments. > Verio Inc. makes no warranty that this email is error or virus free. Thank > you. > -- Juan Mojica Email: jmojica@gmail.com --f46d043be18cbbf6b604daf887b3 Content-Type: application/octet-stream; name="arp_no_int.diff" Content-Disposition: attachment; filename="arp_no_int.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hfu2fp4o0 SW5kZXg6IHN5cy9uZXRpbmV0L2lmX2V0aGVyLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL25ldGluZXQv aWZfZXRoZXIuYwkocmV2aXNpb24gMjQ5Nzc2KQorKysgc3lzL25ldGluZXQvaWZfZXRoZXIuYwko d29ya2luZyBjb3B5KQpAQCAtNTAsNiArNTAsNyBAQAogI2luY2x1ZGUgPHN5cy9wcm9jLmg+CiAj aW5jbHVkZSA8c3lzL3NvY2tldC5oPgogI2luY2x1ZGUgPHN5cy9zeXNsb2cuaD4KKyNpbmNsdWRl IDxzeXMvc3lzdG0uaD4KIAogI2luY2x1ZGUgPG5ldC9pZi5oPgogI2luY2x1ZGUgPG5ldC9pZl9k bC5oPgpAQCAtMTIyLDYgKzEyMywxMSBAQAogCSZWTkVUX05BTUUoYXJwX21heGhvbGQpLCAwLAog CSJOdW1iZXIgb2YgcGFja2V0cyB0byBob2xkIHBlciBBUlAgZW50cnkiKTsKIAorc3RhdGljIGlu dCBsb2dfYXJwX25vX2lmYWNlID0gMTsKK1NZU0NUTF9JTlQoX25ldF9saW5rX2V0aGVyX2luZXQs IE9JRF9BVVRPLCBsb2dfYXJwX25vX2lmYWNlLCBDVExGTEFHX1JXLAorCQkgICAmbG9nX2FycF9u b19pZmFjZSwgMCwKKwkJICAgImxvZyBhcnAgcmVxdWVzdHMgd2l0aCBubyBjb3JyZXNwb25kaW5n IGludGVyZmFjZSIpOworCiBzdGF0aWMgdm9pZAlhcnBfaW5pdCh2b2lkKTsKIHN0YXRpYyB2b2lk CWFycGludHIoc3RydWN0IG1idWYgKik7CiBzdGF0aWMgdm9pZAlhcnB0aW1lcih2b2lkICopOwpA QCAtMjUwLDcgKzI1NiwxMyBAQAogCQl9CiAJCUlGX0FERFJfUlVOTE9DSyhpZnApOwogCQlpZiAo c2lwID09IE5VTEwpIHsKLQkJCXByaW50ZigiJXM6IGNhbm5vdCBmaW5kIG1hdGNoaW5nIGFkZHJl c3NcbiIsIF9fZnVuY19fKTsKKwkJCWlmIChsb2dfYXJwX25vX2lmYWNlKSB7CisJCQkJS0FTU0VS VChmYWxzZSwgKCIlczogY2Fubm90IGZpbmQgbWF0Y2hpbmcgYWRkcmVzcyBmb3IgIgorCQkJCQkJ CQkiJXMgb24gJXNcbiIsIF9fZnVuY19fLCBpbmV0X250b2EoKnRpcCksCisJCQkJCQkJCWlmX25h bWUoaWZwKSkpOworCQkJCWxvZyhMT0dfRVJSLCAiJXM6IGNhbm5vdCBmaW5kIG1hdGNoaW5nIGFk ZHJlc3MgZm9yICIKKwkJCQkJIiVzIG9uICVzXG4iLCBfX2Z1bmNfXywgaW5ldF9udG9hKCp0aXAp LCBpZl9uYW1lKGlmcCkpOworCQkJfQogCQkJcmV0dXJuOwogCQl9CiAJfQo= --f46d043be18cbbf6b604daf887b3-- From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 12:08:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EBFAB406; Tue, 23 Apr 2013 12:08:09 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (lakerest.net [70.155.160.98]) by mx1.freebsd.org (Postfix) with ESMTP id F1ADB1E13; Tue, 23 Apr 2013 12:08:08 +0000 (UTC) Received: from [10.1.1.213] (bsd4.lakerest.net [70.155.160.102]) (authenticated bits=0) by lakerest.net (8.14.4/8.14.3) with ESMTP id r3NC87db094072 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 23 Apr 2013 08:08:07 -0400 (EDT) (envelope-from rrs@lakerest.net) Subject: Re: Default route changes unexpectedly Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Randall Stewart In-Reply-To: Date: Tue, 23 Apr 2013 08:08:07 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> References: <5136FD71.6000408@freebsd.org> To: Nick Rogers X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" , Andre Oppermann X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 12:08:10 -0000 Ok I too have been struck by this *multiple* times on my base home router. I am not sure how its happening, but I have placed in my kernel a = special catch that if the default route is set via the normal route.c path and = it is *not* the default route to my ISP, I will crash the kernel. My thoughts are this is *not* going to happen and its probably some = other memory corruption, but I will start with the most obvious first ;-) Now, I have also put in a cron that checks every 10 min the default = route, and=20 if it finds it *not* correct, it will a) log it, and b) fix it. So if I get a log showing up, I will know its *not* some errant program = in 9.x=20 causing a routing change via a routing socket, and if not I should have = a crash revealing who the guilty party is ;-) I will keep you all informed as I try to gather more information ;-D R On Mar 7, 2013, at 12:07 PM, Nick Rogers wrote: > On Wed, Mar 6, 2013 at 12:25 AM, Andre Oppermann = wrote: >> On 05.03.2013 18:39, Nick Rogers wrote: >>>=20 >>> Hello, >>>=20 >>> I am attempting to create awareness of a serious issue affecting = users >>> of FreeBSD 9.x and PF. There appears to be a bug that allows the >>> kernel's routing table to be corrupted by traffic routing through = the >>> system. Under heavy traffic load, the default route can seemingly >>> randomly change to an IP address that is not directly connected to = the >>> network (i.e., is not configured anywhere). Dhclient is not in the >>> mix, nor is routed, bgpd, etc. Running `route monitor` shows no >>> evidence of the change in the default route. The one commonality >>> between all the systems experiencing this problem seems to be the = use >>> of PF. >>>=20 >>> Obviously this is a serious problem as it causes all Internet-bound >>> traffic to stop routing until the default route is corrected. Some >>> users, including myself, are working around this problem by = installing >>> a script that runs multiple times a second to check if the default >>> route is incorrect and fixing it if necessary, which mitigates the >>> amount of downtime caused by the bug. >>=20 >>=20 >> Can you describe your traffic forwarding setup in more detail? >> Is it only pf, or do you run netgraph, or other things as well? >> Do you use flow routing? >=20 > I use PF for NAT, filtering, and rdr rules. ALTQ for bandwidth > management. I do not use netgraph. I use vlans. PF redirects to squid > as a transproxy. I'm not familiar with flow routing so unless its > enabled in 9.1 by default I do not use it. >=20 >>=20 >> How frequent does this happen? > Every other day during periods of heavier Internet-bound traffic. >=20 >>=20 >> I'm trying to create a stack graph to see which parts of the network >> stack are involved in handling your packet. >>=20 >> -- >> Andre >>=20 >>> Please refer to these past posts for more examples and evidence of >>> other users experiencing this problem: >>>=20 >>> http://forums.freebsd.org/showthread.php?p=3D211610#post211610 >>>=20 >>>=20 >>> = http://freebsd.1045724.n5.nabble.com/Default-route-quot-random-quot-gatewa= y-modification-bug-td5750820.html >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-net/2012-March/031879.html >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-ipfw/2010-September/004361.html= >>>=20 >>> There is also a PR that was incorrectly labeled as an IPFW issue. >>> Myself and others believe this issue is not restricted to the use of >>> IPFW and that the PR should be relabeled. I am inclined to think it = is >>> strictly a PF issue since I am not using IPFW, however there is >>> evidence of the default route changing on people using IPFW for past >>> versions of FreeBSD (7.x/8.x), so perhaps this is related. >>>=20 >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/174749 >>>=20 >>> Another PR for the same problem but specific to IPFW and 8.2-RELEASE >>>=20 >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D157796 >>>=20 >>> I am hoping someone reading this can give the problem the attention = it >>> deserves. Thank you. >>>=20 >>> -Nick >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 ------------------------------ Randall Stewart 803-317-4952 (cell) From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 13:40:10 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 804AC5D3 for ; Tue, 23 Apr 2013 13:40:10 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) by mx1.freebsd.org (Postfix) with ESMTP id 0C2101353 for ; Tue, 23 Apr 2013 13:40:09 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id fd20so549401lab.39 for ; Tue, 23 Apr 2013 06:40:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=qV/5Blg8I23J4SruNDNJhAyAcFYyTlyNdq0AknNt684=; b=cfs3MGhFx6TcVEhDfDK/Qkx68Lf1agbvcwyujhe0MosH4rdd1oy7C/pmCaT1ppnzpm ijmODB1MTIgS4e9q3IpXXajTibHkUOn+RIumnp710p8QVnLnGYNFPeydw51HQLmEGL1N CBXubg+2Q4HsiJtP2ahgd62MgZRR9KXCzTnTTgoB53RGF21zMq1/JySCTZT2pJU93jYd /0H7ICGsJCOBbm5+83/MzkQAJM86FV5qq4j/nILhy+QB2CvsUu1zOyoiGR1+cxUVgs9d mFWoQMtxkA2Syy+dl8jB/1MSA2ofMkxhWlQkyFNFdMhzFulHnDvxZdqRvTPLuB0ODWtx O0FA== MIME-Version: 1.0 X-Received: by 10.152.26.101 with SMTP id k5mr15740797lag.31.1366724408867; Tue, 23 Apr 2013 06:40:08 -0700 (PDT) Received: by 10.112.162.36 with HTTP; Tue, 23 Apr 2013 06:40:08 -0700 (PDT) In-Reply-To: <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> Date: Tue, 23 Apr 2013 14:40:08 +0100 Message-ID: Subject: Re: Default route changes unexpectedly From: Tom Evans To: Randall Stewart Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 13:40:10 -0000 On Tue, Apr 23, 2013 at 1:08 PM, Randall Stewart wrote: > Ok > > I too have been struck by this *multiple* times on my base home router. > I hate "me too" style posts, since often they conflate unrelated issues - however, "me too"! In my scenario, I have a simple home router with a wan if connected to an ADSL modem, an internal if connected to a pretty ordinary switch and the rest of the home network, using pf to NAT the connection (pretty basic stuff). Infrequently, I can no longer connect to or ping the router from internal connections, and have to grab a console, restart netif and routing, and everything then works again. However, I also have an openvpn connection to work running on the router. Work seem to believe that the reason there are 3 huge private network ranges is so that they can use the 10/8 block for DC infrastructure, the 172.16/12 block for offices and the 192.168/16 bit block for VPNs. Until now, I had been assuming - without any proof - that everything works great until openvpn gets told that 192.168.1/8 should be routed down the VPN, at which point everything local is inaccessible. Is there something useful I can look at when this next occurs that would explain why or how it is wedged, so that I can either rule myself in or out of this case? Cheers Tom From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 17:46:15 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D70AC7C3 for ; Tue, 23 Apr 2013 17:46:15 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from aussmtpmrkpc120.us.dell.com (aussmtpmrkpc120.us.dell.com [143.166.82.159]) by mx1.freebsd.org (Postfix) with ESMTP id AD8F1136C for ; Tue, 23 Apr 2013 17:46:15 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.87,536,1363150800"; d="scan'208";a="28535875" Message-ID: <5176C8A2.10106@vangyzen.net> Date: Tue, 23 Apr 2013 12:45:06 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Mark Martinec Subject: Re: ipv6 default router Operation not permitted References: <20130312225018.GA13589@defiant.konundrum.org> <3ABB5AED-DEA9-42F6-82A1-FEA9E8BBBDCF@my.gd> <20130313091727.GA17859@defiant.konundrum.org> <201303131227.57751.Mark.Martinec+freebsd@ijs.si> In-Reply-To: <201303131227.57751.Mark.Martinec+freebsd@ijs.si> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 17:46:15 -0000 On 03/13/2013 06:27, Mark Martinec wrote: > Having multiple IPv6 subnets on the same wire is asking for trouble. > > For example, I believe an ICMP redirect still (in 9.1) does not create > a temporary route: > http://www.freebsd.org/cgi/query-pr.cgi?pr=152791 > which beat us hard time (random unreachability between hosts), > having to rearrange that legacy segment which happened to have > two subnets on the same wire. It may be of little consolation now, but I posted patches for this. See the PR. Eric From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 19:40:11 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 43E10D48; Tue, 23 Apr 2013 19:40:11 +0000 (UTC) (envelope-from scrappy@hub.org) Received: from hub.org (hub.org [200.46.208.146]) by mx1.freebsd.org (Postfix) with ESMTP id 124C31AD0; Tue, 23 Apr 2013 19:40:10 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.188]) by hub.org (Postfix) with ESMTP id 460E3A12C3F; Tue, 23 Apr 2013 16:40:10 -0300 (ADT) Received: from hub.org ([200.46.208.146]) by maia.hub.org (mx1.hub.org [200.46.151.188]) (amavisd-maia, port 10024) with ESMTP id 26632-01; Tue, 23 Apr 2013 19:40:10 +0000 (UTC) Received: from [10.5.250.150] (remote.ilcs.sd63.bc.ca [142.31.148.2]) by hub.org (Postfix) with ESMTPA id 87CD8A12C3E; Tue, 23 Apr 2013 16:40:09 -0300 (ADT) From: "Marc G. Fournier" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: vbox + bce == sporactic ethernet hangs Date: Tue, 23 Apr 2013 12:40:07 -0700 Message-Id: <42DB312E-0DE5-4E89-BCC5-A1509A86DF12@hub.org> To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 19:40:11 -0000 I am running FreeBSD 9-STABLE (updated yesterday: FreeBSD 9.1-STABLE = #15: Mon Apr 22 07:45:07 UTC 2013) with VirtualBox 4.2.6 from ports =85 = the hardware is using a Broadcom ethernet: bce0: mem = 0xf4000000-0xf5ffffff irq 16 at device 0.0 on pci7 miibus0: on bce0 bce0: Ethernet address: 00:22:19:5b:20:bd bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C = (4.4.1); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (UMP 1.1.9) bce0: bce_pulse(): Warning: bootcode thinks driver is absent! (bc_state = =3D 0x00004006) Running with simple jail's on it, the server runs flawlessly until = reboot =85 but as soon as I start running Virtualbox on it, I get = sporadic server 'hangs' =85 never the same time, usually can be = triggered by heavier then normal load on the virtual box (ie. running an = rsync session from the base server into the vbox environment) =85=20 When it happens, I can *usually* connect via the DRAC / remote console = and login =85 but doing an 'ifconfig down' on the device and then back = up makes no difference =85 if I send a ctl-alt-del through the remote = console, more often then not, it will free up whatever is going on, so = that pinging works again, but, of course, I've already hit ctl-alt-del, = so its rebooting even though now I don't need it to =85 Based on a page on the wiki about tuning for vbox, I have set: net.graph.maxdata=3D65536 but I've seen this happen even with that set, so not sure if I'm just = still triggering it, or its something else I'm experiencing =85 So, two questions: 1. is there something I can run to see if I *am* in fact hitting that = limit? =20 2. is there something I can do, like ctl-alt-del, but without the = reboot, to 'free' the ethernet? Thx From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 19:49:25 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3D46B30D for ; Tue, 23 Apr 2013 19:49:25 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by mx1.freebsd.org (Postfix) with ESMTP id 178521B46 for ; Tue, 23 Apr 2013 19:49:24 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id bh4so693385pad.40 for ; Tue, 23 Apr 2013 12:49:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=iYgCl6op4tzK3vu7gY/AADyzIknFx1ewuJYsCsAGVQk=; b=UFndzruC31iMYDuUDOzgJn1s7cpAR93wda9ogWDYVN6RpjcJtn9JU2tPjuyeAnpPOf MJg9sZRfMBbJFmY+SdezpcaCjxCvCd68ZtqaszSFGYi6Hr/BJ2DJWkn0Z70p4ZECS9w4 aYOZTpH4WJeXlEZmVi3utn/6qj8tQByS4ue8I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding:x-gm-message-state; bh=iYgCl6op4tzK3vu7gY/AADyzIknFx1ewuJYsCsAGVQk=; b=Pum2JuV9rXIHj1uCm5B5xruU5iHBgKqkPmCujppA4KQecSoDzEhqgzZ27bA1G3tn4x vnOOtvTzNJLRWOeycEUZ05Wg4+s6eAJk1ujBLynWU8JlIoCbtDcFZUbfFuUXtWg7CcRp O9ecXqvq08pH4OFCazIRWqHEUVs/m30QHDykgwAzA9Bo40JXqpyhunGza1tJOuScueXe 1zsdnb/zr2VJpooYzTjEleffG1dfhyiMc7oNBlpOo4iL5ty8YO5mh321tQtXqXRyUu8H Z5W0W2soe5QW+HHz3wNpeZvmXKo+bIrwSs1Krqoo9Y64bmN9MrVg1wgXbMjq5ElCiBIm z7hw== X-Received: by 10.67.10.133 with SMTP id ea5mr15370131pad.135.1366746564482; Tue, 23 Apr 2013 12:49:24 -0700 (PDT) Received: from [172.30.0.34] (hgfw-01.soe.ucsc.edu. [128.114.61.130]) by mx.google.com with ESMTPS id vk7sm30651827pbc.41.2013.04.23.12.49.22 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 23 Apr 2013 12:49:23 -0700 (PDT) Message-ID: <5176E5C1.9090601@soe.ucsc.edu> Date: Tue, 23 Apr 2013 12:49:21 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (X11; Linux i686; rv:10.0.6esrpre) Gecko/20120717 Thunderbird/10.0.6 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: pf performance? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQn6Kn/OyvXnGBQF0MGDGqRDpVc2da7GOUoEOGK1mUINNH5UvalUdy+6/ZoU2BOdEmD3L60D X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 19:49:25 -0000 Hello all, I have a question here about how FreeBSD (8.1-RELEASE-p13 specifically) behaves when acting as a firewall. I understand the pf process is "giant locked" to a single CPU core when inspecting packets inbound and outbound. I was wondering, how does that manifest when I look at "top -P" on the firewall? Right now I have a dual port Myricom 10G NIC (packets inbound on one interface and outbound on the other), and the mxge driver is "multiplexing" interrupt processing across all the CPU cores for speed. So, when the firewall is busy, I see all the cpu cores quite busy processing interrupts (like 70% or more CPU utilization). But, all CPU work seems to be in interrupts. I don't see anything, or *very* little, in system or user space for CPU utilization. Should the pf process be using some CPU too? If so, how could I tell that? I'm trying to figure out if I'm limited by not having enough CPU to process the interrupts or not enough CPU to process the packet filtering process. Right now it looks like interrupts but I'm not sure. The Myricom folks looked at our debugging info on the mxge driver and say that based on what they see, mxge is dropping packets because the host cannot pull packets out of the NIC buffer fast enough. The host is using a four core Xeon X5677 3.46GHz CPU. We're processing 140,000 packets per second or so, and I see rates up to several gigabits per second, but all my research seems to indicate it can do better than that, and that we should not be dropping packets. Or maybe the question is: why doesn't the host pull the packets from the NIC fast enough? Is the CPU tied up doing something else? Interrupts? Does anyone have any ideas? TIA!! Thanks! -erich From owner-freebsd-net@FreeBSD.ORG Tue Apr 23 23:34:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 46F68203 for ; Tue, 23 Apr 2013 23:34:26 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x233.google.com (mail-bk0-x233.google.com [IPv6:2a00:1450:4008:c01::233]) by mx1.freebsd.org (Postfix) with ESMTP id CEFF514FC for ; Tue, 23 Apr 2013 23:34:25 +0000 (UTC) Received: by mail-bk0-f51.google.com with SMTP id y8so510972bkt.10 for ; Tue, 23 Apr 2013 16:34:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id :x-gm-message-state; bh=qAtpUmoFpeAjMbXw7m6oJ08w2DwPC6Ql0CkBMf+VVAw=; b=fLKjyPy1WLiAMCISK3CHJv2hs6U1Vqvb7FlevNc5mgm/vc2JUM619+0hpNTJCefpEO Dzauhvw3/AAbiiOUEk4/JciS/8v1tW7f2Dv5+EBCvX1MS9/atJkk9dXSwRrVLCFqB1P9 LlonXqhtM8fYE8L6kZ9QqerU0Fc/j9hRPvbBP7/tEW3Rf3gRGnXeFXVAGr3rScpLF0Re lVzg3YO3lamzjN/QKejljYClkSMG70zkKOtMV2Z3LJqMfB3uxOZvs4ViMiqk+b5HhKXZ J3h/NjJ0x51Z1C8eGH49NYJu1zoUp2pKCOf9KU7qgq2pj3YCiAG5cZUEXc1y4CrO9yJa G+Sw== X-Received: by 10.204.173.9 with SMTP id n9mr14033114bkz.47.1366760064484; Tue, 23 Apr 2013 16:34:24 -0700 (PDT) Received: from zvezda.localnet ([2a02:8108:1440:5b:2677:3ff:fe7b:7648]) by mx.google.com with ESMTPSA id cv9sm70532bkb.5.2013.04.23.16.34.23 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 23 Apr 2013 16:34:23 -0700 (PDT) From: Kajetan Staszkiewicz To: freebsd-net@freebsd.org Subject: Re: pf performance? Date: Wed, 24 Apr 2013 01:34:22 +0200 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <5176E5C1.9090601@soe.ucsc.edu> In-Reply-To: <5176E5C1.9090601@soe.ucsc.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201304240134.22740.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQl7suXB+VPb2P5aan2Yr7S3GgmZjw/kETCR5lW708FC/DMSXt4mk1q6+bnQtoebSjzgJc2m X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 23:34:26 -0000 Dnia wtorek, 23 kwietnia 2013 o 21:49:21 Erich Weiler napisa=C5=82(a): > Hello all, >=20 > I have a question here about how FreeBSD (8.1-RELEASE-p13 specifically) > behaves when acting as a firewall. I understand the pf process is > "giant locked" to a single CPU core when inspecting packets inbound and > outbound. I was wondering, how does that manifest when I look at "top > -P" on the firewall? >=20 > Right now I have a dual port Myricom 10G NIC (packets inbound on one > interface and outbound on the other), and the mxge driver is > "multiplexing" interrupt processing across all the CPU cores for speed. > So, when the firewall is busy, I see all the cpu cores quite busy > processing interrupts (like 70% or more CPU utilization). But, all CPU > work seems to be in interrupts. I don't see anything, or *very* little, > in system or user space for CPU utilization. Should the pf process be > using some CPU too? If so, how could I tell that? I'm trying to figure > out if I'm limited by not having enough CPU to process the interrupts or > not enough CPU to process the packet filtering process. Right now it > looks like interrupts but I'm not sure. As far as I understand, processing of packets by pf takes place in receivin= g=20 network card's interrupt handler even up to sending the packet via another= =20 network card (at least in my case, when using route-to targets, which make= =20 routing inside pf). > The Myricom folks looked at our debugging info on the mxge driver and > say that based on what they see, mxge is dropping packets because the > host cannot pull packets out of the NIC buffer fast enough. The host is > using a four core Xeon X5677 3.46GHz CPU. We're processing 140,000 > packets per second or so, and I see rates up to several gigabits per > second, but all my research seems to indicate it can do better than > that, and that we should not be dropping packets. Or maybe the question > is: why doesn't the host pull the packets from the NIC fast enough? Is > the CPU tied up doing something else? Interrupts? As for my performance issues, at first I noticed that I always had some cor= es=20 overloaded and some doing noting. So I performed the following tuning: =2D disabled HT on CPUs =2D deferred netisr and no NIC interrupts assigned to cores used by netisr =2D each core gets only one interrupt But this is in case of NICs with just a single interrupt (so I have netisr = at=20 cpu0 and 1, one NIC on cpu3, one nic on cpu4), it might not help when you h= ave=20 ones that can load all cores. Some more tips: =2D use interrupt coalescing, if you do, tune it to be more agressive =2D create states on *both* sides of your firewall, for me this lowered loa= davg 2-3 times on a machine with around 400 rules. =2D keep state amount low, I was surprised how many states were hanging in "closing" state which has quite a long default timeout. How do you count the 140kpps value? One interface, both, in, out? I'd like = to=20 relate this somehow to my values. =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 10:45:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F3588367; Wed, 24 Apr 2013 10:45:52 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-vb0-x22e.google.com (mail-vb0-x22e.google.com [IPv6:2607:f8b0:400c:c02::22e]) by mx1.freebsd.org (Postfix) with ESMTP id A67611D4B; Wed, 24 Apr 2013 10:45:52 +0000 (UTC) Received: by mail-vb0-f46.google.com with SMTP id 11so1497804vbe.5 for ; Wed, 24 Apr 2013 03:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:from:date:x-google-sender-auth :message-id:subject:to:content-type:content-transfer-encoding; bh=imUBqrViama4CuNF8NViu+0q5XFY5Bl34Cc5cryWUXc=; b=Dw9phbbIDXunbujwzFs0mHsRx+zFAcle65jgJfWC+TS+uhR05h2ffv0FHC6lo875pz g7LSW9XiBy+wHHtciFqV4uf7q2ltFkiDDDUCEBtL6pu0qt+MAAoyFOll73XgN+PXDVHh PnG5BXHihW1Yhsmg3TArIJqZkvvvnCqgD/zk1V/TGRb4lPzOMcxYznDZyF3GhDFisfBv IlPL0YhpnEP4wzdgiPaxTCEMLGn7FXrYkaMOEIWwqPeBeVnb54GRXzs9ffz1NvU9J3th eRr8hkolch7QjyT4Fsh44ltNpKwx+HIJPAGBD9JzxG1D9ZsUMTkj1C4CRhY6jdeO0HD0 Qiew== X-Received: by 10.59.11.199 with SMTP id ek7mr25083623ved.19.1366800352147; Wed, 24 Apr 2013 03:45:52 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Wed, 24 Apr 2013 03:45:32 -0700 (PDT) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Wed, 24 Apr 2013 12:45:32 +0200 X-Google-Sender-Auth: 9hABzZSVg5OXpWs7et0x4p8Ry8U Message-ID: Subject: forwarding/ipfw/pf evolution (in pps) on -current To: "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 10:45:53 -0000 Hi all, here is the result of my simple-and-dummy bench script regarding forwarding/ipfw/pf performance evolution on -current on a single-core server with one flow only. It's the result of more than 810 bench tests (including reboot between each) done twice for validating my methodology. # Disclaimer # 1. It's not a "max performance" bench: The purpose is to graph the variation of the performance only. 2. I know that using a single-core server in 2013 is a stupid idea but it's all I've got on my lab :-( # Why all these benchs ? # I've found performance regression regarding packet forwarding/ipfw/pf speed on -current comparing to 9.1 on my old server. glebius@ ask me to do some bisection hunting on different -current revision for spotting the culprit commit. But as a lazy guy, in place of doing bisection, I've choose about 50 svn revision and graph them all: It's a lot's more easy to script this than a bisection algorithm :-) And the result is interesting=85 # The results # The gnuplot diagram in png format with some confirmed specifics spots is available here: http://gugus69.free.fr/freebsd/benchs/current/current-pps.png A confirmed spot is a measurable change between revision N-1 and revision N= . =3D> Remember that I'm used a single-core before reading the result! The "regression" of the new SMP pf is not really a regression: The system is now usable during this high PPS bench and it was not the case before this improvement. ## gnuplot data ## Available here: http://gugus69.free.fr/freebsd/benchs/current/plot/ It's the data and plot file used for generating the graph: You can use them for zooming on it. ## ministat data ## Available here: http://gugus69.free.fr/freebsd/benchs/current/ministat/ You can use it for comparing result between 2 revision, like as example: ministat -s 242160.ipfw 242161.ipfw ## raw data ## Outpout of pkg-gen during all tests: http://gugus69.free.fr/freebsd/benchs/current/raw/ ## nanobsd images # All binary mages used for these benchs are here: http://gugus69.free.fr/freebsd/benchs/current/nanobsd-images/ There is only one "full" image to be used for the first installation, and all other are "upgrade" image. They use the serial port as default console too. # Methodology used # ## First step: building a small lab ## I've used 3 old unused servers and a good switch: - One server as netmap pkt-gen packet generator (1.38Mpps of minimum size packet); - One server as netmap pkt-gen receiver; - One server with 2 NIC in the middle as a router/firewall, serial connection, and nanobsd image on it (very easy to upgrade): IBM eServer xSeries 306m with one core (Intel Pentium4 3.00GHz, hyper-threading disabled) and a dual NIC 82546GB connected to the PCI-X Bus; - a Cisco Catalyst switch for connecting all (its own statistics can be used as a tie breaker if I've got a doubt regarding the result given by netmap pkt-gen). All servers have another NIC for the admin network (bench script send SSH commands and nanobsd image upgrade over this dedicated NIC). I've used netmap pkt-gen for generating smallest packet size from the generator to the receiver like that: pkt-gen -i em0 -t 0 -l 42 -d 1.1.1.1 -D 00:0e:0c:de:45:df -s 2.2.2.2 -w 10 Results was collected on the pkt-gen receiver. ## Second step: building small nanobsd images ## Now we need lot's of small nanobsd images generated from the svn revision number selected for the bench: cf script [1]. About 50 revisions were selected between 236884 to 249506: Candidate chosen by reading the svn commit log. ## Third step: auto-bench script ## This auto-bench script [2] do these tasks: 1. Upgrading the server to the release to be tested; 2. Uploading configuration set to be tested (forwarding-only, ipfw or pf) & reboot; 3. Start the bench test, collecting the result, and reboot: 5 times for each configuration-set; 4 Loop to next configuration set; 5. Loop to next release. ## Last step: converting result for ministat and gnuplot ## I've used a last script for interpreting the output of pkt-gen receiver for ministat and gnuplot [3]. Because I'm not sure if I've used the good method for preparing my data, here is how I've generated the ministat and gnuplot graph: For just one test, the output of pkt-gen in receive mode is lot's of lines like that: main [1085] 400198 pps main [1085] 400287 pps main [1085] 400240 pps main [1085] 400235 pps main [1085] 400245 pps ... I've calculated the median value [3] (thanks ministat) all these results: This give me only one number for the test. =3D> I did the same for each of the 5 same bench tests (same configuration-set, just a reboot between them). And I've put these 5 numbers in the file named SVN-REV.CONFIG-SET. =3D> From these 5 numbers, I've calculated the "median" value again: This give me a unique performance number that I've used as gnuplot data file. ## Bisection ## >From this first result, I've selected others svn revision to generated: The goal was to spot the exact commit that brings the change. But it was not feasible for all regression spotted, because of unbuildable source or non-bootable resulting nanobsd image. ## Final: a full re-run ## Once all my benchs done, I've wait few days and re-started all tests a second time: Before to publish my result, I would to check that all my results were reproducible. # Annexes # ## configuration sets ## ### common to all configuration ### Forwarding enabled Ethernet flow-control disabled (dev.em.0.fc=3D0 and/or dev.em.0.flow_contro= l=3D0) NIC drivers tunned: hw.em.rx_process_limit: 500 hw.em.txd: 4096 hw.em.rxd: 4096 static ARP entry configured on all server and static MAC/Pport entry on the switch too (prevent the switch to age out the packet receiver's MAC address). ### forwarding ### nothing special ### ipfw ### /etc/ipfw.rules: #!/bin/sh fwcmd=3D"/sbin/ipfw" # Flush out the list before we begin. ${fwcmd} -f flush ${fwcmd} add 3000 allow ip from any to any ### pf ### /etc/pf.conf: set skip on lo0 pass [1] http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bisecti= on-gen.sh [2] http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-l= ab.sh [3] http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-l= ab-ministat.sh From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 11:46:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 96646410; Wed, 24 Apr 2013 11:46:53 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-da0-x22c.google.com (mail-da0-x22c.google.com [IPv6:2607:f8b0:400e:c00::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 6CD731019; Wed, 24 Apr 2013 11:46:53 +0000 (UTC) Received: by mail-da0-f44.google.com with SMTP id z20so855566dae.3 for ; Wed, 24 Apr 2013 04:46:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=P/Tv97zsmdy9DbNq86l2r6x20N+pi4nu1pPputORthY=; b=YULd9XCSvU7ifu3ZUwxvtuKP8lSDmwUMVgZZSSf8EGmJRCfGtlVZHnq7arASJyMM/4 9JGcRoFDLHpmE4zsVlTLAUHrG6/f5DUSj+z9JQ3igjJId3aGy4ZC7ImjfbOg7MIvdDJ5 8FegNmdXShYSYqjBJ5sm+kmFpNJi87M8xoQ/Ke7D7uQ51JjojCmTs8JLCSBlP2tF2Fb3 8mBomv/rcXRxsiXQtK7sDZT7XnakLj4SXYsZG1bd71qu3+FKXxkmA1NbWxriwSndRZRn aUiP2Z5YZbAcHJMFb8t7FfGI3PmNzjZGsmqsW9G/EVTFxfMnykN3MtzMldADFfgkYvAx jiMw== MIME-Version: 1.0 X-Received: by 10.68.44.169 with SMTP id f9mr46456076pbm.29.1366804012886; Wed, 24 Apr 2013 04:46:52 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Wed, 24 Apr 2013 04:46:52 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Wed, 24 Apr 2013 04:46:52 -0700 (PDT) In-Reply-To: References: Date: Wed, 24 Apr 2013 14:46:52 +0300 Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current From: Sami Halabi To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 11:46:53 -0000 Oliver, Great and impressive job. If I interpret the plot as is the result say (approximatly of course): 1. Forwarding using ipfw with single rule degrades ~25% the pps. 2. Forwarding with pf however gets ~50%+ of degredation if performance pps. 3. there some point of improved performance (without fw) that went down again somewhere before Clang got prod. 4. I think that the results don't necessarly can be translated to SMP versions because of scheduler, affinity issues. For now i would continue using ipfw :-) Sami On Apr 24, 2013 1:45 PM, "Olivier Cochard-Labb=E9" wro= te: > Hi all, > > here is the result of my simple-and-dummy bench script regarding > forwarding/ipfw/pf performance evolution on -current on a single-core > server with one flow only. > It's the result of more than 810 bench tests (including reboot between > each) done twice for validating my methodology. > > # Disclaimer # > > 1. It's not a "max performance" bench: The purpose is to graph the > variation of the performance only. > 2. I know that using a single-core server in 2013 is a stupid idea but > it's all I've got on my lab :-( > > # Why all these benchs ? # > > I've found performance regression regarding packet forwarding/ipfw/pf > speed on -current comparing to 9.1 on my old server. > glebius@ ask me to do some bisection hunting on different -current > revision for spotting the culprit commit. > But as a lazy guy, in place of doing bisection, I've choose about 50 > svn revision and graph them all: It's a lot's more easy to script this > than a bisection algorithm :-) > And the result is interesting=85 > > # The results # > > The gnuplot diagram in png format with some confirmed specifics spots > is available here: > http://gugus69.free.fr/freebsd/benchs/current/current-pps.png > > A confirmed spot is a measurable change between revision N-1 and revision > N. > > =3D> Remember that I'm used a single-core before reading the result! > The "regression" of the new SMP pf is not really a regression: The > system is now usable during this high PPS bench and it was not the > case before this improvement. > > ## gnuplot data ## > > Available here: http://gugus69.free.fr/freebsd/benchs/current/plot/ > It's the data and plot file used for generating the graph: You can use > them for zooming on it. > > ## ministat data ## > > Available here: http://gugus69.free.fr/freebsd/benchs/current/ministat/ > > You can use it for comparing result between 2 revision, like as example: > ministat -s 242160.ipfw 242161.ipfw > > ## raw data ## > > Outpout of pkg-gen during all tests: > http://gugus69.free.fr/freebsd/benchs/current/raw/ > > ## nanobsd images # > > All binary mages used for these benchs are here: > http://gugus69.free.fr/freebsd/benchs/current/nanobsd-images/ > > There is only one "full" image to be used for the first installation, > and all other are "upgrade" image. > They use the serial port as default console too. > > # Methodology used # > > ## First step: building a small lab ## > > I've used 3 old unused servers and a good switch: > - One server as netmap pkt-gen packet generator (1.38Mpps of minimum > size packet); > - One server as netmap pkt-gen receiver; > - One server with 2 NIC in the middle as a router/firewall, serial > connection, and nanobsd image on it (very easy to upgrade): IBM > eServer xSeries 306m with one core (Intel Pentium4 3.00GHz, > hyper-threading disabled) and a dual NIC 82546GB connected to the > PCI-X Bus; > - a Cisco Catalyst switch for connecting all (its own statistics can > be used as a tie breaker if I've got a doubt regarding the result > given by netmap pkt-gen). > > All servers have another NIC for the admin network (bench script send > SSH commands and nanobsd image upgrade over this dedicated NIC). > > I've used netmap pkt-gen for generating smallest packet size from the > generator to the receiver like that: > pkt-gen -i em0 -t 0 -l 42 -d 1.1.1.1 -D 00:0e:0c:de:45:df -s 2.2.2.2 -w 1= 0 > Results was collected on the pkt-gen receiver. > > ## Second step: building small nanobsd images ## > > Now we need lot's of small nanobsd images generated from the svn > revision number selected for the bench: cf script [1]. > About 50 revisions were selected between 236884 to 249506: Candidate > chosen by reading the svn commit log. > > ## Third step: auto-bench script ## > > This auto-bench script [2] do these tasks: > 1. Upgrading the server to the release to be tested; > 2. Uploading configuration set to be tested (forwarding-only, ipfw > or pf) & reboot; > 3. Start the bench test, collecting the result, and reboot: 5 > times for each configuration-set; > 4 Loop to next configuration set; > 5. Loop to next release. > > ## Last step: converting result for ministat and gnuplot ## > > I've used a last script for interpreting the output of pkt-gen > receiver for ministat and gnuplot [3]. > > Because I'm not sure if I've used the good method for preparing my > data, here is how I've generated the ministat and gnuplot graph: > > For just one test, the output of pkt-gen in receive mode is lot's of > lines like that: > main [1085] 400198 pps > main [1085] 400287 pps > main [1085] 400240 pps > main [1085] 400235 pps > main [1085] 400245 pps > ... > > I've calculated the median value [3] (thanks ministat) all these > results: This give me only one number for the test. > =3D> I did the same for each of the 5 same bench tests (same > configuration-set, just a reboot between them). And I've put these 5 > numbers in the file named SVN-REV.CONFIG-SET. > =3D> From these 5 numbers, I've calculated the "median" value again: > This give me a unique performance number that I've used as gnuplot > data file. > > ## Bisection ## > > From this first result, I've selected others svn revision to > generated: The goal was to spot the exact commit that brings the > change. > But it was not feasible for all regression spotted, because of > unbuildable source or non-bootable resulting nanobsd image. > > ## Final: a full re-run ## > > Once all my benchs done, I've wait few days and re-started all tests a > second time: Before to publish my result, I would to check that all my > results were reproducible. > > # Annexes # > > ## configuration sets ## > > ### common to all configuration ### > Forwarding enabled > Ethernet flow-control disabled (dev.em.0.fc=3D0 and/or > dev.em.0.flow_control=3D0) > NIC drivers tunned: > hw.em.rx_process_limit: 500 > hw.em.txd: 4096 > hw.em.rxd: 4096 > static ARP entry configured on all server and static MAC/Pport entry > on the switch too (prevent the switch to age out the packet receiver's > MAC address). > > ### forwarding ### > nothing special > > ### ipfw ### > > /etc/ipfw.rules: > #!/bin/sh > fwcmd=3D"/sbin/ipfw" > # Flush out the list before we begin. > ${fwcmd} -f flush > ${fwcmd} add 3000 allow ip from any to any > > ### pf ### > > /etc/pf.conf: > set skip on lo0 > pass > > [1] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bisection= -gen.sh > [2] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-lab= .sh > [3] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-lab= -ministat.sh > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= " > From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 12:11:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4F078B7E; Wed, 24 Apr 2013 12:11:19 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-ve0-x22e.google.com (mail-ve0-x22e.google.com [IPv6:2607:f8b0:400c:c01::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 00708113D; Wed, 24 Apr 2013 12:11:18 +0000 (UTC) Received: by mail-ve0-f174.google.com with SMTP id b10so676942vea.5 for ; Wed, 24 Apr 2013 05:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=fnfxopM8170+F4mF4VYM/LWP7u5e3wVjnwZ6cvncPE8=; b=FZ7Sx5LQGuQHBjx5yLVrjFhyOAZSpuNUdv1yHSYA+oOfUJYEmzsMEV2K5zVUqLcNEv WOLElgSfCcS5elj9ct9dP+xMlm6blsT5QqDip/0Jcee+8yVk8obe1C1hn6IkKFvOjtsq ax6V6aXyMYexMA0pnENcuGIGM8RNBfmeJVeIl0do5ntbm9A1gFyzoijIU9QbyggQ+rKv Ocl20Fxbg9kqv9hO+36qjX2TSzLltClrWDIegAshnu0Tme+XAXPmy0CcBtGCgv/NUucs VxHxNlpCspwaHuEr+m3vnKXZhNfdVK9mgFFyIHbjc28I9loxMvqv17v9SHO3IwsK+jGV 8n3w== X-Received: by 10.59.11.199 with SMTP id ek7mr25203600ved.19.1366805478562; Wed, 24 Apr 2013 05:11:18 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Wed, 24 Apr 2013 05:10:58 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Wed, 24 Apr 2013 14:10:58 +0200 X-Google-Sender-Auth: loOTZXIuqfsVXS_vOYDLgNy_bXM Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current To: Sami Halabi Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 12:11:19 -0000 On Wed, Apr 24, 2013 at 1:46 PM, Sami Halabi wrote: > Oliver, > Great and impressive job. Thanks, > 3. there some point of improved performance (without fw) that went down > again somewhere before Clang got prod. => Yes, I'm still working on detected the commit that create this degradation. > For now i would continue using ipfw :-) Don't use this bench for comparing pf and ipfw performance: Using the single parameter "small packet per second throughput" is not enough for comparing firewalls performance. If you read RFC3511 (Benchmarking Methodology for Firewall Performance) you will notice that we need to compare lot's more parameters like: - IP throughput - Concurrent TCP Connection Capacity - Maximum TCP Connection Establishment Rate - Maximum TCP Connection Tear Down Rate - Denial Of Service Handling - HTTP Transfer Rate - Maximum HTTP Transaction Rate - Illegal Traffic Handling - IP Fragmentation Handling - Latency - etc... And I want to add another: High availability feature like with pfsync :-) Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 12:35:14 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 404839C for ; Wed, 24 Apr 2013 12:35:14 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id AAFD31231 for ; Wed, 24 Apr 2013 12:35:13 +0000 (UTC) Received: (qmail 79341 invoked from network); 24 Apr 2013 13:39:58 -0000 Received: from unknown (HELO [62.48.0.94]) ([62.48.0.94]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 24 Apr 2013 13:39:58 -0000 Message-ID: <5177D17B.5050609@freebsd.org> Date: Wed, 24 Apr 2013 14:35:07 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: =?windows-1252?Q?Olivier_Cochard-Labb=E9?= Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 12:35:14 -0000 On 24.04.2013 12:45, Olivier Cochard-Labbé wrote: > Hi all, > > here is the result of my simple-and-dummy bench script regarding > forwarding/ipfw/pf performance evolution on -current on a single-core > server with one flow only. > It's the result of more than 810 bench tests (including reboot between > each) done twice for validating my methodology. Thanks for your excellent work in doing this benchmark time-series, > - One server with 2 NIC in the middle as a router/firewall, serial > connection, and nanobsd image on it (very easy to upgrade): IBM > eServer xSeries 306m with one core (Intel Pentium4 3.00GHz, > hyper-threading disabled) and a dual NIC 82546GB connected to the > PCI-X Bus; however I want to point out that the Pentium4 has about the worst lock overhead of all cpu architectures, even on UP. This may cause certain changes to look much worse than they are on currently popular architectures. For an estimate and time-series comparison your bench test is very helpful though. -- Andre From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 17:50:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3CE0F259 for ; Wed, 24 Apr 2013 17:50:21 +0000 (UTC) (envelope-from ml@netfence.it) Received: from smtp.eutelia.it (mp1-smtp-6.eutelia.it [62.94.10.166]) by mx1.freebsd.org (Postfix) with ESMTP id BD25F12E8 for ; Wed, 24 Apr 2013 17:50:20 +0000 (UTC) Received: from ns2.biolchim.it (ip-188-188.sn2.eutelia.it [83.211.188.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.eutelia.it (Eutelia) with ESMTP id 41B08659D12 for ; Wed, 24 Apr 2013 19:31:45 +0200 (CEST) Received: from soth.ventu (adsl-ull-39-183.41-151.net24.it [151.41.183.39]) (authenticated bits=0) by ns2.biolchim.it (8.14.7/8.14.7) with ESMTP id r3OHVedv074151 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 24 Apr 2013 19:31:41 +0200 (CEST) (envelope-from ml@netfence.it) Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.6/8.14.5) with ESMTP id r3OHVWOY037038 for ; Wed, 24 Apr 2013 19:31:32 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <517816F4.7070300@netfence.it> Date: Wed, 24 Apr 2013 19:31:32 +0200 From: Andrea Venturoli User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:17.0) Gecko/20130406 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: if_bridge hangs server Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Scanned-By: MIMEDefang 2.73 on 10.1.2.13 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (ns2.biolchim.it [192.168.2.203]); Wed, 24 Apr 2013 19:31:41 +0200 (CEST) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 17:50:21 -0000 Hello. I hope someone can help me with the following problem... The box runs a 8.3p7/i386 and has three physical ethernet interfaces: em0, em1 and fxp1. em0 and em1 are bonded into lagg0, over which carp0 and carp1 run. fxp0 has three vlans: vlan1, vlan2 and vlan3, over which there are respectively carp3/carp4, carp6/carp7, carp9/carp10. To make it clearer, here's the extract from my rc.conf: > cloned_interfaces="lagg0 vlan1 vlan2 vlan3 carp0 carp1 carp3 carp4 carp6 carp7 carp9 carp10" > ifconfig_em0="up" > ifconfig_em1="up" > ifconfig_lagg0="laggproto lacp laggport em0 laggport em1 192.168.101.1 netmask 255.255.255.0" > ifconfig_carp0="vhid 1 pass xxxxxxx 192.168.101.10" > ifconfig_carp1="vhid 2 advskew 200 pass xxxxxxxx 192.168.101.10" > ifconfig_fxp0="up" > ifconfig_vlan1="inet xx.xxx.xx.xx netmask 255.255.255.248 vlan 4 vlandev fxp0" > ifconfig_vlan2="inet xx.xxx.xxx.xxx netmask 255.255.255.248 vlan 2 vlandev fxp0" > ifconfig_vlan3="inet 192.168.2.201 netmask 255.255.255.0 vlan 3 vlandev fxp0" > ifconfig_carp3="vhid 4 pass xxxx xx.xxx.xx.xx" > ifconfig_carp4="vhid 5 advskew 100 pass xxxxxxxx xx.xxx.xx.xx" > ifconfig_carp6="vhid 7 pass xxxxxx xx.xxx.xxx.xxx" > ifconfig_carp7="vhid 8 advskew 100 pass xxxxxxxxxxx xx.xxx.xxx.xxx" > ifconfig_carp9="vhid 10 pass xxxxxxxx 192.168.2.203" > ifconfig_carp10="vhid 11 advskew 100 pass xxxxxxxx 192.168.2.203" Now I need a tap based OpenVPN, so, per instructions I have found, I run: > ifconfig bridge0 create addm lagg0 up After the above command is issued, the box won't live for other five minutes; everythings works fine as before, but the server will soon hang: no crashdump, no message, no reboot, just a sudden freeze. The only way to restart is the reset button. So I'm stuck and don't know what to do. Are there any known issue regarding if_bridge which I might be encountering? Anything related to if_bridge interaction with carp, vlan or lagg? With ipfw? Any hint on how to debug this? Would an upgrade to 9.1 help? bye & Thanks av. From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 18:51:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 427E2A4 for ; Wed, 24 Apr 2013 18:51:05 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (lakerest.net [70.155.160.98]) by mx1.freebsd.org (Postfix) with ESMTP id C4EEF166C for ; Wed, 24 Apr 2013 18:51:04 +0000 (UTC) Received: from [10.1.1.213] ([10.1.1.213]) (authenticated bits=0) by lakerest.net (8.14.4/8.14.3) with ESMTP id r3OIovXA015987 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 24 Apr 2013 14:50:58 -0400 (EDT) (envelope-from rrs@lakerest.net) Subject: Re: Default route changes unexpectedly Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Randall Stewart In-Reply-To: Date: Wed, 24 Apr 2013 14:50:57 -0400 Content-Transfer-Encoding: 7bit Message-Id: References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> To: Tom Evans X-Mailer: Apple Mail (2.1283) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 18:51:05 -0000 All Ok I fixed it ;-) Its in SVN r249848. I will see about getting it to 9 stable, 8 stable and maybe even 8.4 if RE will let me ;-) R On Apr 23, 2013, at 9:40 AM, Tom Evans wrote: > On Tue, Apr 23, 2013 at 1:08 PM, Randall Stewart wrote: >> Ok >> >> I too have been struck by this *multiple* times on my base home router. >> > > I hate "me too" style posts, since often they conflate unrelated > issues - however, "me too"! > > In my scenario, I have a simple home router with a wan if connected to > an ADSL modem, an internal if connected to a pretty ordinary switch > and the rest of the home network, using pf to NAT the connection > (pretty basic stuff). > > Infrequently, I can no longer connect to or ping the router from > internal connections, and have to grab a console, restart netif and > routing, and everything then works again. > > However, I also have an openvpn connection to work running on the > router. Work seem to believe that the reason there are 3 huge private > network ranges is so that they can use the 10/8 block for DC > infrastructure, the 172.16/12 block for offices and the 192.168/16 bit > block for VPNs. Until now, I had been assuming - without any proof - > that everything works great until openvpn gets told that 192.168.1/8 > should be routed down the VPN, at which point everything local is > inaccessible. > > Is there something useful I can look at when this next occurs that > would explain why or how it is wedged, so that I can either rule > myself in or out of this case? > > Cheers > > Tom > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > ------------------------------ Randall Stewart 803-317-4952 (cell) From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 18:57:37 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 31236398 for ; Wed, 24 Apr 2013 18:57:37 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x234.google.com (mail-we0-x234.google.com [IPv6:2a00:1450:400c:c03::234]) by mx1.freebsd.org (Postfix) with ESMTP id BD9D516D0 for ; Wed, 24 Apr 2013 18:57:36 +0000 (UTC) Received: by mail-we0-f180.google.com with SMTP id x43so1844687wey.39 for ; Wed, 24 Apr 2013 11:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=HQ5d80Dhfp7y6XQZIRPtvuoQd0KYISTlU50Dvag1FyM=; b=04dqAJLXxRE+8+CLH1CW6ZoV87IeNfZjj3jJtmeTpsv6f5TuhMc2mR9AWYsM869Q5+ Ixut0ubgU+234DWQxd/dpv2nMghjv/quLN36tK8Ek2hFjxLF2dkBaYzyvpaZ2YpQFJMY 1xMCqvSvleBqHBQTHJqJO0wmO7PhRSEpaoWSU2YOqQ/v9jfkpohYjdFSO2DJteyhaDR4 QYoyzClp15nTealv9awVw97wQLJ1+8qvejtcGxPAtVWw/+C/OQmApOv9cGpHkh32p044 /qBY1QEgeZhZ5/KZCXOhJax+UqPwcBFUxTytgqyaAwMJy977ULuZEhMmNNZcleXwG+j0 nv6A== MIME-Version: 1.0 X-Received: by 10.180.93.134 with SMTP id cu6mr69226848wib.8.1366829855730; Wed, 24 Apr 2013 11:57:35 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Wed, 24 Apr 2013 11:57:35 -0700 (PDT) In-Reply-To: References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> Date: Wed, 24 Apr 2013 11:57:35 -0700 X-Google-Sender-Auth: hZcEIr3e8GNpZAKTlgtYNs93NN0 Message-ID: Subject: Re: Default route changes unexpectedly From: Adrian Chadd To: Randall Stewart Content-Type: text/plain; charset=ISO-8859-1 Cc: Tom Evans , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 18:57:37 -0000 Is this an issue on -7 and -6? (Since people do still run it, and it seems a simple enough fix?) adrian On 24 April 2013 11:50, Randall Stewart wrote: > All > > Ok I fixed it ;-) > > Its in SVN r249848. > > I will see about getting it to 9 stable, 8 stable and maybe even > 8.4 if RE will let me ;-) > > R > On Apr 23, 2013, at 9:40 AM, Tom Evans wrote: > >> On Tue, Apr 23, 2013 at 1:08 PM, Randall Stewart wrote: >>> Ok >>> >>> I too have been struck by this *multiple* times on my base home router. >>> >> >> I hate "me too" style posts, since often they conflate unrelated >> issues - however, "me too"! >> >> In my scenario, I have a simple home router with a wan if connected to >> an ADSL modem, an internal if connected to a pretty ordinary switch >> and the rest of the home network, using pf to NAT the connection >> (pretty basic stuff). >> >> Infrequently, I can no longer connect to or ping the router from >> internal connections, and have to grab a console, restart netif and >> routing, and everything then works again. >> >> However, I also have an openvpn connection to work running on the >> router. Work seem to believe that the reason there are 3 huge private >> network ranges is so that they can use the 10/8 block for DC >> infrastructure, the 172.16/12 block for offices and the 192.168/16 bit >> block for VPNs. Until now, I had been assuming - without any proof - >> that everything works great until openvpn gets told that 192.168.1/8 >> should be routed down the VPN, at which point everything local is >> inaccessible. >> >> Is there something useful I can look at when this next occurs that >> would explain why or how it is wedged, so that I can either rule >> myself in or out of this case? >> >> Cheers >> >> Tom >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > ------------------------------ > Randall Stewart > 803-317-4952 (cell) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 19:11:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3BFB3D72; Wed, 24 Apr 2013 19:11:04 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-oa0-f44.google.com (mail-oa0-f44.google.com [209.85.219.44]) by mx1.freebsd.org (Postfix) with ESMTP id F074117C0; Wed, 24 Apr 2013 19:11:03 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id h1so2100538oag.3 for ; Wed, 24 Apr 2013 12:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=FIHPKjQA0lNPl3JxAsKwZQQSQ2uO5cVhGsd2sHfUUx0=; b=SZ5qJq3XGYiyn+vCpyX9ewBW73gMjnU4E3acg9Vm5YyhmmNeEKktWrlKVZm4dnl7TT JHciIgdD+tZAJPpVmPkarcPd6a8iH0CwKdJKwX554HJucQFZW03xO59pV/6vyzUlGb1s PkxHQ71YUE3BWs89bOKiY2O8L/0FKAoeLcmkqRQWbZA1hFLQn69Cm5ZWqDvtYAPxy5o3 YgpC9yQMLNubjuYyHO0XY/dw9OdGHnL0yLfMUhbaNdc47gmB8jWkrwUwBMoIiGim53m1 doqZmNmkSD1012DB4jR20trQsP9w+rp15uhW3CS0FQGg7I/WumRSy7DAWPS6mrDLXjM+ DF3A== MIME-Version: 1.0 X-Received: by 10.60.56.168 with SMTP id b8mr10981974oeq.5.1366830663366; Wed, 24 Apr 2013 12:11:03 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.60.78.33 with HTTP; Wed, 24 Apr 2013 12:11:03 -0700 (PDT) In-Reply-To: References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> Date: Wed, 24 Apr 2013 15:11:03 -0400 X-Google-Sender-Auth: iEFJJC-vhg780__RdjVkraVzWco Message-ID: Subject: Re: Default route changes unexpectedly From: Ed Maste To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , Randall Stewart X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 19:11:04 -0000 On 24 April 2013 14:57, Adrian Chadd wrote: > Is this an issue on -7 and -6? I believe so, and it should get merged there as well. From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 19:21:42 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2FEA34C6; Wed, 24 Apr 2013 19:21:42 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 96160185A; Wed, 24 Apr 2013 19:21:41 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id l18so1029496wgh.24 for ; Wed, 24 Apr 2013 12:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=/vU4c4SR3hlagofyXo6tvjqfGjGWVJlf9bnBGap3cfw=; b=PbDBc97iqdjh/EzICzhxHwrUSTQSlnpgb25Xe8xaZlSlAV/sX+XT3DYdaUsihbD8Xz OHWbQK5LQskNQbwvnNYFe5kFlO5QqLNeDGkI5veEkVUFf/qrAccQM4VHi7vpMNDNnZQP SaTXHT2Pg+7oGRnziEj2IOJwaEd8EVKRdq0uvVbXwvjIWJwtpN0wq37QaY8YlueWFoH7 AOhQAOWbxGIR0h2SYZSHqcgy9NJQxoSzx5iNoamrnhd0jyJRCgpxO+kQSMKP2YGNFkdc IaCK6hjhkIKsE6+7aJXHE7EXKRBBeK+ERrH7FuMPawMUJL3RgGipXcgFKZ2t0YPvcewy kPcw== MIME-Version: 1.0 X-Received: by 10.194.119.33 with SMTP id kr1mr70707398wjb.36.1366831300651; Wed, 24 Apr 2013 12:21:40 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Wed, 24 Apr 2013 12:21:40 -0700 (PDT) In-Reply-To: References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> Date: Wed, 24 Apr 2013 12:21:40 -0700 X-Google-Sender-Auth: te0He8LmLiD4LAni7cwGGbz9cQw Message-ID: Subject: Re: Default route changes unexpectedly From: Adrian Chadd To: Ed Maste Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , Randall Stewart X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 19:21:42 -0000 On 24 April 2013 12:11, Ed Maste wrote: > On 24 April 2013 14:57, Adrian Chadd wrote: >> Is this an issue on -7 and -6? > > I believe so, and it should get merged there as well. rrs - preeeeeeeety please? :) adrian From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 20:01:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2F21FC54; Wed, 24 Apr 2013 20:01:09 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 045F31BAE; Wed, 24 Apr 2013 20:01:08 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-237-17.lns20.per1.internode.on.net [121.45.237.17]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r3OK13Cj065304 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 24 Apr 2013 13:01:06 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <517839FF.6050202@freebsd.org> Date: Thu, 25 Apr 2013 04:01:03 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: =?windows-1252?Q?Olivier_Cochard-Labb=E9?= Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 20:01:09 -0000 On 4/24/13 6:45 PM, Olivier Cochard-Labbé wrote: > Hi all, > > here is the result of my simple-and-dummy bench script regarding > forwarding/ipfw/pf performance evolution on -current on a single-core > server with one flow only. > It's the result of more than 810 bench tests (including reboot between > each) done twice for validating my methodology. > > very cool bet that was fun.. From owner-freebsd-net@FreeBSD.ORG Wed Apr 24 20:21:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 61D85493; Wed, 24 Apr 2013 20:21:21 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 303A21CDA; Wed, 24 Apr 2013 20:21:21 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r3OKLKts029942; Wed, 24 Apr 2013 16:21:20 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <51783ED0.1040601@sentex.net> Date: Wed, 24 Apr 2013 16:21:36 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: =?windows-1252?Q?Olivier_Cochard-Labb=E9?= Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current References: In-Reply-To: X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 20:21:21 -0000 On 4/24/2013 6:45 AM, Olivier Cochard-Labbé wrote: > # Why all these benchs ? # > > I've found performance regression regarding packet forwarding/ipfw/pf > speed on -current comparing to 9.1 on my old server. BTW, how much of a drop in performance as compared to 9.1 ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 05:40:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 148F7535; Thu, 25 Apr 2013 05:40:32 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-vb0-x236.google.com (mail-vb0-x236.google.com [IPv6:2607:f8b0:400c:c02::236]) by mx1.freebsd.org (Postfix) with ESMTP id B72E71FEC; Thu, 25 Apr 2013 05:40:31 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id w16so2336764vbf.41 for ; Wed, 24 Apr 2013 22:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=V4VxMPPFg/VPsHqf/fCDg2U8ZaOHTO/y26449xjlBN8=; b=Spg20z21k/4an2UQql6X6RmCc8aEeS/ojh2FDj9C0uBflR5YBLZ0GgIVTgiPtLUzeX CabAceBc8D+f+cOYSUWVXUHzdU0aP8WF5fDSRJgtgnurgx2EgU6JMUPWTV92jLVrPGpw d5rQwJqEh7z44HVXCmFv7Cxb6rZVtrT9hCV81ks7bw9JY5Yy5IIksmUxNP5rpkcm+kxx fC+0wA/ikBNYUL1Un8//jDfDx0vjFL48Xb2fZ2zQ2KCUvP6CSdF5ETyjCN08jXXXXoUw 1fP3hNcBsel3RUIGwwvjRWN+yhugXl9+uzr15FF3UnOu2liQ2nBZBSGn3v1fTlPIEn1h ApKA== X-Received: by 10.59.11.199 with SMTP id ek7mr26676929ved.19.1366868430482; Wed, 24 Apr 2013 22:40:30 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Wed, 24 Apr 2013 22:40:10 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Thu, 25 Apr 2013 07:40:10 +0200 X-Google-Sender-Auth: 6QC2VpXWK8UHLrZxpwVYCHugRNM Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current To: Sami Halabi Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 05:40:32 -0000 On Wed, Apr 24, 2013 at 1:46 PM, Sami Halabi wrote: > 3. there some point of improved performance (without fw) that went down > again somewhere before Clang got prod. Found it ! It's commit 242402: "Rework the known mutexes..." ministat -s 242401.forwarding 242402.forwarding x 242401.forwarding + 242402.forwarding +---------------------------------------------------------------------------------------+ | + | |+ + + + x xx x x| | |____A____| | | |_____A_M___| | +---------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 417527 420242 418902 419074 1049.7974 + 5 402211 404828 404096 403689 1237.6696 Difference at 95.0% confidence -15385 +/- 1673.69 -3.67119% +/- 0.399377% (Student's t, pooled s = 1147.58) From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 09:28:31 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0BD543A6 for ; Thu, 25 Apr 2013 09:28:31 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5ADE6183C for ; Thu, 25 Apr 2013 09:28:30 +0000 (UTC) Received: (qmail 99753 invoked from network); 25 Apr 2013 10:32:57 -0000 Received: from unknown (HELO [62.48.0.94]) ([62.48.0.94]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 25 Apr 2013 10:32:57 -0000 Message-ID: <5178F72F.90008@freebsd.org> Date: Thu, 25 Apr 2013 11:28:15 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Olivier_Cochard-Labb=E9?= Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 09:28:31 -0000 On 25.04.2013 07:40, Olivier Cochard-Labbé wrote: > On Wed, Apr 24, 2013 at 1:46 PM, Sami Halabi wrote: >> 3. there some point of improved performance (without fw) that went down >> again somewhere before Clang got prod. > > Found it ! > > It's commit 242402: "Rework the known mutexes..." Again one has to be really careful drawing any firm conclusions from this as it was measured on a Pentium4 and UP kernel (GENERIC would add WITNESS and INVARIANT overhead as well). The Pentium4 is about the worst micro-architecture when it comes to locks and easily regresses. At the same time modern Intel Core i[3-7] and AMD64 may actually improve with these changes. Unless more recent micro-archs have been shown to exhibit the same regression we can't claim this change was bad (other than for Pentium4). -- Andre > ministat -s 242401.forwarding 242402.forwarding > x 242401.forwarding > + 242402.forwarding > +---------------------------------------------------------------------------------------+ > | + > | > |+ + + + > x xx x x| > | > |____A____| | > | |_____A_M___| > | > +---------------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 417527 420242 418902 419074 1049.7974 > + 5 402211 404828 404096 403689 1237.6696 > Difference at 95.0% confidence > -15385 +/- 1673.69 > -3.67119% +/- 0.399377% > (Student's t, pooled s = 1147.58) > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 10:53:24 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 43079A0; Thu, 25 Apr 2013 10:53:24 +0000 (UTC) (envelope-from oliver.pntr@gmail.com) Received: from mail-oa0-f47.google.com (mail-oa0-f47.google.com [209.85.219.47]) by mx1.freebsd.org (Postfix) with ESMTP id 04F031CF9; Thu, 25 Apr 2013 10:53:23 +0000 (UTC) Received: by mail-oa0-f47.google.com with SMTP id n9so2628032oag.20 for ; Thu, 25 Apr 2013 03:53:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:content-transfer-encoding; bh=xwYyMTKnNHOboi9Kxa8I7gKOaQpnn0BpAYnL13brURI=; b=wJtLUijC91hkQBEASb0DsCiSuRG3oPe5jjoQLGcaNu17gKPNaiwKG/p/HkMvFJrstM wQTdjGThZ4IFHuCh0t79zG+9xJZwctEWqdwD6FJ2XpouIwCPF3qp53lPyRERGaMdQK8h ySA2QqPtIq8DV+FRly2SxHXw4esdJawjnvZYLZQPaVRdxXV1UKVXnfMxPxmo6K2LisgM qSG2k7Sp4D0vxmxYAbG8Vzv3Z+u8aCTo3WBwHBoJaYyjlDVPyTs9xbmiztMrMvuZlJ1v dpcnY+1medSgJBDAAWPPOLn0/IW53uWCdMq1XBXQDJibNggayj06jjrtUZ+zmQ7QhQF7 LA+A== MIME-Version: 1.0 X-Received: by 10.182.220.161 with SMTP id px1mr14997147obc.42.1366887203199; Thu, 25 Apr 2013 03:53:23 -0700 (PDT) Received: by 10.182.161.100 with HTTP; Thu, 25 Apr 2013 03:53:23 -0700 (PDT) In-Reply-To: References: Date: Thu, 25 Apr 2013 12:53:23 +0200 Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current From: Oliver Pinter To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 10:53:24 -0000 nice work! On 4/24/13, Olivier Cochard-Labb=E9 wrote: > Hi all, > > here is the result of my simple-and-dummy bench script regarding > forwarding/ipfw/pf performance evolution on -current on a single-core > server with one flow only. > It's the result of more than 810 bench tests (including reboot between > each) done twice for validating my methodology. > > # Disclaimer # > > 1. It's not a "max performance" bench: The purpose is to graph the > variation of the performance only. > 2. I know that using a single-core server in 2013 is a stupid idea but > it's all I've got on my lab :-( > > # Why all these benchs ? # > > I've found performance regression regarding packet forwarding/ipfw/pf > speed on -current comparing to 9.1 on my old server. > glebius@ ask me to do some bisection hunting on different -current > revision for spotting the culprit commit. > But as a lazy guy, in place of doing bisection, I've choose about 50 > svn revision and graph them all: It's a lot's more easy to script this > than a bisection algorithm :-) For the future: use the FreeBSD's git repo, and then the git bisect command. ;) https://www.kernel.org/pub/software/scm/git/docs/git-bisect.html > And the result is interesting=85 > > # The results # > > The gnuplot diagram in png format with some confirmed specifics spots > is available here: > http://gugus69.free.fr/freebsd/benchs/current/current-pps.png > > A confirmed spot is a measurable change between revision N-1 and revision > N. > > =3D> Remember that I'm used a single-core before reading the result! > The "regression" of the new SMP pf is not really a regression: The > system is now usable during this high PPS bench and it was not the > case before this improvement. > > ## gnuplot data ## > > Available here: http://gugus69.free.fr/freebsd/benchs/current/plot/ > It's the data and plot file used for generating the graph: You can use > them for zooming on it. > > ## ministat data ## > > Available here: http://gugus69.free.fr/freebsd/benchs/current/ministat/ > > You can use it for comparing result between 2 revision, like as example: > ministat -s 242160.ipfw 242161.ipfw > > ## raw data ## > > Outpout of pkg-gen during all tests: > http://gugus69.free.fr/freebsd/benchs/current/raw/ > > ## nanobsd images # > > All binary mages used for these benchs are here: > http://gugus69.free.fr/freebsd/benchs/current/nanobsd-images/ > > There is only one "full" image to be used for the first installation, > and all other are "upgrade" image. > They use the serial port as default console too. > > # Methodology used # > > ## First step: building a small lab ## > > I've used 3 old unused servers and a good switch: > - One server as netmap pkt-gen packet generator (1.38Mpps of minimum > size packet); > - One server as netmap pkt-gen receiver; > - One server with 2 NIC in the middle as a router/firewall, serial > connection, and nanobsd image on it (very easy to upgrade): IBM > eServer xSeries 306m with one core (Intel Pentium4 3.00GHz, > hyper-threading disabled) and a dual NIC 82546GB connected to the > PCI-X Bus; > - a Cisco Catalyst switch for connecting all (its own statistics can > be used as a tie breaker if I've got a doubt regarding the result > given by netmap pkt-gen). > > All servers have another NIC for the admin network (bench script send > SSH commands and nanobsd image upgrade over this dedicated NIC). > > I've used netmap pkt-gen for generating smallest packet size from the > generator to the receiver like that: > pkt-gen -i em0 -t 0 -l 42 -d 1.1.1.1 -D 00:0e:0c:de:45:df -s 2.2.2.2 -w 1= 0 > Results was collected on the pkt-gen receiver. > > ## Second step: building small nanobsd images ## > > Now we need lot's of small nanobsd images generated from the svn > revision number selected for the bench: cf script [1]. > About 50 revisions were selected between 236884 to 249506: Candidate > chosen by reading the svn commit log. > > ## Third step: auto-bench script ## > > This auto-bench script [2] do these tasks: > 1. Upgrading the server to the release to be tested; > 2. Uploading configuration set to be tested (forwarding-only, ipfw > or pf) & reboot; > 3. Start the bench test, collecting the result, and reboot: 5 > times for each configuration-set; > 4 Loop to next configuration set; > 5. Loop to next release. > > ## Last step: converting result for ministat and gnuplot ## > > I've used a last script for interpreting the output of pkt-gen > receiver for ministat and gnuplot [3]. > > Because I'm not sure if I've used the good method for preparing my > data, here is how I've generated the ministat and gnuplot graph: > > For just one test, the output of pkt-gen in receive mode is lot's of > lines like that: > main [1085] 400198 pps > main [1085] 400287 pps > main [1085] 400240 pps > main [1085] 400235 pps > main [1085] 400245 pps > ... > > I've calculated the median value [3] (thanks ministat) all these > results: This give me only one number for the test. > =3D> I did the same for each of the 5 same bench tests (same > configuration-set, just a reboot between them). And I've put these 5 > numbers in the file named SVN-REV.CONFIG-SET. > =3D> From these 5 numbers, I've calculated the "median" value again: > This give me a unique performance number that I've used as gnuplot > data file. > > ## Bisection ## > > From this first result, I've selected others svn revision to > generated: The goal was to spot the exact commit that brings the > change. > But it was not feasible for all regression spotted, because of > unbuildable source or non-bootable resulting nanobsd image. > > ## Final: a full re-run ## > > Once all my benchs done, I've wait few days and re-started all tests a > second time: Before to publish my result, I would to check that all my > results were reproducible. > > # Annexes # > > ## configuration sets ## > > ### common to all configuration ### > Forwarding enabled > Ethernet flow-control disabled (dev.em.0.fc=3D0 and/or > dev.em.0.flow_control=3D0) > NIC drivers tunned: > hw.em.rx_process_limit: 500 > hw.em.txd: 4096 > hw.em.rxd: 4096 > static ARP entry configured on all server and static MAC/Pport entry > on the switch too (prevent the switch to age out the packet receiver's > MAC address). > > ### forwarding ### > nothing special > > ### ipfw ### > > /etc/ipfw.rules: > #!/bin/sh > fwcmd=3D"/sbin/ipfw" > # Flush out the list before we begin. > ${fwcmd} -f flush > ${fwcmd} add 3000 allow ip from any to any > > ### pf ### > > /etc/pf.conf: > set skip on lo0 > pass > > [1] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bisection= -gen.sh > [2] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-lab= .sh > [3] > http://sourceforge.net/p/bsdrp/code/HEAD/tree/trunk/BSDRP/tools/bench-lab= -ministat.sh > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= " > From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 15:35:30 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 072F990C; Thu, 25 Apr 2013 15:35:30 +0000 (UTC) (envelope-from pluknet@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id D3C031BAE; Thu, 25 Apr 2013 15:35:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3PFZTiv026853; Thu, 25 Apr 2013 15:35:29 GMT (envelope-from pluknet@freefall.freebsd.org) Received: (from pluknet@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3PFZTKE026852; Thu, 25 Apr 2013 15:35:29 GMT (envelope-from pluknet) Date: Thu, 25 Apr 2013 15:35:29 GMT Message-Id: <201304251535.r3PFZTKE026852@freefall.freebsd.org> To: pluknet@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-net@FreeBSD.org From: pluknet@FreeBSD.org Subject: Re: kern/178116: Kernel panic: general protection fault in tcp_do_segment X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 15:35:30 -0000 Synopsis: Kernel panic: general protection fault in tcp_do_segment Responsible-Changed-From-To: freebsd-amd64->freebsd-net Responsible-Changed-By: pluknet Responsible-Changed-When: Thu Apr 25 15:34:45 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178116 From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 17:16:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DBFCC7B for ; Thu, 25 Apr 2013 17:16:26 +0000 (UTC) (envelope-from ncrogers@gmail.com) Received: from mail-vc0-f179.google.com (mail-vc0-f179.google.com [209.85.220.179]) by mx1.freebsd.org (Postfix) with ESMTP id 9EE021FC7 for ; Thu, 25 Apr 2013 17:16:26 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id hz10so510526vcb.10 for ; Thu, 25 Apr 2013 10:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=rGFUWZywH2nAJcpoBVAphsDwugi1Bqu01JxGzYIVrrE=; b=oK/rsztmPM3/tng6SXjzpsLU1EdHV2OYEtG2Zvvqr/NUE3af/QRrbLOC4PkDMriOpd 04qNFwIHORI1eQJTSt9twWVQPaZesKKS/gvhFuUzZKpoLMzkedmED17G7xAnYSDYUxbZ QvNTT/ZjQ9zoRSZw8enfvU+qHoB45t0yaFI+Vremncz2ni7xSJHFcdSMpiJsXM2RQwiD O9tdiYdXys6ErSjlVhr7xokZ4angJUj5Rd4L5zq1zqyUv4fOojB0Vn2516OnOH0P/Z44 Gfb5uSLNNKxam1JmNbAA76Euq3wwDo1YJyz0UVEG3AMn6/rkga0qRaiTsstqauYbGuA2 hmaA== MIME-Version: 1.0 X-Received: by 10.52.25.99 with SMTP id b3mr23144611vdg.99.1366910180233; Thu, 25 Apr 2013 10:16:20 -0700 (PDT) Received: by 10.52.176.131 with HTTP; Thu, 25 Apr 2013 10:16:20 -0700 (PDT) In-Reply-To: References: <5136FD71.6000408@freebsd.org> <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> Date: Thu, 25 Apr 2013 10:16:20 -0700 Message-ID: Subject: Re: Default route changes unexpectedly From: Nick Rogers To: Randall Stewart Content-Type: text/plain; charset=ISO-8859-1 Cc: Tom Evans , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 17:16:26 -0000 On Wed, Apr 24, 2013 at 11:50 AM, Randall Stewart wrote: > All > > Ok I fixed it ;-) > > Its in SVN r249848. > > I will see about getting it to 9 stable, 8 stable and maybe even > 8.4 if RE will let me ;-) > Great work. Thanks so much. I was afraid this would linger forever! > R > On Apr 23, 2013, at 9:40 AM, Tom Evans wrote: > >> On Tue, Apr 23, 2013 at 1:08 PM, Randall Stewart wrote: >>> Ok >>> >>> I too have been struck by this *multiple* times on my base home router. >>> >> >> I hate "me too" style posts, since often they conflate unrelated >> issues - however, "me too"! >> >> In my scenario, I have a simple home router with a wan if connected to >> an ADSL modem, an internal if connected to a pretty ordinary switch >> and the rest of the home network, using pf to NAT the connection >> (pretty basic stuff). >> >> Infrequently, I can no longer connect to or ping the router from >> internal connections, and have to grab a console, restart netif and >> routing, and everything then works again. >> >> However, I also have an openvpn connection to work running on the >> router. Work seem to believe that the reason there are 3 huge private >> network ranges is so that they can use the 10/8 block for DC >> infrastructure, the 172.16/12 block for offices and the 192.168/16 bit >> block for VPNs. Until now, I had been assuming - without any proof - >> that everything works great until openvpn gets told that 192.168.1/8 >> should be routed down the VPN, at which point everything local is >> inaccessible. >> >> Is there something useful I can look at when this next occurs that >> would explain why or how it is wedged, so that I can either rule >> myself in or out of this case? >> >> Cheers >> >> Tom >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > ------------------------------ > Randall Stewart > 803-317-4952 (cell) > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 18:24:30 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2CE8C1DB for ; Thu, 25 Apr 2013 18:24:30 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) by mx1.freebsd.org (Postfix) with ESMTP id 04BBB12A5 for ; Thu, 25 Apr 2013 18:24:29 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id 4so1972405pdd.31 for ; Thu, 25 Apr 2013 11:24:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=PqZU9zr7hiW0RQv3EFtSghG0FY386vzs5l5oEnV/A8s=; b=CM2mgc7MP3qYxPD2aKZZGbebKhnW7V8m6no8t5oneO9pEki3ZmVrtIOyhT3F/QsbiC dg742FzYX4dojqEmmB2uLxHCH70FjIm0nJGYFXItXJ7uo+4YPO2kgeIEheQEdZFlZf3b Jd3+07FCkwTDha/7vHpc8bkQAeC/3WiYQ0QYY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=PqZU9zr7hiW0RQv3EFtSghG0FY386vzs5l5oEnV/A8s=; b=MEm0XJoBLJ4wseEGG/l7/xHXsBW2XArJx4sqAaLZwiwvYlE+HTXCuYBaQUBHnAgGff lVgBctfVHL0Qj0EE96qmSOlLUHvuLjAZWT/yC+GV/rVBPL6HA8GdQQwDccZQEcYnprlC iZ2cRJzHYWCYds+2hsnVWkgXH88qrj7DiIex4tXBLKxAUNcnxELL1D3Ar0X/pDsWuxxj tmG+bXcv9KpI+5XNHTICx1QVTsI9QusDK5XJSuOGvxUccXTbP3MgpIVyqzbbswiM8NGD kxkcx41dg5Ck1+VUm67//Gt8XsrA21gRv0uYupRVtIYthYA9sNufAZbHiEbw6hBQcbmZ vS3g== X-Received: by 10.66.120.173 with SMTP id ld13mr26821105pab.187.1366914269384; Thu, 25 Apr 2013 11:24:29 -0700 (PDT) Received: from [172.30.0.50] (hgfw-01.soe.ucsc.edu. [128.114.61.130]) by mx.google.com with ESMTPSA id l4sm8294462pbo.6.2013.04.25.11.24.27 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 25 Apr 2013 11:24:28 -0700 (PDT) Message-ID: <517974DA.5090809@soe.ucsc.edu> Date: Thu, 25 Apr 2013 11:24:26 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (X11; Linux i686; rv:10.0.6esrpre) Gecko/20120717 Thunderbird/10.0.6 MIME-Version: 1.0 To: Kajetan Staszkiewicz Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> In-Reply-To: <201304240134.22740.vegeta@tuxpowered.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQnnGdeqRrH5FOpi9OxzCzgPOzEj0n2NLaC3XSThhd9dAWVqaE04mchInXRk2JzdPJi7I2VY Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 18:24:30 -0000 > As far as I understand, processing of packets by pf takes place in receiving > network card's interrupt handler even up to sending the packet via another > network card (at least in my case, when using route-to targets, which make > routing inside pf). That's interesting. So even though pf is giant locked, you can still scale the maximum capacity of your firewall, in this case, simply by adding more CPU cores? To handle the extra interrupts? So more cores = more packets per second, if you give each extra core an additional interrupt queue? > How do you count the 140kpps value? One interface, both, in, out? I'd like to > relate this somehow to my values. Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx spike to 150kpps occasionally. This is a pfSense box, which includes RRD graphs of packet rates, that's how I'm getting the number. I'm not sure how they are obtaining that metric under the hood. But we have not disabled HT and some other items, so that number will change is my guess. We also may add another CPU die to the mix to see if we can add interrupt queues to more cores to increase performance. From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 18:30:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5E9EE2CD; Thu, 25 Apr 2013 18:30:36 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 9E2E512DD; Thu, 25 Apr 2013 18:30:35 +0000 (UTC) Received: by mail-we0-f175.google.com with SMTP id t11so2869039wey.6 for ; Thu, 25 Apr 2013 11:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=6RujqdcKsCG2kK//nsD7ha8p6gsRTJe7sY3Fg3CkRfQ=; b=A/nn/8W3LOcKlFRytkrYVeGxeNLmFw54vwgYEBFtVs9eC/oViQkfxOhA9Co9aTYBc5 ue8P2MqPqOnp1b++BH3wViR1l2cjr7ehpMJmLmJWvSLzbS3l6PJMapf7OA8HPEJlqcDs TjL727vE0or4ZUYhVMuN//6waOnJLiOfNAd1WktRQqI/rnbGF4Xy4FRIdYq/Xxoyrxe3 5nR80mdlyPnEcET4bfH66YWfgmEFzZ4wYE4ZLkkPA8Dn+xYjdxHjuzhSug5KX5N5PQiH /3chKa7z27qr1h1lM1eeNqbnP6wDsMjhp66LhsLCXBUHcCmHKXNS/f364WTo8rrNNJZ6 AHMw== MIME-Version: 1.0 X-Received: by 10.194.93.133 with SMTP id cu5mr77829059wjb.56.1366914634787; Thu, 25 Apr 2013 11:30:34 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Thu, 25 Apr 2013 11:30:34 -0700 (PDT) In-Reply-To: <5178F72F.90008@freebsd.org> References: <5178F72F.90008@freebsd.org> Date: Thu, 25 Apr 2013 11:30:34 -0700 X-Google-Sender-Auth: SQY0pn_R5Fs2nhMgxiqXJSxXjZ0 Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current From: Adrian Chadd To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 Cc: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= , "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 18:30:36 -0000 On 25 April 2013 02:28, Andre Oppermann wrote: > Again one has to be really careful drawing any firm conclusions from this > as it was measured on a Pentium4 and UP kernel (GENERIC would add WITNESS > and INVARIANT overhead as well). > > The Pentium4 is about the worst micro-architecture when it comes to locks > and easily regresses. At the same time modern Intel Core i[3-7] and AMD64 > may actually improve with these changes. Unless more recent micro-archs > have been shown to exhibit the same regression we can't claim this change > was bad (other than for Pentium4). Sure, but he's done the heavy lifting. It'll be interesting to compare these results on a variety of platforms, not just the modern desktop/server style CPUs. Eg, if someone has the time, spinning this stuff up on the multi-core MIPS stuff in the netperf cluster (that's supposed to be a network forwarding engine) would be nice. And to be honest - having a set of performance checks for the same SVN revision but different physical machines is a good comparison point. It may be that we can start classifying different kinds of platform silliness from this which could lead to some better coding guidelines. Adrian From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 18:31:54 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5B0FC47C for ; Thu, 25 Apr 2013 18:31:54 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by mx1.freebsd.org (Postfix) with ESMTP id EC09C1305 for ; Thu, 25 Apr 2013 18:31:53 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id c10so9038061wiw.12 for ; Thu, 25 Apr 2013 11:31:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=+q6AQsSBbGYGsh3yFvs+o8uEflnQzQlb6r/YvNdITdk=; b=ELRC4BPxDO7BMODABk2M/tR1fpmNPg7/4eIrRlCdaKO1xblhj6Z3CszmhVp/YsJeTI T+5AN7cz3PvO0ZMsn2v4CgOQuN1Dlnb4UU+d4FIfphvrCDO41i7ux7wghd2oAdZgVJXi GbNKPUQWxp1juiOOb0zuuFTENbAGSwOtZCg4G93BwxxxVPI2KToNjgKkiDapB1BW0od4 4g//HA6AfdPFg6XlCcWIxLHolwVl0EZRcEl47tFD97SOszcr/uC8bX410gVM+wuTt9VR jKEBS12/Ru4kKxqTK8C3ymQvoLtSw1CMJwoV6ALjmAIDIw+qVDvMT8d3noHmRR5ayMGS Vj9g== MIME-Version: 1.0 X-Received: by 10.194.119.33 with SMTP id kr1mr77539141wjb.36.1366914713107; Thu, 25 Apr 2013 11:31:53 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Thu, 25 Apr 2013 11:31:53 -0700 (PDT) In-Reply-To: <517974DA.5090809@soe.ucsc.edu> References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> <517974DA.5090809@soe.ucsc.edu> Date: Thu, 25 Apr 2013 11:31:53 -0700 X-Google-Sender-Auth: 5gGKIqVLpXw_1Qy-NK0pfXIyRaY Message-ID: Subject: Re: pf performance? From: Adrian Chadd To: Erich Weiler Content-Type: text/plain; charset=ISO-8859-1 Cc: Kajetan Staszkiewicz , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 18:31:54 -0000 ... please ask the pfsense guys to either migrate to -9, or backport the -head pf (with the locking fixes!) to -8 for that. Otherwise you're very likely going to be wasting time on something you can't really push that much harder. ADrian On 25 April 2013 11:24, Erich Weiler wrote: >> As far as I understand, processing of packets by pf takes place in >> receiving >> network card's interrupt handler even up to sending the packet via another >> network card (at least in my case, when using route-to targets, which make >> routing inside pf). > > > That's interesting. So even though pf is giant locked, you can still scale > the maximum capacity of your firewall, in this case, simply by adding more > CPU cores? To handle the extra interrupts? So more cores = more packets > per second, if you give each extra core an additional interrupt queue? > > >> How do you count the 140kpps value? One interface, both, in, out? I'd like >> to >> relate this somehow to my values. > > > Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx > spike to 150kpps occasionally. This is a pfSense box, which includes RRD > graphs of packet rates, that's how I'm getting the number. I'm not sure how > they are obtaining that metric under the hood. But we have not disabled HT > and some other items, so that number will change is my guess. We also may > add another CPU die to the mix to see if we can add interrupt queues to more > cores to increase performance. > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 19:04:39 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 51CEAEE0 for ; Thu, 25 Apr 2013 19:04:39 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) by mx1.freebsd.org (Postfix) with ESMTP id 2899A14E8 for ; Thu, 25 Apr 2013 19:04:39 +0000 (UTC) Received: by mail-pd0-f177.google.com with SMTP id p11so1996806pdj.36 for ; Thu, 25 Apr 2013 12:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=GhI1+QeLL29KBJHu4mc+C6r6v5MC8Ef3smaU9XrJSm0=; b=Bt7lTZFQcKaTQv1VsXnNWMTTpdnycqA6ZYwZUif1z6MbeQXZkVADr22375+dDzmCGp wwSO02mmoqdJxwUd/R3lo23img154Z0eA4QS1pvKzPStvlCvW11QQ3srVyG1kuDPTmxu 9T854KP5kKgK/9yFzxLnKV5RaCAIay3QSvW/A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=GhI1+QeLL29KBJHu4mc+C6r6v5MC8Ef3smaU9XrJSm0=; b=ICaasKy4YrqIgAMKF87HlnbxBv3TluKAow8+p+t7diKWNMhU0vWnzwxtJUIEBziiV8 /pJml4aHyDHjhi3xvSJfgq16nSJ/KXvFGWR07/BLQyxNkg9JJxma9fT+S1ICzIA3Kxad 4LN3UIItdUtd8D/HBpbxgqpEUbii6piBA7hkcZY0BGoT4kg0Bp+XP8hCJxLwOwm1YdEF DfWayZRnT7i7Vu4YnSluzDrh54af0Xa5lUhHHa3CvEm9BphUgMJ6ygWRKHt0w/0t90Rj ItI5HNV9mupYEl0Bw7wAbGn7K4s3ZGOZ/E647kYhx+g1MyXdoYzJN643yqEDRklqiDNv ynNw== X-Received: by 10.66.145.134 with SMTP id su6mr26237489pab.198.1366916673840; Thu, 25 Apr 2013 12:04:33 -0700 (PDT) Received: from [172.30.0.50] (hgfw-01.soe.ucsc.edu. [128.114.61.130]) by mx.google.com with ESMTPSA id vv6sm9279258pab.6.2013.04.25.12.04.32 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 25 Apr 2013 12:04:32 -0700 (PDT) Message-ID: <51797E3F.1030400@soe.ucsc.edu> Date: Thu, 25 Apr 2013 12:04:31 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (X11; Linux i686; rv:10.0.6esrpre) Gecko/20120717 Thunderbird/10.0.6 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> <517974DA.5090809@soe.ucsc.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQnPV2RG4Oy78N5gSNiuMbZfj9pTOtP96+kKmAk5l2+4tywCfeNNaQGLpsbC4Oybc15uihic Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 19:04:39 -0000 > ... please ask the pfsense guys to either migrate to -9, or backport > the -head pf (with the locking fixes!) to -8 for that. > > Otherwise you're very likely going to be wasting time on something you > can't really push that much harder. I can ask for that (and will soon, likely), but to play with my current setup in the meantime, can we logically say that if I have 4 cores, and one interrupt queue is assigned to each core, and under I load I see each core (via "top -P") at 100% in interrupt usage, would it be safe to say that more cores (with additional interrupt queues accordingly) would mean more interrupts overall being processed, which would mean more pps? From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 20:04:16 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 56E95964 for ; Thu, 25 Apr 2013 20:04:16 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) by mx1.freebsd.org (Postfix) with ESMTP id E7C281755 for ; Thu, 25 Apr 2013 20:04:15 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id e11so1691952wgh.1 for ; Thu, 25 Apr 2013 13:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=W/3J8lk/bhvDq32ngsL7O9ItBKfF+k2cjb6v2nB7n5U=; b=VhG99kUfqQWZPuvNERj6p+WULmTP22yrlbGeUf2uMqOy22keTTWgASyoRktaKGfw6A B9dCUOIOMXopaN7TLhRFr2qiWMZmrpoUXHoHwuHHN9/UwtnPiGypA1CAs2QiVt8Mg5kt 5duZzgLWUgH3sN35xHsAt+JobdVyuBCyaWK3nKjT9nBkFpcWcVqRl/4ggOJoW+2UbgEX JwrdFop8c2l/VM8gxqsg0fF1Vypt5qNIjLcmurFn1bwxBskhthjweodpcM5WJhP65SsK tLC1ZKHw54mXbki+bHOXTWArqfEzC3z9XSVVLhBsToR7eB7j1CUAXU2P11UGI0d1O4Na GmBA== MIME-Version: 1.0 X-Received: by 10.180.149.200 with SMTP id uc8mr45798wib.3.1366920255030; Thu, 25 Apr 2013 13:04:15 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Thu, 25 Apr 2013 13:04:14 -0700 (PDT) In-Reply-To: <51797E3F.1030400@soe.ucsc.edu> References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> <517974DA.5090809@soe.ucsc.edu> <51797E3F.1030400@soe.ucsc.edu> Date: Thu, 25 Apr 2013 13:04:14 -0700 X-Google-Sender-Auth: vCbEIXRwvgiklEJopnW2h1Ly5xE Message-ID: Subject: Re: pf performance? From: Adrian Chadd To: Erich Weiler Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 20:04:16 -0000 If it contends on the global pf lock, you're short of luck. There may be some hack to enable in sysctl that defers part of the packet processing into a taskqueue, but I dont' know if that's for general IP processing or just socket iO processing. One of the network stack peeps will know. ADrian On 25 April 2013 12:04, Erich Weiler wrote: >> ... please ask the pfsense guys to either migrate to -9, or backport >> the -head pf (with the locking fixes!) to -8 for that. >> >> Otherwise you're very likely going to be wasting time on something you >> can't really push that much harder. > > > I can ask for that (and will soon, likely), but to play with my current > setup in the meantime, can we logically say that if I have 4 cores, and one > interrupt queue is assigned to each core, and under I load I see each core > (via "top -P") at 100% in interrupt usage, would it be safe to say that more > cores (with additional interrupt queues accordingly) would mean more > interrupts overall being processed, which would mean more pps? From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 21:48:37 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B160D16D for ; Thu, 25 Apr 2013 21:48:37 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) by mx1.freebsd.org (Postfix) with ESMTP id 8E53C1DFD for ; Thu, 25 Apr 2013 21:48:37 +0000 (UTC) Received: by mail-pd0-f177.google.com with SMTP id p11so2030229pdj.8 for ; Thu, 25 Apr 2013 14:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=rz4nfSo9nIax6srUnZB4e+jL6qfC1tGHaVKNxEtSUSA=; b=xTc+0lL/5DI8hukJw7io009D4KwGN08bte1n/o7++jLqWiJAe2FmA/1XKQ1lnXurFt Rbzl7PVLvTvna7Ryqr3BGgnCsLw2ZLjHWx049jJVKySDuilOonV6RgNZE1EKPqM9Kk+x IfzoAdt08BjrWKBpK6oIfWXcJeNEYmIUAgWIFp0CTirGKwgBJLyfyGJ7q6EOwwrBVVvt P6msccnIA7TPY1kuhthXqxoEt8WtzXcGcETU1+kqu5xKcwgdOy20I5HCQAKh4SHIbTOR IAzY9PPXuXskVJNLV6hv9tQ1DYsS2h0R/WhhJ7N1hdEzhaUq/Vyzg+CJWQUOvWjBMIQl 3qyw== MIME-Version: 1.0 X-Received: by 10.66.220.104 with SMTP id pv8mr26921061pac.72.1366926516981; Thu, 25 Apr 2013 14:48:36 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Thu, 25 Apr 2013 14:48:36 -0700 (PDT) In-Reply-To: References: Date: Fri, 26 Apr 2013 00:48:36 +0300 Message-ID: Subject: Re: using netmap From: Sami Halabi To: Andreas Nilsson Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , Luigi Rizzo X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 21:48:37 -0000 Okay, i figured out the includes, now it runs and seg faults: root@fbsd1:~/netmap # ./n Starting... em0 registered... sysctl passed mem mapped... nifp mapped...32766 rx ring def... rx ring start... Segmentation fault (core dumped) root@fbsd1:~/netmap # so apparently it faults at the initialization ring = NETMAP_RXRING(nifp, 0); any ideas? here is the new code: int main() { struct netmap_if *nifp = NULL; struct nmreq req; int i, len, fd; char *buf, *mem, *txt; printf("Starting...\n"); fd = open("/dev/netmap", 0); strcpy(req.nr_name, "em0"); // register the interface printf("em0 registered...\n"); ioctl(fd, NIOCREGIF, &req); // offset of the structure printf("sysctl passed\n"); mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); printf("mem mapped...\n"); nifp = NETMAP_IF(mem, req.nr_offset); printf("nifp mapped...%u\n",(long)nifp); for (;;) { struct pollfd x[1]; printf("rx ring def...\n"); struct netmap_ring *ring; printf("rx ring start...\n"); ring = NETMAP_RXRING(nifp, 0); printf("rx ring polled...\n"); x[0].fd = fd; x[0].events = POLLIN; poll(x, 1, 1000); for ( ; ring->avail > 0 ; ring->avail--) { i = ring->cur; printf("i=%d\n",&i); buf = NETMAP_BUF(ring, i); printf("buff read...\n"); //use_data(buf, ring->slot[i].len); txt = malloc(sizeof(char) * ring->slot[i].len +1); strncpy(txt,buf,ring->slot[i].len); txt[ring->slot[i].len]='\0'; printf("len is: %d\n",&ring->slot[i].len); ring->cur = NETMAP_RING_NEXT(ring, i); } } return 0; } On Mon, Apr 15, 2013 at 9:15 PM, Andreas Nilsson wrote: > > > > On Mon, Apr 15, 2013 at 7:52 PM, Sami Halabi wrote: > >> Hi, >> I would like to start using netmap. >> >> as a start i copied the example from netmap >> page: >> >> #include >> #include >> #include >> #include >> >> int main() { >> >> struct netmap_if *nifp; >> struct nmreq req; >> int i, len; >> char *buf; >> FILE* fd; >> >> fd = open("/dev/netmap", 0); >> strcpy(req.nr_name, "em1"); // register the interface >> ioctl(fd, NIOCREG, &req); // offset of the structure >> mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); >> nifp = NETMAP_IF(mem, req.nr_offset); >> for (;;) { >> struct pollfd x[1]; >> struct netmap_ring *ring = NETMAP_RX_RING(nifp, 0); >> >> x[0].fd = fd; >> x[0].events = POLLIN; >> poll(x, 1, 1000); >> for ( ; ring->avail > 0 ; ring->avail--) { >> i = ring->cur; >> buf = NETMAP_BUF(ring, i); >> use_data(buf, ring->slot[i].len); >> ring->cur = NETMAP_NEXT(ring, i); >> } >> } >> >> return 0; >> } >> >> >> and tried to compile: >> root@fbsd1:~/netmap # gcc -O2 -pipe -Werror -Wall -I ../sys -Wextra -c >> n.c >> In file included from n.c:4: >> /usr/include/net/netmap.h:139: error: expected specifier-qualifier-list >> before 'uint32_t' >> /usr/include/net/netmap.h:228: error: expected ':', ',', ';', '}' or >> '__attribute__' before 'num_slots' >> /usr/include/net/netmap.h:255: error: 'IFNAMSIZ' undeclared here (not in a >> function) >> /usr/include/net/netmap.h:256: error: expected ':', ',', ';', '}' or >> '__attribute__' before 'ni_version' >> /usr/include/net/netmap.h:292: error: expected specifier-qualifier-list >> before 'uint32_t' >> cc1: warnings being treated as errors >> n.c: In function 'main': >> n.c:14: warning: implicit declaration of function 'open' >> n.c:14: warning: assignment makes pointer from integer without a cast >> n.c:15: warning: implicit declaration of function 'strcpy' >> n.c:15: warning: incompatible implicit declaration of built-in function >> 'strcpy' >> n.c:16: warning: implicit declaration of function 'ioctl' >> n.c:16: error: 'NIOCREG' undeclared (first use in this function) >> n.c:16: error: (Each undeclared identifier is reported only once >> n.c:16: error: for each function it appears in.) >> n.c:17: error: 'mem' undeclared (first use in this function) >> n.c:17: error: 'struct nmreq' has no member named 'nr_memsize' >> n.c:17: error: 'PROT_READ' undeclared (first use in this function) >> n.c:17: error: 'PROT_WRITE' undeclared (first use in this function) >> n.c:17: warning: passing argument 5 of 'mmap' makes integer from pointer >> without a cast >> n.c:18: error: 'struct nmreq' has no member named 'nr_offset' >> n.c:20: error: array type has incomplete element type >> n.c:21: warning: implicit declaration of function 'NETMAP_RX_RING' >> n.c:21: warning: initialization makes pointer from integer without a cast >> n.c:24: error: 'POLLIN' undeclared (first use in this function) >> n.c:25: warning: implicit declaration of function 'poll' >> n.c:26: error: 'struct netmap_ring' has no member named 'avail' >> n.c:26: error: 'struct netmap_ring' has no member named 'avail' >> n.c:27: error: 'struct netmap_ring' has no member named 'cur' >> n.c:28: error: 'struct netmap_ring' has no member named 'nr_buf_size' >> n.c:29: warning: implicit declaration of function 'use_data' >> n.c:29: error: 'struct netmap_ring' has no member named 'slot' >> n.c:30: error: 'struct netmap_ring' has no member named 'cur' >> n.c:30: warning: implicit declaration of function 'NETMAP_NEXT' >> n.c:20: warning: unused variable 'x' >> n.c:10: warning: unused variable 'len' >> root@fbsd1:~/netmap # >> >> >> can you help me figure it out? >> >> Thanks in advance, >> >> -- >> Sami Halabi >> Information Systems Engineer >> NMS Projects Expert >> FreeBSD SysAdmin Expert >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > Hello Sami, > > First, I'm by no means an expert on NETMAP, but the url you provided says > that it's a snapshot, not a ready progam. There are a few fully working > examples in tools/tools/netmap/ subdir of at least 9-STABLE and 10-CURRENT. > Maybe you can find an example to start with there instead? > > Quite a few of the errors are from regular syscalls, and each of them have > a manpage that says which files to include, and compiling stuff with > -Werror assumes you can deal with "picky" compilers. > > Hope you come up with a working example. > > Best regards > Andreas > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 22:21:14 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 89D69A97 for ; Thu, 25 Apr 2013 22:21:14 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x234.google.com (mail-bk0-x234.google.com [IPv6:2a00:1450:4008:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id 1E17B1F6E for ; Thu, 25 Apr 2013 22:21:13 +0000 (UTC) Received: by mail-bk0-f52.google.com with SMTP id je9so566198bkc.11 for ; Thu, 25 Apr 2013 15:21:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=yfV7qOpEJpUSiqkrguJ+7OH+xAqQh5eA1CFy/G/Zk3E=; b=QxiQFqiVFpoJgcyhk+/9O3A9Xk9YWzMcBdqK8KToEQ/TevSgsHrEpzSQS324rUvQfx 4cAfSNI8GuuEvPpOe/rzYE+YMpU0lMOnmctBNXqJ6WmyZIKYYOmZ2Fq1fyV/g0wn0B2q qLNzD8gG/cdnvmj5tstV5c2yeAJ8ObCieOSJVaU17bWsYX4Upy8atlCMuOI7DdtmvWlY OOhDAjlSp8LA3ySF73H1UGbiWiunOd53OCXeGTq3B4qRw+0AIfoT9C2u2ak1PZjS8W50 vP4YeNDdDOPB81TIAXWlN9HTM/oHDgiHMM4FX0gUOkcr2Q4NvqCQzip0K53Tzq1PV3Mi enDQ== X-Received: by 10.205.45.130 with SMTP id uk2mr10484934bkb.68.1366928472880; Thu, 25 Apr 2013 15:21:12 -0700 (PDT) Received: from zvezda.localnet ([2a02:8108:1440:5b:2677:3ff:fe7b:7648]) by mx.google.com with ESMTPSA id m11sm2308623bkz.0.2013.04.25.15.21.11 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 25 Apr 2013 15:21:12 -0700 (PDT) From: Kajetan Staszkiewicz To: Erich Weiler Subject: Re: pf performance? Date: Fri, 26 Apr 2013 00:21:11 +0200 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> <517974DA.5090809@soe.ucsc.edu> In-Reply-To: <517974DA.5090809@soe.ucsc.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201304260021.11209.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQkoWc/Nvz7N2T+/3bES7ir0i78Gv26mjSnqJhLsP9+6b03h6OC3T139w8vp3RAiMS2hinuu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 22:21:14 -0000 Dnia czwartek, 25 kwietnia 2013 o 20:24:26 Erich Weiler napisa=C5=82(a): > > As far as I understand, processing of packets by pf takes place in > > receiving network card's interrupt handler even up to sending the packet > > via another network card (at least in my case, when using route-to > > targets, which make routing inside pf). >=20 > That's interesting. So even though pf is giant locked, you can still > scale the maximum capacity of your firewall, in this case, simply by > adding more CPU cores? To handle the extra interrupts? So more cores =3D > more packets per second, if you give each extra core an additional > interrupt queue? There is still some code outside pf that packets from the network pass thro= ugh. =20 > > How do you count the 140kpps value? One interface, both, in, out? I'd > > like to relate this somehow to my values. >=20 > Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx > spike to 150kpps occasionally. Unfortunately at this moment I have no single machine with such traffic,=20 although maybe I can aggregate some traffic later and check the cpu usage t= hen. > This is a pfSense box, which includes > RRD graphs of packet rates, that's how I'm getting the number. I'm not > sure how they are obtaining that metric under the hood. But we have not > disabled HT and some other items, so that number will change is my > guess. We also may add another CPU die to the mix to see if we can add > interrupt queues to more cores to increase performance. How many pf rules do you have?. And, as I asked in my previous post, do you= =20 create states on both sides of the firewall? =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Thu Apr 25 22:52:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C6088D1E for ; Thu, 25 Apr 2013 22:52:53 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) by mx1.freebsd.org (Postfix) with ESMTP id 9C82C103B for ; Thu, 25 Apr 2013 22:52:53 +0000 (UTC) Received: by mail-pd0-f171.google.com with SMTP id t12so2090555pdi.30 for ; Thu, 25 Apr 2013 15:52:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=QTZnSsaVRRp6dtAgVCd8J3SUQ5wGdHImF42B7FyYlm4=; b=WDD/SZUM157AwwiETXEjKYDC2geOnZCtmetjPirxtT4UlMUxeJFQZRQFDfCj+PCecG eS0KfVQDTCaaAx/lXb04WLUVeL8AkmLmmHpwNyUf3m0H0USiJfaXiI0OKXqx5RaEgK4x iTFqaSrT8UfyR9XhtTQ25IP1qIeC+g1NgfuDo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=QTZnSsaVRRp6dtAgVCd8J3SUQ5wGdHImF42B7FyYlm4=; b=CiELl+TqVK9mFL6mDwG0boNeuwm8xq31eOPlYD0RypxTXqor6M32V7c1ebUXsUZ41+ 8lf9O6sZzaarcGi6Kyns1OSfp7Wz2vbcajq8I0hFmeehUGvOk93XhcmOeFt+XXup+COE 25+OvnDEG9n9OYCfDvhsLmd5qCE3Pr0yjKEkBOceau9KVwtIjqkIsaiOLPyZEG0kaIC5 PmtFIzDVpRF0t5lLieO11VE8Fn+RnZ6zylrnDj/JwVdKhKUAFkzLXJ5ot6aHmJI/gZQb HWgoaFh7trXDl6gkHpKXNj3+pU8kXMs8BAWrZi8vbn08xhZxN0vf+KYgQwI9//XDOJ+M PG/A== X-Received: by 10.68.226.230 with SMTP id rv6mr17527019pbc.55.1366930366903; Thu, 25 Apr 2013 15:52:46 -0700 (PDT) Received: from [172.30.0.50] (hgfw-01.soe.ucsc.edu. [128.114.61.130]) by mx.google.com with ESMTPSA id ih1sm8900755pbb.44.2013.04.25.15.52.44 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 25 Apr 2013 15:52:45 -0700 (PDT) Message-ID: <5179B3BB.3070101@soe.ucsc.edu> Date: Thu, 25 Apr 2013 15:52:43 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (X11; Linux i686; rv:10.0.6esrpre) Gecko/20120717 Thunderbird/10.0.6 MIME-Version: 1.0 To: Kajetan Staszkiewicz Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <201304240134.22740.vegeta@tuxpowered.net> <517974DA.5090809@soe.ucsc.edu> <201304260021.11209.vegeta@tuxpowered.net> In-Reply-To: <201304260021.11209.vegeta@tuxpowered.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQl9GWLvMtIFTd7TsbfVRDG6KFdBkTcABHxeJSVLLfAp1miILAfvY6sxX4egh1PpYg8IECrB Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Apr 2013 22:52:53 -0000 > How many pf rules do you have?. And, as I asked in my previous post, do you > create states on both sides of the firewall? One interface has 12 rules and other other interface has one rule. We do create states on both sides. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 03:43:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 298B9547 for ; Fri, 26 Apr 2013 03:43:35 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by mx1.freebsd.org (Postfix) with ESMTP id 015621ABB for ; Fri, 26 Apr 2013 03:43:34 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id bg4so2215379pad.23 for ; Thu, 25 Apr 2013 20:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=ab0qoIerzi4rdM6POadH9BpDS7+FVuWpGQN/ER6Qn4c=; b=V8ZPMWd3DX5Sa6kao/TePNTj+sGZa3vJmHQQtj9u1pEYPnbJ1HZkU2WwNYhkoBlLzd 7ZovY6eyO0fSqy2B+LujhgAg5oF6AR9gP79ERcjtu42a/1Gn1gW/RhSTBIO5yw5eX6n0 aNbm3K2eTdPKWMOcgbr7LagSzjhuPyz39/nBg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=ab0qoIerzi4rdM6POadH9BpDS7+FVuWpGQN/ER6Qn4c=; b=Ap0jlvXklhqnVms9YIvC6ZZvhEi0xVCB1yXGzw4sVKC9eaXA6MubvbWiIYIHDb5p8/ +abRNiEkR2AI8CQ2AccuIrCD9ZECgeYiRllNu/2XhOF+TMEcnPm45Zwaw5Izl42z9t4U C+lGujN338k1Fqb3KSICbJ7l2dO1IS8fXGrdJ2iOMdhKyew5lrGXz4nE8Rw4dlp5SZVz 78zVHIV3znJAhT5gexxBGGD5Lb1smQtwsZ5yDnFlnx8fi2DOVts5hDBrg7fEJscWy7zf 79DLnGrTGa4XW3EMiETeXVN/ezqRpsrsXxBUcgoFuzlASBwXk9UhfaXjH/SG04M7/m5o P5zA== X-Received: by 10.68.90.36 with SMTP id bt4mr56275532pbb.42.1366947814311; Thu, 25 Apr 2013 20:43:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.159.97 with HTTP; Thu, 25 Apr 2013 20:43:04 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Thu, 25 Apr 2013 23:43:04 -0400 Message-ID: Subject: Re: using netmap To: Sami Halabi Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlsSjlknm4lInBDTp2cG6BWzDayrMRZfgLcK3gjLJweMI4YEYC0IGlLY9QnijrTd/Gvnoek Cc: "freebsd-net@freebsd.org" , Luigi Rizzo , Andreas Nilsson X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 03:43:35 -0000 [ please bottom post or reply inline ] On 25 April 2013 17:48, Sami Halabi wrote: > Okay, > i figured out the includes, now it runs and seg faults: Don't forget to show the working headers ;) > any ideas? > > here is the new code: > int main() { > > struct netmap_if *nifp = NULL; > struct nmreq req; > int i, len, fd; > char *buf, *mem, *txt; > > printf("Starting...\n"); > fd = open("/dev/netmap", 0); > strcpy(req.nr_name, "em0"); // register the interface > printf("em0 registered...\n"); > ioctl(fd, NIOCREGIF, &req); // offset of the structure is req fully initialized? I don't think this ioctl is correct. Everything goes wrong after this as a result. > printf("sysctl passed\n"); Things will not always crash if you make a wrong call. Try checking return codes before printing something like this. > mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); > printf("mem mapped...\n"); > > nifp = NETMAP_IF(mem, req.nr_offset); > printf("nifp mapped...%u\n",(long)nifp); this seems wrong: ^^ ^^^^^^ > for (;;) { > struct pollfd x[1]; > printf("rx ring def...\n"); > struct netmap_ring *ring; > printf("rx ring start...\n"); > ring = NETMAP_RXRING(nifp, 0); > printf("rx ring polled...\n"); > > x[0].fd = fd; > x[0].events = POLLIN; > poll(x, 1, 1000); > for ( ; ring->avail > 0 ; ring->avail--) { > i = ring->cur; > printf("i=%d\n",&i); I think this is wrong. > buf = NETMAP_BUF(ring, i); > printf("buff read...\n"); > //use_data(buf, ring->slot[i].len); > txt = malloc(sizeof(char) * ring->slot[i].len +1); > strncpy(txt,buf,ring->slot[i].len); > txt[ring->slot[i].len]='\0'; > printf("len is: %d\n",&ring->slot[i].len); Also this. > ring->cur = NETMAP_RING_NEXT(ring, i); > } > } > > return 0; > } -- Eitan Adler From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 06:23:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B47B7598 for ; Fri, 26 Apr 2013 06:23:35 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-pb0-x236.google.com (mail-pb0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) by mx1.freebsd.org (Postfix) with ESMTP id 8EEC51FF6 for ; Fri, 26 Apr 2013 06:23:35 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id jt11so1042140pbb.27 for ; Thu, 25 Apr 2013 23:23:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=68bmtdoU/RknTvJ6iGJweyHG2GYsEbFVq6X1zZ++TSM=; b=nyc4za9ut0eGniOkFG5vB1uVvLiA5+AfDI3/6I9IQ0iCurnHO9fz8mrQNBjf/HSAXo srY94urxafEi5IxMI2cBk115HSHGAHHdV5Nu8141+C4acORC9SG8REElCpF5wViVSnEa vjaHSJ5+VTQSLPsYo979osLl8wyYP+iF+C6nYkudKbWwsf/Y7cSm3Lnb7VKlIyJquxly egDIRT67Xvaud1tm/b5dzEp8hB0wnGXhaRNsKZxqSLb/Qg0S7vrgpNsoZoECtjRtopL1 fV0Q/b6fd6UALNTPmtR3MUAiJoW5FP/AuSMi4jlAXYMi9sKHh+5o17VpLKyLf2Gun9kP AgGw== MIME-Version: 1.0 X-Received: by 10.66.161.33 with SMTP id xp1mr29517398pab.36.1366957415281; Thu, 25 Apr 2013 23:23:35 -0700 (PDT) Received: by 10.70.100.132 with HTTP; Thu, 25 Apr 2013 23:23:35 -0700 (PDT) In-Reply-To: References: Date: Fri, 26 Apr 2013 09:23:35 +0300 Message-ID: Subject: Re: using netmap From: Sami Halabi To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , Luigi Rizzo , Andreas Nilsson X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 06:23:35 -0000 Hi Eitan, Thank your for your response. the ioctl is the example was in Luigi netmap page... maybe Luigi can help here??????? can you say why the print's are wrong? i fetched wrking headers from other tools without too much checking, maybe some are irrelevant but for my tests i didn't worry about it yet, so here is it: =================================================== #include #include /* signal */ #include #include #include /* PRI* macros */ #include /* strcmp */ #include /* open */ #include /* close */ #include /* getifaddrs */ #include /* PROT_* */ #include /* ioctl */ #include #include /* sockaddr.. */ #include /* ntohs */ #include #include /* sysctl */ #include /* timersub */ #include #include /* ifreq */ #include #include #include #include #include #ifndef MY_PCAP /* use the system's pcap if available */ #ifdef NO_PCAP #define PCAP_ERRBUF_SIZE 512 typedef void pcap_t; struct pcap_pkthdr; #define pcap_inject(a,b,c) ((void)a, (void)b, (void)c, -1) #define pcap_dispatch(a, b, c, d) (void)c #define pcap_open_live(a, b, c, d, e) ((void)e, NULL) #else /* !NO_PCAP */ #include // XXX do we need it ? #endif /* !NO_PCAP */ #endif // XXX hack #include /* pthread_* */ #include /* le64toh */ #include #include /* pthread w/ affinity */ #include /* cpu_set */ #include /* LLADDR */ =================================================== Thanks in advance, Sami On Fri, Apr 26, 2013 at 6:43 AM, Eitan Adler wrote: > [ please bottom post or reply inline ] > > On 25 April 2013 17:48, Sami Halabi wrote: > > Okay, > > i figured out the includes, now it runs and seg faults: > > Don't forget to show the working headers ;) > > > any ideas? > > > > here is the new code: > > int main() { > > > > struct netmap_if *nifp = NULL; > > struct nmreq req; > > int i, len, fd; > > char *buf, *mem, *txt; > > > > printf("Starting...\n"); > > fd = open("/dev/netmap", 0); > > strcpy(req.nr_name, "em0"); // register the interface > > printf("em0 registered...\n"); > > ioctl(fd, NIOCREGIF, &req); // offset of the structure > > is req fully initialized? > > I don't think this ioctl is correct. Everything goes wrong after this > as a result. > > > printf("sysctl passed\n"); > > Things will not always crash if you make a wrong call. Try checking > return codes before printing something like this. > > > mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); > > printf("mem mapped...\n"); > > > > nifp = NETMAP_IF(mem, req.nr_offset); > > printf("nifp mapped...%u\n",(long)nifp); > > this seems wrong: ^^ ^^^^^^ > > > > for (;;) { > > struct pollfd x[1]; > > printf("rx ring def...\n"); > > struct netmap_ring *ring; > > printf("rx ring start...\n"); > > ring = NETMAP_RXRING(nifp, 0); > > printf("rx ring polled...\n"); > > > > x[0].fd = fd; > > x[0].events = POLLIN; > > poll(x, 1, 1000); > > for ( ; ring->avail > 0 ; ring->avail--) { > > i = ring->cur; > > printf("i=%d\n",&i); > > I think this is wrong. > > > buf = NETMAP_BUF(ring, i); > > printf("buff read...\n"); > > //use_data(buf, ring->slot[i].len); > > txt = malloc(sizeof(char) * ring->slot[i].len +1); > > strncpy(txt,buf,ring->slot[i].len); > > txt[ring->slot[i].len]='\0'; > > printf("len is: %d\n",&ring->slot[i].len); > > Also this. > > > ring->cur = NETMAP_RING_NEXT(ring, i); > > } > > } > > > > return 0; > > } > > > > -- > Eitan Adler > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 08:13:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CBE94CE7 for ; Fri, 26 Apr 2013 08:13:01 +0000 (UTC) (envelope-from steve.read@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 959BB14F6 for ; Fri, 26 Apr 2013 08:13:01 +0000 (UTC) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 43B6B27063A8 for ; Fri, 26 Apr 2013 10:07:25 +0200 (CEST) Received: from stever.netasq.com (unknown [10.2.0.1]) by work.netasq.com (Postfix) with ESMTPA id 0760327045DF for ; Fri, 26 Apr 2013 10:07:25 +0200 (CEST) Message-ID: <517A35BC.4060305@netasq.com> Date: Fri, 26 Apr 2013 10:07:24 +0200 From: Steve Read User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130305 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: using netmap References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 08:13:01 -0000 On 26.04.2013 08:23, Sami Halabi wrote: > Hi Eitan, > Thank your for your response. > the ioctl is the example was in Luigi netmap page... maybe Luigi can help > here??????? > > can you say why the print's are wrong? They print the addresses of the variables, not their values. int i = 1234; printf("i=%d\n", &i); /* Note the &, prints the address of i, or not, at its whim */ printf("i=%d\n", i); /* Note no &, prints the value of i */ I say "at its whim" because once you have a mismatch between the type requested (%d requests int) and the type provided (&i provides int *), you are in the terrain of undefined behaviour, and **ANYTHING** can happen. Be glad you aren't running on the notoriously twitchy DeathStation 9000. -- Steve From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 08:23:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A12128DD for ; Fri, 26 Apr 2013 08:23:09 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 429FE16F1 for ; Fri, 26 Apr 2013 08:23:09 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id E55C87300B; Fri, 26 Apr 2013 10:24:57 +0200 (CEST) Date: Fri, 26 Apr 2013 10:24:57 +0200 From: Luigi Rizzo To: Sami Halabi Subject: Re: using netmap Message-ID: <20130426082457.GA33737@onelab2.iet.unipi.it> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Eitan Adler , Andreas Nilsson , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 08:23:09 -0000 On Fri, Apr 26, 2013 at 09:23:35AM +0300, Sami Halabi wrote: > Hi Eitan, > Thank your for your response. > the ioctl is the example was in Luigi netmap page... maybe Luigi can help > here??????? the thing i suggest is take the pkt-gen source from the FreeBSD tree tools/tools/netmap/ and start modifying that one. The samples on my web page are 2 years old, and probably wrong in various ways even then. They were meant to be something to put on a slide, so a lot of details were missing. pkt-gen.c, on the other hand, is supposed to build and be usable. As others pointed out, the snippets of code you posted have a lot of very basic programming errors which suggest that the problems you are seeing are not related to netmap. Of course we may be wrong, but the odds are against you, so it is better to start from something known-working. Try to build your code with "cc -O2 -Wall -Werror" so the compiler will find at least the most egregious bugs and abort the compilation until you fix them. cheers luigi > can you say why the print's are wrong? > > > i fetched wrking headers from other tools without too much checking, maybe > some > are irrelevant but for my tests i didn't worry about it yet, so here is it: > =================================================== > #include > #include /* signal */ > #include > #include > #include /* PRI* macros */ > #include /* strcmp */ > #include /* open */ > #include /* close */ > #include /* getifaddrs */ > > #include /* PROT_* */ > #include /* ioctl */ > #include > #include /* sockaddr.. */ > #include /* ntohs */ > #include > #include /* sysctl */ > #include /* timersub */ > > #include > #include /* ifreq */ > > #include > #include > #include > > #include > #include > > #ifndef MY_PCAP /* use the system's pcap if available */ > > #ifdef NO_PCAP > #define PCAP_ERRBUF_SIZE 512 > typedef void pcap_t; > struct pcap_pkthdr; > #define pcap_inject(a,b,c) ((void)a, (void)b, (void)c, -1) > #define pcap_dispatch(a, b, c, d) (void)c > #define pcap_open_live(a, b, c, d, e) ((void)e, NULL) > #else /* !NO_PCAP */ > #include // XXX do we need it ? > #endif /* !NO_PCAP */ > > #endif // XXX hack > > #include /* pthread_* */ > #include /* le64toh */ > #include > > #include /* pthread w/ affinity */ > #include /* cpu_set */ > #include /* LLADDR */ > =================================================== > > > Thanks in advance, > Sami > > > On Fri, Apr 26, 2013 at 6:43 AM, Eitan Adler wrote: > > > [ please bottom post or reply inline ] > > > > On 25 April 2013 17:48, Sami Halabi wrote: > > > Okay, > > > i figured out the includes, now it runs and seg faults: > > > > Don't forget to show the working headers ;) > > > > > any ideas? > > > > > > here is the new code: > > > int main() { > > > > > > struct netmap_if *nifp = NULL; > > > struct nmreq req; > > > int i, len, fd; > > > char *buf, *mem, *txt; > > > > > > printf("Starting...\n"); > > > fd = open("/dev/netmap", 0); > > > strcpy(req.nr_name, "em0"); // register the interface > > > printf("em0 registered...\n"); > > > ioctl(fd, NIOCREGIF, &req); // offset of the structure > > > > is req fully initialized? > > > > I don't think this ioctl is correct. Everything goes wrong after this > > as a result. > > > > > printf("sysctl passed\n"); > > > > Things will not always crash if you make a wrong call. Try checking > > return codes before printing something like this. > > > > > mem = mmap(NULL, req.nr_memsize, PROT_READ|PROT_WRITE, 0, fd, 0); > > > printf("mem mapped...\n"); > > > > > > nifp = NETMAP_IF(mem, req.nr_offset); > > > printf("nifp mapped...%u\n",(long)nifp); > > > > this seems wrong: ^^ ^^^^^^ > > > > > > > for (;;) { > > > struct pollfd x[1]; > > > printf("rx ring def...\n"); > > > struct netmap_ring *ring; > > > printf("rx ring start...\n"); > > > ring = NETMAP_RXRING(nifp, 0); > > > printf("rx ring polled...\n"); > > > > > > x[0].fd = fd; > > > x[0].events = POLLIN; > > > poll(x, 1, 1000); > > > for ( ; ring->avail > 0 ; ring->avail--) { > > > i = ring->cur; > > > printf("i=%d\n",&i); > > > > I think this is wrong. > > > > > buf = NETMAP_BUF(ring, i); > > > printf("buff read...\n"); > > > //use_data(buf, ring->slot[i].len); > > > txt = malloc(sizeof(char) * ring->slot[i].len +1); > > > strncpy(txt,buf,ring->slot[i].len); > > > txt[ring->slot[i].len]='\0'; > > > printf("len is: %d\n",&ring->slot[i].len); > > > > Also this. > > > > > ring->cur = NETMAP_RING_NEXT(ring, i); > > > } > > > } > > > > > > return 0; > > > } > > > > > > > > -- > > Eitan Adler > > > > > > -- > Sami Halabi > Information Systems Engineer > NMS Projects Expert > FreeBSD SysAdmin Expert > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 08:26:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 42CF2BB4 for ; Fri, 26 Apr 2013 08:26:12 +0000 (UTC) (envelope-from bredehorn@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mx1.freebsd.org (Postfix) with ESMTP id D86F81755 for ; Fri, 26 Apr 2013 08:26:11 +0000 (UTC) Received: from 3capp-gmx-bs46.server.lan ([172.19.170.98]) by mrigmx.server.lan (mrigmx002) with ESMTP (Nemesis) id 0M1TnD-1UlW8p0p45-00tV8S for ; Fri, 26 Apr 2013 10:26:10 +0200 Received: from [93.159.253.121] by 3capp-gmx-bs46.server.lan with HTTP; Fri Apr 26 10:26:10 CEST 2013 MIME-Version: 1.0 Message-ID: From: "Rainer Bredehorn" To: "net FreeBSD" Subject: Aw: PF IPv6 fragment support Content-Type: text/plain; charset=UTF-8 Date: Fri, 26 Apr 2013 10:26:10 +0200 (CEST) Importance: normal Sensitivity: Normal In-Reply-To: References: X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K0:80w6oosKg79KW+NQ0WvCDbK+FwoQ8960kOM01JBBJ2u I0VubBMuZFGxrnF3g7DBlXYCrFB5XD+0V3U8iQItOPzS1iQzZN VyIjt24J0nxq/6giVcE4Mpj05/lXpfOzgLmkZRTv1/8fb9iN3G pgnkhX20GN1/dWdPbXkJhRqYLBTlONy9H0ftYoyj0d8rxMY4fH vCPkAVcX/fyJOArWaYt1IBU2LhcGMdzsNYqCrvRGFvcDBXtfxC QD05a7BAQAy9aENHCT2CuDEST/w5kdVHci80Fzg5cdtdkzLXlC Ubi8VE= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 08:26:12 -0000 > I'm using FreeBSD 8.3 which doesn't support IPv6 fragments in PF. > Does FreeBSD 9.x PF support IPv6 fragments? > I can't find it in the 9.0 or 9.1 manpages. For pf.conf they are the same as in FreeBSD 8.3. I've modified the kernel PF implementation to pass IPv6 fragments. The first fragment is handled by the PF rules of course ignoring possible checksums. All other fragments are passed by PF to the IP stack. This can be done state-full but reassembling fragments is not supported. That's what I wanted. Rainer. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 09:37:39 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DE92688B; Fri, 26 Apr 2013 09:37:39 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 8E8FC1CF0; Fri, 26 Apr 2013 09:37:39 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id j3so1982395qcs.20 for ; Fri, 26 Apr 2013 02:37:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=0ScdP2JZZnHoojBh6ImIlqIdMlnK/sL/cPVa+UNa6V4=; b=tGKCeQ5Uc79LPfSP0nAXgA0SbxfSuMph0UWQm8gMwS0lhfDveszXvU6S3NOdOGfHuy z3ObRXO6y0M7t69aS5ZyJ5HtGYSklkvlt4c/1tYzRf214cuCV5Ri/3+26v+kb7Ci/RWV lPLxbH+sgOk8N2KV+cZH5JaSp56tPQABSB3wuN11YCOO4MCCpMFRhoXwdacYyU9KUtLX qdOUWzxeSoz2eLawrqpyNtxDAKJSj142y0TRnV+z3Jx+Su3dI/4REVICcQ9k40eUQhCJ pddKFs7mKBoaCtHTkiJDj3zihIWug4rMMBGzal/8WHMZFZJiJ//lQK8Pwnp1+e82Giwa e79Q== MIME-Version: 1.0 X-Received: by 10.224.65.1 with SMTP id g1mr13611412qai.64.1366969059130; Fri, 26 Apr 2013 02:37:39 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.49.49.37 with HTTP; Fri, 26 Apr 2013 02:37:39 -0700 (PDT) In-Reply-To: References: Date: Fri, 26 Apr 2013 11:37:39 +0200 X-Google-Sender-Auth: sSoHjT_b1hHsDEQ7LQuHNjkFot0 Message-ID: Subject: Re: forwarding/ipfw/pf evolution (in pps) on -current From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" , Sami Halabi X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 09:37:39 -0000 Hello, would you mind running a performance test with a snapshot of tomorrow from this link http://snapshots.pfsense.org/ There are some optimizations in pfSense and it would be nicer to compare to FreeBSD itself how it behaves. That is before the lock changes in HEAD since its FreeBSD 8. Regards, Ermal On Thu, Apr 25, 2013 at 7:40 AM, Olivier Cochard-Labb=E9 wrote: > On Wed, Apr 24, 2013 at 1:46 PM, Sami Halabi wrote: > > 3. there some point of improved performance (without fw) that went down > > again somewhere before Clang got prod. > > Found it ! > > It's commit 242402: "Rework the known mutexes..." > > ministat -s 242401.forwarding 242402.forwarding > x 242401.forwarding > + 242402.forwarding > > +------------------------------------------------------------------------= ---------------+ > | + > | > |+ + + + > x xx x x| > | > |____A____| | > | |_____A_M___| > | > > +------------------------------------------------------------------------= ---------------+ > N Min Max Median Avg Stdd= ev > x 5 417527 420242 418902 419074 1049.79= 74 > + 5 402211 404828 404096 403689 1237.66= 96 > Difference at 95.0% confidence > -15385 +/- 1673.69 > -3.67119% +/- 0.399377% > (Student's t, pooled s =3D 1147.58) > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 Ermal From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 11:30:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 699C413A for ; Fri, 26 Apr 2013 11:30:45 +0000 (UTC) (envelope-from nodens2099@gmail.com) Received: from mail-ee0-f44.google.com (mail-ee0-f44.google.com [74.125.83.44]) by mx1.freebsd.org (Postfix) with ESMTP id 019E1124B for ; Fri, 26 Apr 2013 11:30:44 +0000 (UTC) Received: by mail-ee0-f44.google.com with SMTP id t10so1715074eei.31 for ; Fri, 26 Apr 2013 04:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=wtlZWnb38LqOwMIe7txZoPpTql/ZA8/ULpUKWLOKIDY=; b=c96cYkubqxvwcxx3bkzRem8TS37KPQh24JRo4F2z0L7K5H9MTn1OmIFVtlZgDTcvnQ 7iqFWGMWU/AdRf8aw66Jh85EMATvRWdSnRGDJdEoCZ1k2QLIUCHpxh6lwyoP/fgQ9Kha 3j2dG4SlykYSVDrWejXoxavEFvj7JkUnCcJ5xzaaVvPdRquIOURmz1QfufyvnUJWfYNu GCjga/V32RQZAAyhz6YmzIQYdnZs57ZGUciRBbPYbaQ/gzydC7fAJ7R7++tyI6vkpqKL BDMkkd6jtqtxk2Ors9EFlfEMHT/je0+mCtn0XyDkZsj4NoxYT9KEi2tCuvIL9Zsstv3L yqWw== X-Received: by 10.14.115.131 with SMTP id e3mr12529280eeh.43.1366975837765; Fri, 26 Apr 2013 04:30:37 -0700 (PDT) Received: from 11A0116.eolas.loc ([178.237.98.13]) by mx.google.com with ESMTPSA id k43sm15512709een.2.2013.04.26.04.30.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 04:30:36 -0700 (PDT) Message-ID: <517A657B.7060003@gmail.com> Date: Fri, 26 Apr 2013 13:31:07 +0200 From: =?UTF-8?B?IkNsw6ltZW50IEhlcm1hbm4gKG5vZGVucyki?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: High CPU interrupt load on intel I350T4 with igb on 8.3 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 11:30:45 -0000 Hi list, We use pf+ALTQ for trafic shaping on some routers. We are switching to new servers : Dell PowerEdge R620 with 2 8-cores Intel Processor (E5-2650L), 8GB RAM and Intel I350T4 (quad port) using igb driver. The old hardware is using em driver, the CPU load is high but mostly due to kernel and a large pf ruleset. On the new hardware, we see high CPU Interrupt load (up to 95%), even though there is not much trafic currently (peaks about 150Mbps and 40Kpps). All queues are used and binded to a cpu according to top, but a lot of CPU time is spent on igb queues (interrupt or wait). The load is fine when we stay below 20Kpps. We see no mbuf shortage, no dropped packet, but there is little margin left on CPU time (about 25% idle at best, most of CPU time is spent on interrupts), which is disturbing. We have done some tuning, but to no avail : sysctl.conf : # mbufs kern.ipc.nmbclusters=65536 # Sockets kern.ipc.somaxconn=8192 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.udp.recvspace=65535 net.inet.udp.maxdgram=57344 net.local.stream.recvspace=65535 net.local.stream.sendspace=65535 # IGB dev.igb.0.rx_processing_limit=4096 dev.igb.1.rx_processing_limit=4096 dev.igb.2.rx_processing_limit=4096 dev.igb.3.rx_processing_limit=4096 /boot/loader.conf : vm.kmem_size=1G hw.igb.max_interrupt_rate="32000" # maximum number of interrupts/sec generated by single igb(4) (default 8000) hw.igb.txd="2048" # number of transmit descriptors allocated by the driver (2048 limit) hw.igb.rxd="2048" # number of receive descriptors allocated by the driver (2048 limit) hw.igb.rx_process_limit="1000" # maximum number of received packets to process at a time, The default of 100 is # too low for most firewalls. (-1 means unlimited) Kernel HZ is 1000. The IGB /boot/loader.conf tuning was our last attempt, it didn't change anything. Does anyone have any pointer ? How could we lower CPU interrupt load ? should we set hw.igb.max_interrupt_rate lower instead of higher ? From what we saw here and there, we should be able to do much better with this hardware. relevant sysctl (igb1 and igb2 only, other interfaces are unused) : sysctl dev.igb | grep -v ": 0$" dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.3.1 dev.igb.1.%driver: igb dev.igb.1.%location: slot=0 function=1 dev.igb.1.%pnpinfo: vendor=0x8086 device=0x1521 subvendor=0x8086 subdevice=0x5001 class=0x020000 dev.igb.1.%parent: pci5 dev.igb.1.nvm: -1 dev.igb.1.enable_aim: 1 dev.igb.1.fc: 3 dev.igb.1.rx_processing_limit: 4096 dev.igb.1.eee_disabled: 1 dev.igb.1.link_irq: 2 dev.igb.1.device_control: 1209795137 dev.igb.1.rx_control: 67141658 dev.igb.1.interrupt_mask: 4 dev.igb.1.extended_int_mask: 2147483981 dev.igb.1.fc_high_water: 33168 dev.igb.1.fc_low_water: 33152 dev.igb.1.queue0.interrupt_rate: 71428 dev.igb.1.queue0.txd_head: 1318 dev.igb.1.queue0.txd_tail: 1318 dev.igb.1.queue0.tx_packets: 84663594 dev.igb.1.queue0.rxd_head: 717 dev.igb.1.queue0.rxd_tail: 715 dev.igb.1.queue0.rx_packets: 43899597 dev.igb.1.queue0.rx_bytes: 8905556030 dev.igb.1.queue1.interrupt_rate: 90909 dev.igb.1.queue1.txd_head: 693 dev.igb.1.queue1.txd_tail: 693 dev.igb.1.queue1.tx_packets: 57543349 dev.igb.1.queue1.rxd_head: 1033 dev.igb.1.queue1.rxd_tail: 1032 dev.igb.1.queue1.rx_packets: 54821897 dev.igb.1.queue1.rx_bytes: 9944955108 dev.igb.1.queue2.interrupt_rate: 100000 dev.igb.1.queue2.txd_head: 350 dev.igb.1.queue2.txd_tail: 350 dev.igb.1.queue2.tx_packets: 62320990 dev.igb.1.queue2.rxd_head: 1962 dev.igb.1.queue2.rxd_tail: 1939 dev.igb.1.queue2.rx_packets: 43909016 dev.igb.1.queue2.rx_bytes: 8673941461 dev.igb.1.queue3.interrupt_rate: 14925 dev.igb.1.queue3.txd_head: 647 dev.igb.1.queue3.txd_tail: 647 dev.igb.1.queue3.tx_packets: 58776199 dev.igb.1.queue3.rxd_head: 692 dev.igb.1.queue3.rxd_tail: 691 dev.igb.1.queue3.rx_packets: 55138996 dev.igb.1.queue3.rx_bytes: 9310217354 dev.igb.1.queue4.interrupt_rate: 100000 dev.igb.1.queue4.txd_head: 1721 dev.igb.1.queue4.txd_tail: 1721 dev.igb.1.queue4.tx_packets: 54337209 dev.igb.1.queue4.rxd_head: 1609 dev.igb.1.queue4.rxd_tail: 1598 dev.igb.1.queue4.rx_packets: 46546503 dev.igb.1.queue4.rx_bytes: 8818182840 dev.igb.1.queue5.interrupt_rate: 11627 dev.igb.1.queue5.txd_head: 254 dev.igb.1.queue5.txd_tail: 254 dev.igb.1.queue5.tx_packets: 53117182 dev.igb.1.queue5.rxd_head: 701 dev.igb.1.queue5.rxd_tail: 685 dev.igb.1.queue5.rx_packets: 43014837 dev.igb.1.queue5.rx_bytes: 8699057447 dev.igb.1.queue6.interrupt_rate: 55555 dev.igb.1.queue6.txd_head: 8 dev.igb.1.queue6.txd_tail: 8 dev.igb.1.queue6.tx_packets: 52654088 dev.igb.1.queue6.rxd_head: 1057 dev.igb.1.queue6.rxd_tail: 1041 dev.igb.1.queue6.rx_packets: 45227030 dev.igb.1.queue6.rx_bytes: 9494489640 dev.igb.1.queue7.interrupt_rate: 5235 dev.igb.1.queue7.txd_head: 729 dev.igb.1.queue7.txd_tail: 729 dev.igb.1.queue7.tx_packets: 61926105 dev.igb.1.queue7.rxd_head: 146 dev.igb.1.queue7.rxd_tail: 140 dev.igb.1.queue7.rx_packets: 51781775 dev.igb.1.queue7.rx_bytes: 8901279226 dev.igb.1.mac_stats.missed_packets: 1657 dev.igb.1.mac_stats.recv_no_buff: 405 dev.igb.1.mac_stats.total_pkts_recvd: 384332760 dev.igb.1.mac_stats.good_pkts_recvd: 384331103 dev.igb.1.mac_stats.bcast_pkts_recvd: 15510 dev.igb.1.mac_stats.mcast_pkts_recvd: 52957 dev.igb.1.mac_stats.rx_frames_64: 195496498 dev.igb.1.mac_stats.rx_frames_65_127: 133346124 dev.igb.1.mac_stats.rx_frames_128_255: 5254911 dev.igb.1.mac_stats.rx_frames_256_511: 9700049 dev.igb.1.mac_stats.rx_frames_512_1023: 16885886 dev.igb.1.mac_stats.rx_frames_1024_1522: 23647635 dev.igb.1.mac_stats.good_octets_recvd: 74284029276 dev.igb.1.mac_stats.good_octets_txd: 544536708502 dev.igb.1.mac_stats.total_pkts_txd: 485327419 dev.igb.1.mac_stats.good_pkts_txd: 485327419 dev.igb.1.mac_stats.bcast_pkts_txd: 72 dev.igb.1.mac_stats.mcast_pkts_txd: 52820 dev.igb.1.mac_stats.tx_frames_64: 57820809 dev.igb.1.mac_stats.tx_frames_65_127: 51586341 dev.igb.1.mac_stats.tx_frames_128_255: 7050579 dev.igb.1.mac_stats.tx_frames_256_511: 7887126 dev.igb.1.mac_stats.tx_frames_512_1023: 10130891 dev.igb.1.mac_stats.tx_frames_1024_1522: 350851673 dev.igb.1.interrupts.asserts: 551135045 dev.igb.1.interrupts.rx_pkt_timer: 384326679 dev.igb.1.interrupts.tx_queue_empty: 485323376 dev.igb.1.interrupts.tx_queue_min_thresh: 6324386 dev.igb.1.host.rx_pkt: 4424 dev.igb.1.host.tx_good_pkt: 4043 dev.igb.1.host.rx_good_bytes: 74284030864 dev.igb.1.host.tx_good_bytes: 544536708502 dev.igb.2.%desc: Intel(R) PRO/1000 Network Connection version - 2.3.1 dev.igb.2.%driver: igb dev.igb.2.%location: slot=0 function=2 dev.igb.2.%pnpinfo: vendor=0x8086 device=0x1521 subvendor=0x8086 subdevice=0x5001 class=0x020000 dev.igb.2.%parent: pci5 dev.igb.2.nvm: -1 dev.igb.2.enable_aim: 1 dev.igb.2.fc: 3 dev.igb.2.rx_processing_limit: 4096 dev.igb.2.eee_disabled: 1 dev.igb.2.link_irq: 2 dev.igb.2.device_control: 1209795137 dev.igb.2.rx_control: 67141658 dev.igb.2.interrupt_mask: 4 dev.igb.2.extended_int_mask: 2147483959 dev.igb.2.fc_high_water: 33168 dev.igb.2.fc_low_water: 33152 dev.igb.2.queue0.interrupt_rate: 13698 dev.igb.2.queue0.txd_head: 1618 dev.igb.2.queue0.txd_tail: 1618 dev.igb.2.queue0.tx_packets: 46401106 dev.igb.2.queue0.rxd_head: 831 dev.igb.2.queue0.rxd_tail: 827 dev.igb.2.queue0.rx_packets: 69356350 dev.igb.2.queue0.rx_bytes: 68488772907 dev.igb.2.queue1.interrupt_rate: 5405 dev.igb.2.queue1.txd_head: 190 dev.igb.2.queue1.txd_tail: 190 dev.igb.2.queue1.tx_packets: 55965886 dev.igb.2.queue1.rxd_head: 268 dev.igb.2.queue1.rxd_tail: 256 dev.igb.2.queue1.rx_packets: 58958084 dev.igb.2.queue1.rx_bytes: 69154569937 dev.igb.2.queue2.interrupt_rate: 83333 dev.igb.2.queue2.txd_head: 568 dev.igb.2.queue2.txd_tail: 568 dev.igb.2.queue2.tx_packets: 44974648 dev.igb.2.queue2.rxd_head: 371 dev.igb.2.queue2.rxd_tail: 219 dev.igb.2.queue2.rx_packets: 67037407 dev.igb.2.queue2.rx_bytes: 72042326102 dev.igb.2.queue3.interrupt_rate: 12658 dev.igb.2.queue3.txd_head: 867 dev.igb.2.queue3.txd_tail: 867 dev.igb.2.queue3.tx_packets: 55962467 dev.igb.2.queue3.rxd_head: 85 dev.igb.2.queue3.rxd_tail: 1953 dev.igb.2.queue3.rx_packets: 60972965 dev.igb.2.queue3.rx_bytes: 70397176035 dev.igb.2.queue4.interrupt_rate: 90909 dev.igb.2.queue4.txd_head: 1920 dev.igb.2.queue4.txd_tail: 1920 dev.igb.2.queue4.tx_packets: 47660931 dev.igb.2.queue4.rxd_head: 1397 dev.igb.2.queue4.rxd_tail: 1379 dev.igb.2.queue4.rx_packets: 59110758 dev.igb.2.queue4.rx_bytes: 68919201478 dev.igb.2.queue5.interrupt_rate: 111111 dev.igb.2.queue5.txd_head: 886 dev.igb.2.queue5.txd_tail: 886 dev.igb.2.queue5.tx_packets: 45103990 dev.igb.2.queue5.rxd_head: 812 dev.igb.2.queue5.rxd_tail: 799 dev.igb.2.queue5.rx_packets: 59030312 dev.igb.2.queue5.rx_bytes: 69234293962 dev.igb.2.queue6.interrupt_rate: 5208 dev.igb.2.queue6.txd_head: 1926 dev.igb.2.queue6.txd_tail: 1926 dev.igb.2.queue6.tx_packets: 46215046 dev.igb.2.queue6.rxd_head: 692 dev.igb.2.queue6.rxd_tail: 689 dev.igb.2.queue6.rx_packets: 58256050 dev.igb.2.queue6.rx_bytes: 68429172749 dev.igb.2.queue7.interrupt_rate: 50000 dev.igb.2.queue7.txd_head: 126 dev.igb.2.queue7.txd_tail: 126 dev.igb.2.queue7.tx_packets: 52451455 dev.igb.2.queue7.rxd_head: 968 dev.igb.2.queue7.rxd_tail: 885 dev.igb.2.queue7.rx_packets: 65946491 dev.igb.2.queue7.rx_bytes: 70263478849 dev.igb.2.mac_stats.missed_packets: 958 dev.igb.2.mac_stats.recv_no_buff: 69 dev.igb.2.mac_stats.total_pkts_recvd: 498658079 dev.igb.2.mac_stats.good_pkts_recvd: 498657121 dev.igb.2.mac_stats.bcast_pkts_recvd: 16867 dev.igb.2.mac_stats.mcast_pkts_recvd: 52957 dev.igb.2.mac_stats.rx_frames_64: 59089332 dev.igb.2.mac_stats.rx_frames_65_127: 52880118 dev.igb.2.mac_stats.rx_frames_128_255: 7526966 dev.igb.2.mac_stats.rx_frames_256_511: 8468389 dev.igb.2.mac_stats.rx_frames_512_1023: 10434770 dev.igb.2.mac_stats.rx_frames_1024_1522: 360257545 dev.igb.2.mac_stats.good_octets_recvd: 558910494322 dev.igb.2.mac_stats.good_octets_txd: 84618858153 dev.igb.2.mac_stats.total_pkts_txd: 394726904 dev.igb.2.mac_stats.good_pkts_txd: 394726904 dev.igb.2.mac_stats.bcast_pkts_txd: 48 dev.igb.2.mac_stats.mcast_pkts_txd: 52821 dev.igb.2.mac_stats.tx_frames_64: 196605932 dev.igb.2.mac_stats.tx_frames_65_127: 134602807 dev.igb.2.mac_stats.tx_frames_128_255: 5705236 dev.igb.2.mac_stats.tx_frames_256_511: 10267168 dev.igb.2.mac_stats.tx_frames_512_1023: 17165496 dev.igb.2.mac_stats.tx_frames_1024_1522: 30380265 dev.igb.2.interrupts.asserts: 465994260 dev.igb.2.interrupts.rx_pkt_timer: 498647546 dev.igb.2.interrupts.tx_queue_empty: 394720657 dev.igb.2.interrupts.tx_queue_min_thresh: 24424555 dev.igb.2.host.rx_pkt: 9575 dev.igb.2.host.tx_good_pkt: 6248 dev.igb.2.host.rx_good_bytes: 558910513984 dev.igb.2.host.tx_good_bytes: 84618858217 Thanks for your help. Cheers, -- Clement (nodens) From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 13:42:27 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 13689EF4 for ; Fri, 26 Apr 2013 13:42:27 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 97EE81A85 for ; Fri, 26 Apr 2013 13:42:26 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3QDgPFC077401; Fri, 26 Apr 2013 17:42:25 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3QDgOJS077400; Fri, 26 Apr 2013 17:42:24 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 26 Apr 2013 17:42:24 +0400 From: Gleb Smirnoff To: Erich Weiler Subject: Re: pf performance? Message-ID: <20130426134224.GV76816@FreeBSD.org> References: <5176E5C1.9090601@soe.ucsc.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <5176E5C1.9090601@soe.ucsc.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 13:42:27 -0000 Erich, On Tue, Apr 23, 2013 at 12:49:21PM -0700, Erich Weiler wrote: E> I have a question here about how FreeBSD (8.1-RELEASE-p13 specifically) E> behaves when acting as a firewall. I understand the pf process is E> "giant locked" to a single CPU core when inspecting packets inbound and E> outbound. I was wondering, how does that manifest when I look at "top E> -P" on the firewall? The pf isn't a process, so you can't see it in top. pf has some helper threads however, but packet processing isn't performed by any of them. The pf is kind of a library in kernel. The packets are processed by NIC interrupt handler threads, and these threads enter the library to perform packet filtering. Since in FreeBSD 8 this library is covered by a single lock (it isn't the Giant, but it is kind of "local pf giant"), processing is serialized - threads enter the library one by one, and they are blocked on enter in case if other thread already works inside. In FreeBSD 10 pf is no longer under single lock. On your hardware, I'd expect a measurable performance gain if you migrate to 10. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 14:19:31 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2A05B84A for ; Fri, 26 Apr 2013 14:19:31 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id A50691C08 for ; Fri, 26 Apr 2013 14:19:30 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id fp13so1388033lab.36 for ; Fri, 26 Apr 2013 07:19:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dudu.ro; s=google; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=PxzHzQpmqTjirfQulr1VeffyAiNkq7r2k0qDrCbk7OI=; b=ge7T+byZ8yinIFVmsGJx40iiKo/vLOOzzWeVt0ZWnSsZXDKOXNFhVdMilEznsB9vOT zMlBt6DTAFA7GOpeAw10EQ0fvEXnkLaK6GOX2NPjKpX4lF8FHTJ81x6TsszVHzI7JknZ Pa/qqdmvkoW53OD/+2UqOH8VfzDpxv0reVpts= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=PxzHzQpmqTjirfQulr1VeffyAiNkq7r2k0qDrCbk7OI=; b=MlZnLU7gRvfYgjUSFMaiXMFr/KEDCzfKy37FuKqozPpKZwcokbv2Ee0Px6rkdRDKtD o7z0Ue+QvlOnOe+ee5xkxwuLyrUFHXMl/ckmOPs4pvR8+1WHe9cMJTE1mSACPMSVkJEZ M7lcqrRbhj3qUx5/9MqG63eZeLFewfNjNbolMRTpqgboNx5UcOz6Kr7qEYByhOnsW5cV haMkiq+1XdEYfJQ5LLgGGAkr68IA3zTBBm2TUb/09qOLkZV2tZRYSWSURcQ/bQzBV8nP tYpZ17YfINHErXTKlzEy8k1cCukn5PGgjVpbtHaAr0e0hWNarjL+kzi8zEaHaO8WvuD2 vq7Q== X-Received: by 10.112.167.168 with SMTP id zp8mr9748811lbb.58.1366985969363; Fri, 26 Apr 2013 07:19:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.186.233 with HTTP; Fri, 26 Apr 2013 07:18:49 -0700 (PDT) In-Reply-To: <517472A1.6020100@uj.edu.pl> References: <517472A1.6020100@uj.edu.pl> From: Vlad Galu Date: Fri, 26 Apr 2013 15:18:49 +0100 Message-ID: Subject: Re: Implementation of SCPS To: Konrad Witaszczyk X-Gm-Message-State: ALoCoQnoOTx0yf1QcSBZlZMMgl8Q/+p5uUVfRG/KRvIImF2xodkFGoQ9L7Vy11v6nE56dKTFqWYf Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: bms@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 14:19:31 -0000 It is definitely interesting. On the other hand, I work for one of the major players in the field, so I am quite subjective. On Mon, Apr 22, 2013 at 12:13 AM, Konrad Witaszczyk < konrad.witaszczyk@uj.edu.pl> wrote: > Hi, > > I'm looking for information about an implementation of SCPS in FreeBSD ( > https://wiki.freebsd.org/**IdeasPage#SCPS.2C_Space_** > Communication_Protocol_**Standards). > Could you tell me what's the current status of this project? Do you think > that it's a good project for Google Summer of Code? > > > Regards, > Konrad > ______________________________**_________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org > " > -- Good, fast & cheap. Pick any two. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 14:36:05 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C3A78BBF; Fri, 26 Apr 2013 14:36:05 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 86AEE1CF6; Fri, 26 Apr 2013 14:36:05 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id ta17so3574271obb.26 for ; Fri, 26 Apr 2013 07:36:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=JN6w+XBB5tKU8qwgQRa7l7nZDPuxaAWSgob8wgq7aAI=; b=YwqFUYAE32H9r/BAaNxen6FjKRKRaHKagEfBseH/vKqdJ2h3kNsmTcxxzjoh/1x7yb Tj1I8eOX6cGspa2fkHPUlgLmlPfMRioYKxpXQ1FVQT/+BI25+S6Q7YPhHKyJkQbE8fLb Jj0o6piLMe0oXbMOm/JNJ3hZUekhAhuqz87uCymDEB52R3GhbC3+oMAoOIljOHwGRNB6 s5T/VZnS6QkpcR61SX/ox+LHSowFxHndZlEp3XxRgCq4+cGYsy+OwdrWYRBjf+3Flkiz 2X8AZdq0qz8RsBBlRZIGRo66f6jgGPUwZ2rY1NQsR3okhypWZv0XM5Ea6FSdqDULMK/H KKeQ== MIME-Version: 1.0 X-Received: by 10.60.162.71 with SMTP id xy7mr18061499oeb.88.1366986963797; Fri, 26 Apr 2013 07:36:03 -0700 (PDT) Received: by 10.76.162.230 with HTTP; Fri, 26 Apr 2013 07:36:03 -0700 (PDT) In-Reply-To: References: <517472A1.6020100@uj.edu.pl> Date: Fri, 26 Apr 2013 10:36:03 -0400 Message-ID: Subject: Re: Implementation of SCPS From: Outback Dingo To: Vlad Galu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Konrad Witaszczyk , bms@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 14:36:05 -0000 On Fri, Apr 26, 2013 at 10:18 AM, Vlad Galu wrote: > It is definitely interesting. On the other hand, I work for one of the > major players in the field, so I am quite subjective. > > On Mon, Apr 22, 2013 at 12:13 AM, Konrad Witaszczyk < > konrad.witaszczyk@uj.edu.pl> wrote: > > > Hi, > > > > I'm looking for information about an implementation of SCPS in FreeBSD ( > > https://wiki.freebsd.org/**IdeasPage#SCPS.2C_Space_** > > Communication_Protocol_**Standards< > https://wiki.freebsd.org/IdeasPage#SCPS.2C_Space_Communication_Protocol_Standards > >). > > Could you tell me what's the current status of this project? Do you think > > that it's a good project for Google Summer of Code? > > > > > > Regards, > > Konrad > > ______________________________**_________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/**mailman/listinfo/freebsd-net< > http://lists.freebsd.org/mailman/listinfo/freebsd-net> > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org< > freebsd-net-unsubscribe@freebsd.org> > > " > > > > there was at one time some freebsd code for SCPS floating out there on the internet someone had done, i guess google FreeBSD SCPS and see if its still out there, not sure what state its in. > > > -- > Good, fast & cheap. Pick any two. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 14:49:42 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 99A2C80 for ; Fri, 26 Apr 2013 14:49:42 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by mx1.freebsd.org (Postfix) with ESMTP id 71AA81D7A for ; Fri, 26 Apr 2013 14:49:42 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id bh4so2541188pad.12 for ; Fri, 26 Apr 2013 07:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=QUPR75uGDlip1Fgso75giP5mvNYEUyFk3KI3+tieJwY=; b=Xq6O71L7CSuTve8Lv0v6bb3DsOOBbAmqml4MljR94ShL2SdYSigSNEKNwmdCtx7A69 Lyl5SAQrr28nn9nVH8MYDGJPNdXBrEKdniaG3nuFOllrGPc8F5guL8Pzab5gKWEF68AR 9GKDeZ1Cxpy+MkgqhtiDIN0OlmR/e2+mfSabU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=QUPR75uGDlip1Fgso75giP5mvNYEUyFk3KI3+tieJwY=; b=fr9ovI2hecjGh/mlMOxVWkebe8wSJ1vqmaeXxwRjHWEw05vLMCHfZbxgz30x2KnCra lI/PBHHJKs1KsegpYTa85SyuyiAxRS2kspb8wS0QOrazFtHSyYzCbauvjGOS7OAz4UoH bAfj5PHjWPhnl125apuAjvgU2CwZR7G+uZ2NnkfnITsAni8rO+ZHoqFA8oOiscxg041L /3KvNCAvbnJ3oMQH85q6luvQJKup/BIn1Ati7hszYe6JxvltWwz9jzsVa7EAafoaDFOv /31mJ5bqEAVhhLNn7g01y8VNgOXkB81YddEv5LVih+0M9+F1/otwZUphrab7g62xif/U 56rQ== X-Received: by 10.69.0.132 with SMTP id ay4mr58970979pbd.62.1366987774970; Fri, 26 Apr 2013 07:49:34 -0700 (PDT) Received: from [192.168.0.146] (50-0-69-3.dsl.static.fusionbroadband.com. [50.0.69.3]) by mx.google.com with ESMTPSA id at4sm11958827pbc.40.2013.04.26.07.49.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 07:49:33 -0700 (PDT) Message-ID: <517A93FE.7020209@soe.ucsc.edu> Date: Fri, 26 Apr 2013 07:49:34 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Gleb Smirnoff Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> In-Reply-To: <20130426134224.GV76816@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQniaH6UGYKzX4/5uc1XQHjLbrDkJfByixHg8EvGK7TvO8aHjKcDQwGzGaz/U++CVgxok+/m Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 14:49:42 -0000 > The pf isn't a process, so you can't see it in top. pf has some helper > threads however, but packet processing isn't performed by any of them. But the work pf does would show up in 'system' on top right? So if I see all my CPUs tied up 100% in 'interrupts' and very little 'system', would it be a reasonable assumption to think that if I got more CPU cores to handle the interrupts that eventually I would see 'system' load increase as the interrupt load became faster to be handled? And thus increase my bandwidth? In other words, until I see like 100% system usage in one core, I would have room to grow? Sorry for being dense, just trying to squeeze every ounce of blood out of this box that I can... ;) Unfortunately I can't move to FreeBSD 10 because this is a pfSense box and they are locked on 8.1 at the moment for the version I'm running. They are eyeing version 10 for a future release but it may be a year or more before that gets released. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 15:05:25 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 28BDF614 for ; Fri, 26 Apr 2013 15:05:25 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [IPv6:2a01:4f8:141:52a3:186::]) by mx1.freebsd.org (Postfix) with ESMTP id DFA581E46 for ; Fri, 26 Apr 2013 15:05:24 +0000 (UTC) Received: from [10.10.1.100] (unknown [217.71.4.82]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id 865BAD596E; Fri, 26 Apr 2013 17:05:23 +0200 (CEST) X-DKIM: OpenDKIM Filter v2.5.2 mail.tyknet.dk 865BAD596E DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1366988723; bh=zNjVfbZsl1F5Lh7hffNr98/hM99glvh/w5q0+2VIQ74=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=D/BfhwR7cnp8CNCTuw7XRHnbKRXq52rmLWksyR9JtEa2I9idRx0d6NDWmnMdKHCpw 5b+UuGEykLVykQSSe+5/iqP4jzdiqBM7bKH/ox3O9NYWP+ncrh+LiwnAhvFaiQ+oCQ YEV7mOSUFtBCHn31uumEoO17v4ZLTC182o+uiDc8= Message-ID: <517A97AB.2050103@gibfest.dk> Date: Fri, 26 Apr 2013 17:05:15 +0200 From: Thomas Steen Rasmussen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Rainer Bredehorn Subject: Re: Aw: PF IPv6 fragment support References: In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 15:05:25 -0000 On 26-04-2013 10:26, Rainer Bredehorn wrote: >> I'm using FreeBSD 8.3 which doesn't support IPv6 fragments in PF. >> Does FreeBSD 9.x PF support IPv6 fragments? >> I can't find it in the 9.0 or 9.1 manpages. For pf.conf they are the same as in FreeBSD 8.3. > I've modified the kernel PF implementation to pass IPv6 fragments. > The first fragment is handled by the PF rules of course ignoring possible checksums. > All other fragments are passed by PF to the IP stack. > This can be done state-full but reassembling fragments is not supported. > > That's what I wanted. > > Rainer. Care to share ? :) /Thomas From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 15:54:44 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8D097657 for ; Fri, 26 Apr 2013 15:54:44 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0498B100C for ; Fri, 26 Apr 2013 15:54:43 +0000 (UTC) Received: (qmail 8436 invoked from network); 26 Apr 2013 16:58:59 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 26 Apr 2013 16:58:59 -0000 Message-ID: <517AA337.8050505@freebsd.org> Date: Fri, 26 Apr 2013 17:54:31 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Erich Weiler Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> In-Reply-To: <517A93FE.7020209@soe.ucsc.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 15:54:44 -0000 On 26.04.2013 16:49, Erich Weiler wrote: >> The pf isn't a process, so you can't see it in top. pf has some helper >> threads however, but packet processing isn't performed by any of them. > > But the work pf does would show up in 'system' on top right? So if I see all my CPUs tied up 100% > in 'interrupts' and very little 'system', would it be a reasonable assumption to think that if I got > more CPU cores to handle the interrupts that eventually I would see 'system' load increase as the > interrupt load became faster to be handled? And thus increase my bandwidth? Having the work of pf show up in 'interrupts' or 'system' depends on the network driver and how it handles sending packets up the stack. In most cases drivers deliver packets from interrupt context. > In other words, until I see like 100% system usage in one core, I would have room to grow? You have room to grow if 'idle' is more than 0% and the interrupts of the networks cards are running on different cores. If one core gets all the interrupts a second idle core doesn't get the chance to help out. IIRC the interrupt allocation to cores is done at interrupt registration time or driver attach time. It can be re-distributed at run time on most architecture but I'm not sure we have an easily accessible API for that. -- Andre From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 16:04:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 635E4BD2 for ; Fri, 26 Apr 2013 16:04:22 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51]) by mx1.freebsd.org (Postfix) with ESMTP id 398D5107D for ; Fri, 26 Apr 2013 16:04:22 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id jh10so2570909pab.38 for ; Fri, 26 Apr 2013 09:04:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=e2G2ymCvrRHMQBVbT1V7Ak8cR9j7oBLJfLnp+SGp7EQ=; b=hLKASNOxIq4tf0mZTphgCdKAD82LGoI4tXf7/L/nyR8XUZY5PeJ96g4rjfH98PHrJ+ zXm8PgpF293we66fRs7mlf3grfK3eZh1IPriPbzD48FF0PtGCLUNU7vRXG1Rd6RVWnJM rsfpDYJllNdaR3YZ+Rk4/iiNeCG2DUKOvSuhQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=e2G2ymCvrRHMQBVbT1V7Ak8cR9j7oBLJfLnp+SGp7EQ=; b=QHH7YMQS48umRKskUl4fsEIFF9BGvETKZu31IaWKhANq/NMUvogMY+ZkioAr58RlJb HKq5nNX7CZ++0xqIHFnOBtj2zBoEUx5KqXQjITYQc8eJEMs0H1WXxi8mpJ7MXKg/SlP+ pkTL/NbIagvJlPTG8m5pYJ5hinOw2TwulVigoyeuBnc1la2AJ0cmnI8Jgy/2TYp2mFdh YpuQcNYrMLx+2XUndphVSOCXr+jGPoT18jMBGtbHz+mLXAyHOMVpzi9G6hAoKcvSOW9k g3qbl0bJD/uy5Zc+8cIUsIMwYejpHUrzUNZhV8JUn+Ef1juunpV1SP5vzeCChjVkCS0r h8Mg== X-Received: by 10.66.246.168 with SMTP id xx8mr35315912pac.107.1366992256731; Fri, 26 Apr 2013 09:04:16 -0700 (PDT) Received: from [192.168.0.146] (50-0-69-3.dsl.static.fusionbroadband.com. [50.0.69.3]) by mx.google.com with ESMTPSA id l4sm12245777pbo.6.2013.04.26.09.04.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 09:04:15 -0700 (PDT) Message-ID: <517AA57F.2000501@soe.ucsc.edu> Date: Fri, 26 Apr 2013 09:04:15 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Andre Oppermann Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> <517AA337.8050505@freebsd.org> In-Reply-To: <517AA337.8050505@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQlY7SzOv/69Fdi+PU2OnV3amncRHcEy/8poyOK5K2CMo2xpzI5ETsI1GhcTcwwoAVomsMAX Cc: Paul Tatarsky , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:04:22 -0000 >> But the work pf does would show up in 'system' on top right? So if I >> see all my CPUs tied up 100% >> in 'interrupts' and very little 'system', would it be a reasonable >> assumption to think that if I got >> more CPU cores to handle the interrupts that eventually I would see >> 'system' load increase as the >> interrupt load became faster to be handled? And thus increase my >> bandwidth? > > Having the work of pf show up in 'interrupts' or 'system' depends on the > network driver and how it handles sending packets up the stack. In most > cases drivers deliver packets from interrupt context. Ah, I see. Definitely appears for me in interrupts then. I've got the mxge driver doing the work here. So, given that I can spread out the interrupts to every core (like, pin an interrupt queue to each core), I can have all my cores work on the process. But seeing as though the pf bit is still serialized I'm not sure that I understand how it is serialized when many CPUs are handling interrupts, and hence doing the work of pf as well? Wouldn't that indicate that the work of pf is being handled by many cores, as many cores are handling the interrupts? Or are you saying that pf *is* being handled by many cores, but just in a serialized nature? Like, packet 1 is handled by core 0, then packet 2 is handled by core 1, packet 3 is handled by core 4, etc? Such that even though multiple cores are handling it, they are just doing so serially and not in parallel? And if so, maybe it still helps to have many cores? Thanks for all the excellent info! >> In other words, until I see like 100% system usage in one core, I >> would have room to grow? > > You have room to grow if 'idle' is more than 0% and the interrupts of > the networks cards are running on different cores. If one core gets > all the interrupts a second idle core doesn't get the chance to help > out. IIRC the interrupt allocation to cores is done at interrupt > registration time or driver attach time. It can be re-distributed > at run time on most architecture but I'm not sure we have an easily > accessible API for that. > From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 16:05:33 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2A9F4C6E; Fri, 26 Apr 2013 16:05:33 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id E05C4108E; Fri, 26 Apr 2013 16:05:32 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id ta17so3710411obb.12 for ; Fri, 26 Apr 2013 09:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=MPmd2DknYQmHvzdZsL1ogzIbaPeqmbkUfhiN1afHquM=; b=f+T6YGEpLiaA+mqOaCDyDzZp0e9EyZNptnJ22glGSsk5o0DJtwWk0FHBISwLL33sNN gQ9U6uMg4SEcgSA5kvZofFcgN1zErYN3rj7FbmNXbvjNzsnbNFQr9mVhaoeJlKO0Rba3 0mFkgvzM0cs0jZhmcZkkvojDgIxsl14Sfz87fY++pL4APxJDvF5DRr34Yboo9ZoA9Yhk +3PlADiQbkYnUq/4je5kSZuO5GRz+sbkjano35mfpgvQ42EIAx98+xj2VOMmqmhZSLjS 3JcBLyv3YoK6MbVwTbOU4kw/9ybt3m3viJceCKOt/pgOAyGuQi/kJ5UkPSeW4bUtLuLA Qq3A== MIME-Version: 1.0 X-Received: by 10.182.96.1 with SMTP id do1mr17765581obb.17.1366992332536; Fri, 26 Apr 2013 09:05:32 -0700 (PDT) Received: by 10.76.156.198 with HTTP; Fri, 26 Apr 2013 09:05:32 -0700 (PDT) In-Reply-To: <517A93FE.7020209@soe.ucsc.edu> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> Date: Fri, 26 Apr 2013 12:05:32 -0400 Message-ID: Subject: Re: pf performance? From: Ryan Stone To: Erich Weiler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:05:33 -0000 On Fri, Apr 26, 2013 at 10:49 AM, Erich Weiler wrote: > The pf isn't a process, so you can't see it in top. pf has some helper >> threads however, but packet processing isn't performed by any of them. >> > > But the work pf does would show up in 'system' on top right? So if I see > all my CPUs tied up 100% in 'interrupts' and very little 'system', would it > be a reasonable assumption to think that if I got more CPU cores to handle > the interrupts that eventually I would see 'system' load increase as the > interrupt load became faster to be handled? And thus increase my bandwidth? > > In other words, until I see like 100% system usage in one core, I would > have room to grow? > Probably not. Mutexes in FreeBSD use "adaptive spinning". This means that when a thread is unable to acquire a mutex, if the owner of the mutex is still running on another CPU core then the thread will spin and wait for the mutex to become free rather than block. This improves the latency of acquiring the mutex and saves us many expensive calls through the scheduler, but the downside is that you have one heavily contended mutex you can have many cores wasted spinning on the lock. In your case it is likely that your interrupt threads are spinning on the pf lock. You can partially confirm this by profiling the system. The quick way is: kldload hwpmc pmcstat -S unhalted-cycles -T (if unhalted-cycles does not work, try instructions). If _mtx_lock_sleep is the top function then you are hitting mutex contention and more cores are unlikely to help. In this case you might actually be able to improve performance by *reducing* the number of threads that are contending in pf, for example by using fewer receive queues in your NIC. If this machine is dedicated for pf then setting sysctl net.isr.direct=0 might also improve performance, by forcing all packets to go through a single netisr thread (assuming that net.isr.maxthreads is 1). Note that this will apply to traffic that does not go through pf, so if this machine were doing other network things (e.g. serving NFS) then it would bottleneck your other traffic behind your pf traffic. From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 16:22:44 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E38186A2; Fri, 26 Apr 2013 16:22:44 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-vc0-f171.google.com (mail-vc0-f171.google.com [209.85.220.171]) by mx1.freebsd.org (Postfix) with ESMTP id 970A7113E; Fri, 26 Apr 2013 16:22:44 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id ha12so1391716vcb.30 for ; Fri, 26 Apr 2013 09:22:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=EDdYEM31WwHrwYrJ/omflnJeMfbbijlhS0LQLD8y8oM=; b=OsVPtL+VvHlU3rm88USaNydMtV3T0l01lTgq8GlHK7Z0RJ8TO6eiQXE8x3t4ye4Pua QEra2+4yici8j3gybyQtkkUMHdu/hupJsBVWk5je2eFn6U/xItELDjkeq5KP3inQlnXX mtlvHZGahNaJoNXvXAxBEN8mfRIiQLSaqfUY696YuMwmZQG9KcyIUFSpA3dGNIr+SS02 dmM4Vb2mS8l+Z5KItOi0HEMRzioz/PxxPATWkpYtXMV8LL6MCBBYuNC/4HkAqPQGO/E3 2y6swRCTszjx7D5C5mU96rZa6Qzyoma1GUXwxCmGLEJSOZZpI5f91r9vP+lWJ8v7AtF0 byIw== X-Received: by 10.52.100.5 with SMTP id eu5mr24760797vdb.66.1366993358627; Fri, 26 Apr 2013 09:22:38 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Fri, 26 Apr 2013 09:22:18 -0700 (PDT) In-Reply-To: <20130426134224.GV76816@FreeBSD.org> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Fri, 26 Apr 2013 18:22:18 +0200 X-Google-Sender-Auth: 1avJEO2F1a6NeuG0u6g-ZZiYHbU Message-ID: Subject: Re: pf performance? To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:22:45 -0000 On Fri, Apr 26, 2013 at 3:42 PM, Gleb Smirnoff wrote: > > In FreeBSD 10 pf is no longer under single lock. On your hardware, > I'd expect a measurable performance gain if you migrate to 10. Compairing 9.1 and current (249908) on my new test-server (HP ProLiant DL320 G5, dual-core Xeon 3050, dual Intel NIC). Like usual: one unidirectional flow of small packets, values in packet-per-seconds: x 9.1 + current N Min Max Median Avg Stddev x 5 379991 381508 381229 380892.6 667.69926 + 5 332833 335502 334726 334223.2 1142.8266 Difference at 95.0% confidence -46669.4 +/- 1364.98 -12.2526% +/- 0.358363% (Student's t, pooled s = 935.915) Regards, Olivier From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 16:28:39 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 39460B12 for ; Fri, 26 Apr 2013 16:28:39 +0000 (UTC) (envelope-from nodens2099@gmail.com) Received: from mail-ea0-x22b.google.com (mail-ea0-x22b.google.com [IPv6:2a00:1450:4013:c01::22b]) by mx1.freebsd.org (Postfix) with ESMTP id C57BB118D for ; Fri, 26 Apr 2013 16:28:38 +0000 (UTC) Received: by mail-ea0-f171.google.com with SMTP id b10so871719eae.16 for ; Fri, 26 Apr 2013 09:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=TyeugP1wn79GQiH7B9pjxoy18NYgiIxvOlSfXFahYEA=; b=cKMt4hAjRI/QHXVy/wI5YD6qaKbZ7Hc6IC3ydhIznaIqjH/OZ/is+NMzxc0WXuTX8H Cw5Bxy5kaP7EYuxnszlOW15j+O0YCRXS0g4NFbyeE808OUWQraPRvCEzAOvS+cVlhag3 ZniCF0n+9cnqamhENxH3xALicuYR4PsyDWnZJlYnYnRDTS8cdZJ/n0RcKyRJdLSfGRXj jQVxzm/ACz09rc9/Xm8bjhkOMRyfoRbDq1G6PArCvuWznp1XsKpEh4tz7Sx06315zJmN 5UID6gKjo8xin4giyBhk9YKtx6RH6Juf9SFVTjyYOItvrggHObhG5VLNteA/KhAwDXfB m3eQ== X-Received: by 10.14.115.131 with SMTP id e3mr15004898eeh.43.1366993717913; Fri, 26 Apr 2013 09:28:37 -0700 (PDT) Received: from 11A0116.eolas.loc ([178.237.98.13]) by mx.google.com with ESMTPSA id v48sm16799693eeg.7.2013.04.26.09.28.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 09:28:37 -0700 (PDT) Message-ID: <517AAB58.6020703@gmail.com> Date: Fri, 26 Apr 2013 18:29:12 +0200 From: =?ISO-8859-1?Q?=22Cl=E9ment_Hermann_=28nodens=29=22?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> <517AA337.8050505@freebsd.org> In-Reply-To: <517AA337.8050505@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:28:39 -0000 Hi, this thread seems it seems to be related to my problem (see High CPU interrupt load on intekl i350T4 with igb on 8.3). So let me jump in ;) Le 26/04/2013 17:54, Andre Oppermann a écrit : > On 26.04.2013 16:49, Erich Weiler wrote: >>> The pf isn't a process, so you can't see it in top. pf has some helper >>> threads however, but packet processing isn't performed by any of them. >> >> But the work pf does would show up in 'system' on top right? So if I >> see all my CPUs tied up 100% >> in 'interrupts' and very little 'system', would it be a reasonable >> assumption to think that if I got >> more CPU cores to handle the interrupts that eventually I would see >> 'system' load increase as the >> interrupt load became faster to be handled? And thus increase my >> bandwidth? > > Having the work of pf show up in 'interrupts' or 'system' depends on the > network driver and how it handles sending packets up the stack. In most > cases drivers deliver packets from interrupt context. > That is very interesting. Do you think my High CPU interrupt problem could be related to the fact that we use pf + altq ? >> In other words, until I see like 100% system usage in one core, I >> would have room to grow? > > You have room to grow if 'idle' is more than 0% and the interrupts of > the networks cards are running on different cores. If one core gets > all the interrupts a second idle core doesn't get the chance to help > out. IIRC the interrupt allocation to cores is done at interrupt > registration time or driver attach time. It can be re-distributed > at run time on most architecture but I'm not sure we have an easily > accessible API for that. > Would it be possible to use cpu affinity to reserve a core to pf ? For instance, use only 7 queues per card and keep a core available to pf ? Does that make sense or would the pf lock impact the interrupts of the interface queues ? I see a lot of "WAIT" state on the queues when I do a top -CHSIz. Cheers, -- Clément (nodens) From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 16:50:29 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 675AA789 for ; Fri, 26 Apr 2013 16:50:29 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) by mx1.freebsd.org (Postfix) with ESMTP id 3DF69127E for ; Fri, 26 Apr 2013 16:50:29 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id ld11so292831pab.5 for ; Fri, 26 Apr 2013 09:50:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=0VUF74/GTgk/xMTnYcjsA8WM2h03UT42QyieuzySdaU=; b=eysRZQ3x98kC60yPGo7wjs5kVN9NgQL+X7gWX/9VjJ5qAe3nkWosekiGvmYRZPaTqV PqNteVWzVkeLpEc69sKLbFOH72P25vNpbS7BfFtWHzhE9Z1vomhR2qbh0bc+WZiQsADq wd+xuy4ncapBM84uqg/ACvkYztVg/RD1kY9gA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=0VUF74/GTgk/xMTnYcjsA8WM2h03UT42QyieuzySdaU=; b=V0BLqXQOcMJS7P5NN5xZ1cQMThU3/Q9aXDwkcLgCHP7cIHXSvBzFhPLH+4emWlixEk s6UIF2c/DO+EjfXwAmsFvFDm9LSkNNL58Tq6F2z2J6IEGLX2iTluyIH7wmeWBw1yztHf nmuexbXIF9MmP0z/NFs+rMkyLiDtvItGEOoOM5xfm3qBH48LcrgulvaR+Fxc6slVXX/E esQty+DnoJMF4N6oUqjlpZnlVuF2H6mLi9vuM7yxr163CuiFHqo3ceAswQyMH7KzrpWE RG6M1N+RRvEQ3USZwLHrnUu3yzxdV6fSxGuXTWdUHVMjLmsiDR3mFsR6zG7TDyYeztwD /HiQ== X-Received: by 10.66.88.38 with SMTP id bd6mr23694735pab.184.1366995028761; Fri, 26 Apr 2013 09:50:28 -0700 (PDT) Received: from [10.4.0.18] (fw01.cghub.ucsc.edu. [192.35.223.10]) by mx.google.com with ESMTPSA id zy5sm3721842pbb.43.2013.04.26.09.50.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 09:50:27 -0700 (PDT) Message-ID: <517AB053.9070505@soe.ucsc.edu> Date: Fri, 26 Apr 2013 09:50:27 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Ryan Stone Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQlpiVWAtvdALWZJZ3k/HBTsbGt6ipEfboLm1QboyTSPJFFTKAjQeUSM0BIPRMs7GM3la4Er Cc: Paul Tatarsky , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:50:29 -0000 > In other words, until I see like 100% system usage in one core, I > would have room to grow? > > > Probably not. Mutexes in FreeBSD use "adaptive spinning". This means > that when a thread is unable to acquire a mutex, if the owner of the > mutex is still running on another CPU core then the thread will spin and > wait for the mutex to become free rather than block. This improves the > latency of acquiring the mutex and saves us many expensive calls through > the scheduler, but the downside is that you have one heavily contended > mutex you can have many cores wasted spinning on the lock. In your case > it is likely that your interrupt threads are spinning on the pf lock. > You can partially confirm this by profiling the system. The quick way is: Truly fascinating! So my cores that are sucking up CPU on interrupts may be just "spinning" waiting for a lock and not doing real work? That would explain a lot. > kldload hwpmc > pmcstat -S unhalted-cycles -T > (if unhalted-cycles does not work, try instructions). We'll try this. It doesn't look like the hwpmc module exists on this box but we'll bring it in and try that. > If _mtx_lock_sleep is the top function then you are hitting mutex > contention and more cores are unlikely to help. In this case you might > actually be able to improve performance by *reducing* the number of > threads that are contending in pf, for example by using fewer receive > queues in your NIC. > > If this machine is dedicated for pf then setting sysctl net.isr.direct=0 > might also improve performance, by forcing all packets to go through a > single netisr thread (assuming that net.isr.maxthreads is 1). Note that > this will apply to traffic that does not go through pf, so if this > machine were doing other network things (e.g. serving NFS) then it would > bottleneck your other traffic behind your pf traffic. It's only doing pf stuff so this is something we'll try. Thanks for the enlightening information!! From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 17:16:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 235525A7; Fri, 26 Apr 2013 17:16:36 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 8A20013A2; Fri, 26 Apr 2013 17:16:35 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id r3so3819572wey.17 for ; Fri, 26 Apr 2013 10:16:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=SupAv+7+37CGo3WR+bcT4XVJn8rEHO1sCuFWOdgue1o=; b=glmQAgmpD9M7lO0xA3hIF3x6ibLbfKJKepPfCmEFCQw8GhLF6dQbr3D7uPOWYEvHNq lbILYSzkUmyIh8pubvozdmcUTAq4DcdqtzZu4RJwx+urtZi/ixL8MHqpijHivET+xb15 g8VLG23bllc3HBXswvDVE6F9uIPQ8YHzlFY9jkNAhWBPV8DSFnBTCOnwhUfd7DnSNSh+ MLK4LE2m04kaFkkLV8kolQusZptyCUdeAmCH2UyXotW+qZT3m3uwu1Nb4XinUbLDnlh8 tHG1UBDSnuQ/3vmpxjrNEe1OKvxsHHyTEZRqJygztFvXhvJbfv22laO3Hh2SoTy+Ka09 QSFw== MIME-Version: 1.0 X-Received: by 10.180.87.170 with SMTP id az10mr5523601wib.3.1366996594729; Fri, 26 Apr 2013 10:16:34 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Fri, 26 Apr 2013 10:16:34 -0700 (PDT) In-Reply-To: References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> Date: Fri, 26 Apr 2013 10:16:34 -0700 X-Google-Sender-Auth: VgPi071wJEKUvXvKL3wFiHOjfhw Message-ID: Subject: Re: pf performance? From: Adrian Chadd To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 17:16:36 -0000 On 26 April 2013 09:22, Olivier Cochard-Labb=E9 wrote: > On Fri, Apr 26, 2013 at 3:42 PM, Gleb Smirnoff wrot= e: >> >> In FreeBSD 10 pf is no longer under single lock. On your hardware, >> I'd expect a measurable performance gain if you migrate to 10. > > Compairing 9.1 and current (249908) on my new test-server (HP ProLiant > DL320 G5, dual-core Xeon 3050, dual Intel NIC). > Like usual: one unidirectional flow of small packets, values in > packet-per-seconds: > > x 9.1 > + current > N Min Max Median Avg Stdd= ev > x 5 379991 381508 381229 380892.6 667.699= 26 > + 5 332833 335502 334726 334223.2 1142.82= 66 > Difference at 95.0% confidence > -46669.4 +/- 1364.98 > -12.2526% +/- 0.358363% > (Student's t, pooled s =3D 935.915) Do you have witness, etc enabled in -current? adrian From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 17:19:31 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 86753670; Fri, 26 Apr 2013 17:19:31 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 5217D13C0; Fri, 26 Apr 2013 17:19:31 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r3QHJUfa085829; Fri, 26 Apr 2013 13:19:30 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <517AB733.7020302@sentex.net> Date: Fri, 26 Apr 2013 13:19:47 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Olivier_Cochard-Labb=E9?= Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 17:19:31 -0000 On 4/26/2013 12:22 PM, Olivier Cochard-Labbé wrote: > On Fri, Apr 26, 2013 at 3:42 PM, Gleb Smirnoff wrote: >> >> In FreeBSD 10 pf is no longer under single lock. On your hardware, >> I'd expect a measurable performance gain if you migrate to 10. > > Compairing 9.1 and current (249908) on my new test-server (HP ProLiant > DL320 G5, dual-core Xeon 3050, dual Intel NIC). > Like usual: one unidirectional flow of small packets, values in > packet-per-seconds: > > x 9.1 > + current > N Min Max Median Avg Stddev > x 5 379991 381508 381229 380892.6 667.69926 > + 5 332833 335502 334726 334223.2 1142.8266 > Difference at 95.0% confidence > -46669.4 +/- 1364.98 > -12.2526% +/- 0.358363% > (Student's t, pooled s = 935.915) Is that because pf is slower on a single flow, or packet forwarding in general is slower on HEAD ? How different is 9.1 and HEAD in just forwarding performance? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 17:25:41 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 635B18DC; Fri, 26 Apr 2013 17:25:41 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-ve0-x236.google.com (mail-ve0-x236.google.com [IPv6:2607:f8b0:400c:c01::236]) by mx1.freebsd.org (Postfix) with ESMTP id 0338D144D; Fri, 26 Apr 2013 17:25:40 +0000 (UTC) Received: by mail-ve0-f182.google.com with SMTP id da11so1984494veb.13 for ; Fri, 26 Apr 2013 10:25:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=hnDiUcR80vL3JND3A0ki8UgO2K8A9Q6cTecLAlP9SZw=; b=DvEpzoiAWchqo9sPPD4Lt4mQyTNbsp2VllpGDPO80Gy+fwx1YLHrNfsRPMF4RAbTN2 49KkS0IKK9cVxdmQNFy5EAnf64ayy/k8xr2mg0FwSRK628nI6hvoW1WgjzX6GdHimReP 07oXngaxqcjZFYmadni2bZlH5mXBx8O5M1O6OkDkZfeQ0Zpns4qxHmCCzxqOELz0NrDV Pcij5t9/EjIlC3Qv4ppvrfygFcI0kFZapELp8h6wLQ7ZW6r6r0BDyhzzG99LUwf0dGkl CJQ6D6N5ssXDKOYFZEgjKgJ3OoVYC/Ntc0hPWti7eht3bcAroXxwfy5RZpgulBDPpCHk GLuw== X-Received: by 10.58.143.100 with SMTP id sd4mr29517839veb.39.1366997140475; Fri, 26 Apr 2013 10:25:40 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.59.9.103 with HTTP; Fri, 26 Apr 2013 10:25:19 -0700 (PDT) In-Reply-To: References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Fri, 26 Apr 2013 19:25:19 +0200 X-Google-Sender-Auth: Fch4efK8YIgCDn43SAop0-eLRHk Message-ID: Subject: Re: pf performance? To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 17:25:41 -0000 On Fri, Apr 26, 2013 at 7:16 PM, Adrian Chadd wrote: > Do you have witness, etc enabled in -current? > Hi Adrian, Of course not :-) The src.conf have: MALLOC_PRODUCTION= And the kernel have: include GENERIC nomakeoption DEBUG nooptions DDB nooptions GDB nooptions DEADLKRES nooptions INVARIANTS nooptions INVARIANT_SUPPORT nooptions WITNESS nooptions WITNESS_SKIPSPIN nooptions MALLOC_DEBUG_MAXZONE Regards, From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 17:40:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8C99370 for ; Fri, 26 Apr 2013 17:40:05 +0000 (UTC) (envelope-from weiler@soe.ucsc.edu) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) by mx1.freebsd.org (Postfix) with ESMTP id 634291527 for ; Fri, 26 Apr 2013 17:40:05 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id bi5so2628266pad.31 for ; Fri, 26 Apr 2013 10:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=VfmdXg3eyrFP/5MUZXmvCCl7/dwKjZnMEMYrVATniJg=; b=b6B2QkGosKkXNNrKtbt8d/SoD+SuvLOiSbwSV6qsaFJS5DQEkSFM0Qh2lDGyLksBDv //Uzkv/Ih7+P2QyRgAqN/JlKmxFPOdXhopgqVa8lM7yhAzfjKsjRWUBK+fQHlPQQVL33 +3yaAnBppQcqPxo8yOEN230XWWuL6Hy6Yclro= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding:x-gm-message-state; bh=VfmdXg3eyrFP/5MUZXmvCCl7/dwKjZnMEMYrVATniJg=; b=iCCgWoIBvvijWAp0Asf4pA8uVxzehJ6ZtXEQ3XQjR2hkIj2UhmtxxGiRU5bnxLt1Yi m4Uc0+hzb3/Epe/Hr6q1g3D13aSVVXB1dP9qJvThucZkkxyhqPRZh+n8baSR23PWd8yY YvhklagqgHKQEMyULb6KUJz+xHEz2iIYJndAUsatc1ebM2RGuRHiDvd9bokuTGyIW5d8 8fyukxDMCTCTlXMV/7KchWgU0MNFH7E1G82oln5YxMkY9Q9ykOBbvL1X7EC7GvmE9etD XvWhwBjCxbaG0X0SuIUNJUwjncy9Z+nX5RxzI60NUSZYkHtoQ5PU8KlLu75EI6fiZXWZ iwzw== X-Received: by 10.66.26.166 with SMTP id m6mr1185144pag.68.1366998004970; Fri, 26 Apr 2013 10:40:04 -0700 (PDT) Received: from [10.4.0.18] (fw01.cghub.ucsc.edu. [192.35.223.10]) by mx.google.com with ESMTPSA id ra9sm13597403pab.16.2013.04.26.10.40.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Apr 2013 10:40:03 -0700 (PDT) Message-ID: <517ABBF3.3050001@soe.ucsc.edu> Date: Fri, 26 Apr 2013 10:40:03 -0700 From: Erich Weiler User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Ryan Stone Subject: Re: pf performance? References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <517A93FE.7020209@soe.ucsc.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQkdutliSyea6egdL/nFhg5SN9f3bspPsN9O+zOFQzLgmZJiCebJg3pc6991W2ti8DA3yRv7 Cc: Paul Tatarsky , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 17:40:05 -0000 > If this machine is dedicated for pf then setting sysctl net.isr.direct=0 > might also improve performance, by forcing all packets to go through a > single netisr thread (assuming that net.isr.maxthreads is 1). Note that > this will apply to traffic that does not go through pf, so if this > machine were doing other network things (e.g. serving NFS) then it would > bottleneck your other traffic behind your pf traffic. Hmm, we see this: [2.0.2-RELEASE][admin@fire02.cghub]/root(1): sysctl net.isr.direct net.isr.direct: 1 [2.0.2-RELEASE][admin@fire02.cghub]/root(2): sysctl net.isr.maxthreads net.isr.maxthreads: 3 Should I set net.isr.maxthreads=1 before setting net.isr.direct=0 ?? From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 17:50:03 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 36B8979B for ; Fri, 26 Apr 2013 17:50:03 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x230.google.com (mail-bk0-x230.google.com [IPv6:2a00:1450:4008:c01::230]) by mx1.freebsd.org (Postfix) with ESMTP id BF71215C0 for ; Fri, 26 Apr 2013 17:50:02 +0000 (UTC) Received: by mail-bk0-f48.google.com with SMTP id it19so593898bkc.21 for ; Fri, 26 Apr 2013 10:50:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=n5WSIHshJSCB6rg6q/LqQ/nc0tzhkDJtBXnAuf8VFRU=; b=a1DaZiM7DnlmfylZehm5SyP9VOJ4U2pTHqVv1fDcbIhT+E4cbaT4I/iGHF+8hrhFnZ T4a7ScCinlJ9Ey3+NkRep2x/TUX5SoRXoze/kZ3A6gV85u51kXfdEoZVAM+HTkJvmcSZ uTfApsDeNALP9phzWKn+fkU0fBUsh/MG9zk6713/WqaxPqioAdHAmY8NRnOG1vQtODgN jvA1pGE9UdHusuyBlJLQ9l+ejWQxjsjuGJpS95FsdH08pCHOKnepsDI+X2uH1yIk4umH GXH5ZgBVY5XSToOCzH2Bxvw4vYHiDpX7/q/ir+lds32FdQwgMSqf/7iDCqsZ0PODZ9Pz wMGw== X-Received: by 10.204.226.80 with SMTP id iv16mr10451399bkb.48.1366998600863; Fri, 26 Apr 2013 10:50:00 -0700 (PDT) Received: from zvezda.localnet ([2a02:8108:1440:5b:2677:3ff:fe7b:7648]) by mx.google.com with ESMTPSA id w6sm3680999bkz.17.2013.04.26.10.49.59 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 26 Apr 2013 10:50:00 -0700 (PDT) From: Kajetan Staszkiewicz To: Erich Weiler Subject: Re: pf performance? Date: Fri, 26 Apr 2013 19:49:59 +0200 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <5176E5C1.9090601@soe.ucsc.edu> <201304260021.11209.vegeta@tuxpowered.net> <5179B3BB.3070101@soe.ucsc.edu> In-Reply-To: <5179B3BB.3070101@soe.ucsc.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201304261949.59317.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQnYRRfcJm/pQUdInPSHho1anB+y/RvaBZy1ShIi1TByQ0PXwvlYfsFHxcuhlcolttQp0z0g Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 17:50:03 -0000 Dnia pi=C4=85tek, 26 kwietnia 2013 o 00:52:43 Erich Weiler napisa=C5=82(a): > > How many pf rules do you have?. And, as I asked in my previous post, do > > you create states on both sides of the firewall? >=20 > One interface has 12 rules and other other interface has one rule. We > do create states on both sides. That's not too many rules, but are you sure you also create states for=20 "postrouting" traffic? When you do "pass (quick) in on $public some other=20 conditions", you also should have a general "pass quick out on $internal" (= and=20 vice versa), as close to the top of pf.conf, of course unless you need sepa= rate=20 pre and post routing pf filtering rules. =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Fri Apr 26 19:50:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4A714446 for ; Fri, 26 Apr 2013 19:50:21 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-bk0-x22c.google.com (mail-bk0-x22c.google.com [IPv6:2a00:1450:4008:c01::22c]) by mx1.freebsd.org (Postfix) with ESMTP id D29121B20 for ; Fri, 26 Apr 2013 19:50:20 +0000 (UTC) Received: by mail-bk0-f44.google.com with SMTP id jk14so1810096bkc.3 for ; Fri, 26 Apr 2013 12:50:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=C64S8wnE3fhpfQ4ejBVUbJIbOZOvVrcLKwwzHJPwF4g=; b=HOjXIPT27sQ6wspZVpezD27O039pB7nCyGDvzKU8bgAkxy4bnNcuh3ru2TvrSyy9XO SVv0fanfavEZVoYmxna1fODEq5NLNjfk/NiQH0QePSjWuX2m/HkNfBqcqJsyvmT29VM1 xtcqykS/OnQcsauZ5LXI/LXB11dzsHWofgEXEufxEhzaYaBAVUFhlPIYNEDJ6fGypK0X vcpFMcgvAcyKNeWQuAq6T+ziyC9mlYHoMN6qhX4DvpomlpFsptMwKeOjr3xCwtvPpJ/Z T9IC6UcewuJpkNb2RmqpREHUy8uwGVfV/ecNbd5ut3HiYVD/2dcDA/1fh3MGNuB+w5Cw oSdA== X-Received: by 10.205.108.137 with SMTP id ec9mr19169818bkc.6.1367005818856; Fri, 26 Apr 2013 12:50:18 -0700 (PDT) Received: from zvezda.localnet ([2a02:8108:1440:5b:2677:3ff:fe7b:7648]) by mx.google.com with ESMTPSA id jz9sm3811507bkb.1.2013.04.26.12.50.17 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 26 Apr 2013 12:50:18 -0700 (PDT) From: Kajetan Staszkiewicz To: Erich Weiler Subject: Re: pf performance? Date: Fri, 26 Apr 2013 21:50:17 +0200 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <5176E5C1.9090601@soe.ucsc.edu> <517974DA.5090809@soe.ucsc.edu> <201304260021.11209.vegeta@tuxpowered.net> In-Reply-To: <201304260021.11209.vegeta@tuxpowered.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201304262150.17215.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQlSf93+9iOYI6ggWCWFVirOP0wqjAoB1vnel9sG52fZcMg6qlLmLCS2uzGIWdfIYrzyutXF Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 19:50:21 -0000 Dnia pi=C4=85tek, 26 kwietnia 2013 o 00:21:11 Kajetan Staszkiewicz napisa= =C5=82(a): > > > How do you count the 140kpps value? One interface, both, in, out? I'd > > > like to relate this somehow to my values. > >=20 > > Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx > > spike to 150kpps occasionally. >=20 > Unfortunately at this moment I have no single machine with such traffic, > although maybe I can aggregate some traffic later and check the cpu usage > then. OK, got my CPU usage for 30/40kpps, 55/340Mbit/s (public side, in/out). CPU is 4-core E5540 @ 2.53GHz with HT disabled. Cpu usage looks more or less like this: CPU 0: 0.0% user, 0.0% nice, 0.4% system, 3.9% interrupt, 95.7% idle CPU 1: 0.0% user, 0.0% nice, 1.2% system, 18.1% interrupt, 80.7% idle CPU 2: 0.0% user, 0.0% nice, 1.6% system, 5.5% interrupt, 92.9% idle CPU 3: 0.0% user, 0.0% nice, 0.8% system, 26.8% interrupt, 72.4% idle Public network card is pinned to cpu 2, internal to cpu 3, each card has on= ly a=20 single irq. Netisr threads are limited to cpu 0 and 1, I use deferred netis= r.=20 So yes, I have 2x less pps than you, but also I have quite a slower cpu and= =20 there still seems to be much cpu power left. How many interrupts/s do you have? What about the number of states? =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 03:01:53 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C0EB5A74 for ; Sat, 27 Apr 2013 03:01:53 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm8-vm4.bullet.mail.ne1.yahoo.com (nm8-vm4.bullet.mail.ne1.yahoo.com [98.138.91.168]) by mx1.freebsd.org (Postfix) with ESMTP id 8F3621930 for ; Sat, 27 Apr 2013 03:01:53 +0000 (UTC) Received: from [98.138.90.53] by nm8.bullet.mail.ne1.yahoo.com with NNFMP; 27 Apr 2013 03:01:47 -0000 Received: from [98.138.87.7] by tm6.bullet.mail.ne1.yahoo.com with NNFMP; 27 Apr 2013 03:01:46 -0000 Received: from [127.0.0.1] by omp1007.mail.ne1.yahoo.com with NNFMP; 27 Apr 2013 03:01:46 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 979025.68748.bm@omp1007.mail.ne1.yahoo.com Received: (qmail 99115 invoked by uid 60001); 27 Apr 2013 03:01:46 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1367031706; bh=jxvnoMAZmdpQYBCUvWh/R/ZKlJ6VuiHzpbLzVTooWhA=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=w0Ri4XR/zp2qUXUHL+VNXYOsFi/jyInnvNDaCROqCFySZAFF00LaHNfsfiRqKkiCF0l/snkMmbl3hB0zzaF/12kFNLPvLA5lCoBihnLR/FCi++gb5AyvnodRzb48LGF+FC7zKKST64fvR7FGUYXgBVvudBG/wo9Ry6UE53EsjzM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=v4U5km0DmIqYk9ANETPR/WJphOQYEEnBoUH4n2TV5pC5TgGk3xWkW7eNiMfTY0xBjWNGB1MyRTITUs0eDZd0LwIPfG22saRkqOtYzmoV49FlAha2GQAD/egQwRBdmyD6IVH6uEnDFfqz62yeDU+j+HWuOPjJvsgV940S3M7Ug3Q=; X-YMail-OSG: wZaFKoYVM1lFAabP.jPB8oN3mw37NbR9DjUg5pjPk58nS.5 I0OVXtmfyOLenIZGD1Cu1BZElDWhwWOen3YSwDpuJd3tIpizQknIVOfSk4fX O.KQ4QjE0Rl5L.J2qz5O6cxHl9frehmw1TVkLZ_4NZhTsyuKsx29KDsljiKk i.lTDVWXNwGUfJCWzPriyE9brZJhdZNn1FfmF.dbeiaNenlD2u_89UVKrgQl YVhOZERSiNo9aAOZjDWc8Am2A.F7Wxwt5m3.bOxF_gyyqEYdbbuDKacXMxZZ .keA4.nWoexIQodNm36bcsMRU_59CjU1_zaZdgMTjNKq6ka3pKIkXJxsiQic m6YJBUofQjJJBQrFQ6WzAIk1dK8nd74pxWO.s4JY.8y27Bee42b_1tqFl9fV QGusiJr_DA.wFNlJGznHW74i9N5Njkdw5gu0UjNbfPhiDnJJwL88sUt8vCh1 A6OT4ZKocX.5tW6FufmEA4CxRinXMf7PFdMmpj4kofnthnA.eRGD3m2.Dgi3 ZyUoO6Ps- Received: from [98.203.118.124] by web121602.mail.ne1.yahoo.com via HTTP; Fri, 26 Apr 2013 20:01:46 PDT X-Rocket-MIMEInfo: 002.001, DQotLS0gT24gRnJpLCA0LzI2LzEzLCBFcmljaCBXZWlsZXIgPHdlaWxlckBzb2UudWNzYy5lZHU.IHdyb3RlOg0KDQo.IEZyb206IEVyaWNoIFdlaWxlciA8d2VpbGVyQHNvZS51Y3NjLmVkdT4NCj4gU3ViamVjdDogUmU6IHBmIHBlcmZvcm1hbmNlPw0KPiBUbzogIkFuZHJlIE9wcGVybWFubiIgPGFuZHJlQGZyZWVic2Qub3JnPg0KPiBDYzogIlBhdWwgVGF0YXJza3kiIDxwYXVsQHNvZS51Y3NjLmVkdT4sIGZyZWVic2QtbmV0QGZyZWVic2Qub3JnDQo.IERhdGU6IEZyaWRheSwgQXByaWwgMjYsIDIwMTMsIDEBMAEBAQE- X-Mailer: YahooMailClassic/15.1.8 YahooMailWebService/0.8.141.536 Message-ID: <1367031706.98947.YahooMailClassic@web121602.mail.ne1.yahoo.com> Date: Fri, 26 Apr 2013 20:01:46 -0700 (PDT) From: Barney Cordoba Subject: Re: pf performance? To: Andre Oppermann , Erich Weiler MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Paul Tatarsky , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 03:01:53 -0000 --- On Fri, 4/26/13, Erich Weiler wrote: > From: Erich Weiler > Subject: Re: pf performance? > To: "Andre Oppermann" > Cc: "Paul Tatarsky" , freebsd-net@freebsd.org > Date: Friday, April 26, 2013, 12:04 PM > >> But the work pf does would > show up in 'system' on top right?=A0 So if I > >> see all my CPUs tied up 100% > >> in 'interrupts' and very little 'system', would it > be a reasonable > >> assumption to think that if I got > >> more CPU cores to handle the interrupts that > eventually I would see > >> 'system' load increase as the > >> interrupt load became faster to be handled?=A0 > And thus increase my > >> bandwidth? > >=20 > > Having the work of pf show up in 'interrupts' or > 'system' depends on the > > network driver and how it handles sending packets up > the stack.=A0 In most > > cases drivers deliver packets from interrupt context. >=20 > Ah, I see.=A0 Definitely appears for me in interrupts > then.=A0 I've got the mxge driver doing the work > here.=A0 So, given that I can spread out the interrupts > to every core (like, pin an interrupt queue to each core), I > can have all my cores work on the process.=A0 But seeing > as though the pf bit is still serialized I'm not sure that I > understand how it is serialized when many CPUs are handling > interrupts, and hence doing the work of pf as well?=A0 > Wouldn't that indicate that the work of pf is being handled > by many cores, as many cores are handling the interrupts? >=20 you're thinking exactly backwards. You're creating lock contention by having a bunch of receive processes going into a single threaded pf process. Think of it like a six lane highway that has 5 lanes closed a mile up the road. The result isn't that you go the same speed as a 1 lane highway; what you have is a parking lot. The only thing you're doing by spreading the interrupts is using up more cycles on more cores. What you *should* be doing, if you can engineer it, is use 1 path through the pf filter. You could have 4 queues feed a single process that dequeues and runs through the filter. The problem with that is that the pf process IS the bottleneck in that its slower than the receive processes, so you'd best just use the other cores to do userland stuff. You could use cpuset to make sure that no userland process uses the interrupt core, and dedicate 1 cpu to packet filtering. 1 modern CPU can easily handle a gig of traffic. There's no need to spread in most case. BC > Or are you saying that pf *is* being handled by many cores, > but just in a serialized nature?=A0 Like, packet 1 is > handled by core 0, then packet 2 is handled by core 1, > packet 3 is handled by core 4, etc?=A0 Such that even > though multiple cores are handling it, they are just doing > so serially and not in parallel?=A0 And if so, maybe it > still helps to have many cores? >=20 > Thanks for all the excellent info! >=20 > >> In other words, until I see like 100% system usage > in one core, I > >> would have room to grow? > >=20 > > You have room to grow if 'idle' is more than 0% and the > interrupts of > > the networks cards are running on different > cores.=A0 If one core gets > > all the interrupts a second idle core doesn't get the > chance to help > > out.=A0 IIRC the interrupt allocation to cores is > done at interrupt > > registration time or driver attach time.=A0 It can > be re-distributed > > at run time on most architecture but I'm not sure we > have an easily > > accessible API for that. From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 05:53:52 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E9AA4820 for ; Sat, 27 Apr 2013 05:53:52 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 5E71E1F43 for ; Sat, 27 Apr 2013 05:53:51 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3R5rneJ080911; Sat, 27 Apr 2013 09:53:49 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3R5rn2q080910; Sat, 27 Apr 2013 09:53:49 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 27 Apr 2013 09:53:49 +0400 From: Gleb Smirnoff To: Olivier Cochard-Labb? Subject: Re: pf performance? Message-ID: <20130427055349.GW76816@glebius.int.ru> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 05:53:53 -0000 On Fri, Apr 26, 2013 at 06:22:18PM +0200, Olivier Cochard-Labb? wrote: O> > In FreeBSD 10 pf is no longer under single lock. On your hardware, O> > I'd expect a measurable performance gain if you migrate to 10. O> O> Compairing 9.1 and current (249908) on my new test-server (HP ProLiant O> DL320 G5, dual-core Xeon 3050, dual Intel NIC). O> Like usual: one unidirectional flow of small packets, values in O> packet-per-seconds: O> O> x 9.1 O> + current O> N Min Max Median Avg Stddev O> x 5 379991 381508 381229 380892.6 667.69926 O> + 5 332833 335502 334726 334223.2 1142.8266 O> Difference at 95.0% confidence O> -46669.4 +/- 1364.98 O> -12.2526% +/- 0.358363% O> (Student's t, pooled s = 935.915) As I already mentioned this is expected and okay result. With an empty state table you've got a fast pf processing, threads do not spend a lot of time in pf, so probability of contention is low, even in case of single lock. Not speaking that you got only 2 cores. In the new pf in 10 we need to do two atomic operations per packet: read-lock the global pf rwlock, then acquire hash slot mutex. While in old pf we only acquired the single pf mutex. So in case when state table is 1 state it is expected that old pf can outperform new one, due to cheaper locking. Not speaking that probability of outperforming is the more the less cores you got. You got only 2. This is not the case the new pf was coded for. But the setup the Erich is running is the case. We probably can get more performance out of new pf simply converting the rwlock to rmlock, may be we will get these 12% in vacuous test back. But I'd like someone with decent hardware and traffic to test that first. I don't want to do this convertsion blindly w/o benchmark and stability test. Unfortunately, as you see, most people avoid running head, waiting at least for 10.0-RELEASE, or even for pfSense catching up on FreeBSD 10. So probably this change won't be tested soon, and thus won't happen soon, -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 08:53:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D525D77F; Sat, 27 Apr 2013 08:53:12 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 99A001336; Sat, 27 Apr 2013 08:53:12 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:c1ca:3dbc:5137:8c40]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id A7A354AC57; Sat, 27 Apr 2013 12:53:10 +0400 (MSK) Date: Sat, 27 Apr 2013 12:53:03 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <15810136646.20130427125303@serebryakov.spb.ru> To: Gleb Smirnoff Subject: Re: pf performance? In-Reply-To: <20130427055349.GW76816@glebius.int.ru> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <20130427055349.GW76816@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Olivier Cochard-Labb? , "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 08:53:12 -0000 Hello, Gleb. You wrote 27 =D0=B0=D0=BF=D1=80=D0=B5=D0=BB=D1=8F 2013 =D0=B3., 9:53:49: GS> On Fri, Apr 26, 2013 at 06:22:18PM +0200, Olivier Cochard-Labb? wrote: GS> But I'd like someone with decent hardware and traffic to test that firs= t. I GS> don't want to do this convertsion blindly w/o benchmark and stability t= est. GS> Unfortunately, as you see, most people avoid running head, waiting at l= east GS> for 10.0-RELEASE, or even for pfSense catching up on FreeBSD 10. So pro= bably GS> this change won't be tested soon, and thus won't happen soon, I'm running fresh -HEAD on my home router, but my hardware is not "decent" (Atom D2500) :( --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 13:03:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AD732323 for ; Sat, 27 Apr 2013 13:03:12 +0000 (UTC) (envelope-from jim@netgate.com) Received: from mail-oa0-f44.google.com (mail-oa0-f44.google.com [209.85.219.44]) by mx1.freebsd.org (Postfix) with ESMTP id 783191930 for ; Sat, 27 Apr 2013 13:03:11 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id h1so4837237oag.3 for ; Sat, 27 Apr 2013 06:03:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to:x-gm-message-state; bh=9x0Rq83bvFQI+cv5WM2uAl1S3+Wx5ftO3KCm5V+PuXM=; b=FNa7Gve52Eq1zl9ZxPAGFINiuuxxCLLE9hPm4oh5zOy6uRUBOm0emYIUbLEcuEwJnZ rX34FdVHj8x3M8c2hA4yco/XpbXQ+gYoHd/F8D0bwvfNthxOaioJNenmXTX6TyXYB4bE x8tO8tspZfKaZxtJhFdubtJGOGa8pMG3Z7ea3w4aDkTN2nn1EMFbjrmTXDipkSTiywic h9/u55Dh1Udt30roLeUs2eLu19x+A6RsdOMzA4hQKSd+9yIUbioGnol0NXIh8vuHmLbw o5AMsig5kvT0XV9jKQ4aLPRy/WzDD1oJbFju8x6Z86CrEqR2cbj4zq9cf8g2NcuARJwW zbYA== X-Received: by 10.60.97.170 with SMTP id eb10mr20335388oeb.2.1367067790611; Sat, 27 Apr 2013 06:03:10 -0700 (PDT) Received: from [29.9.160.101] (66-87-72-101.pools.spcsdns.net. [66.87.72.101]) by mx.google.com with ESMTPSA id jw8sm12015969obb.14.2013.04.27.06.03.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 27 Apr 2013 06:03:09 -0700 (PDT) References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <20130427055349.GW76816@glebius.int.ru> Mime-Version: 1.0 (1.0) In-Reply-To: <20130427055349.GW76816@glebius.int.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10B329) From: Jim Thompson Subject: Re: pf performance? Date: Sat, 27 Apr 2013 07:58:04 -0500 To: Gleb Smirnoff X-Gm-Message-State: ALoCoQl2AekI2Lb4IUzxX99AscaOdcZT0IKfoXUt3cuc2dBWwW9T9utOWEAsnzmnJPJGTFC+ATft Cc: Olivier Cochard-Labb? , "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 13:03:12 -0000 On Apr 27, 2013, at 12:53 AM, Gleb Smirnoff wrote: > Unfortunately, as you see, most people avoid running head, waiting at leas= t for 10.0-RELEASE, or even for pfSense catching up on FreeBSD 10. So probab= ly this change won't be tested soon, and thus won't happen soon, Gleb,=20 As a minor part of the pfSense team, I believe you are mistaken.=20 I'm out of the office right now, but when I return, I'd already planned to d= uplicate the test in a test harness with several multi-core boxes acting as s= ource & sink, and the DUT to include several popular platforms for pfSense r= unning a set of software including running same across -HEAD, 9-STABLE, and 8= .3, both with the pfSense patches (as pfSense), and without.=20 I doubt I'll get this done prior to BSDcan, but I'll get it done, if only fo= r internal reasons. =20 Jim From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 15:14:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EA685FDC for ; Sat, 27 Apr 2013 15:14:45 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 773C71DAD for ; Sat, 27 Apr 2013 15:14:44 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3RFEYAa082441; Sat, 27 Apr 2013 19:14:34 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3RFEYlF082440; Sat, 27 Apr 2013 19:14:34 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 27 Apr 2013 19:14:34 +0400 From: Gleb Smirnoff To: Jim Thompson Subject: Re: pf performance? Message-ID: <20130427151434.GA76816@glebius.int.ru> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> <20130427055349.GW76816@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Olivier Cochard-Labb? , "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 15:14:46 -0000 Jim, On Sat, Apr 27, 2013 at 07:58:04AM -0500, Jim Thompson wrote: J> > Unfortunately, as you see, most people avoid running head, waiting at least for 10.0-RELEASE, or even for pfSense catching up on FreeBSD 10. So probably this change won't be tested soon, and thus won't happen soon, J> As a minor part of the pfSense team, I believe you are mistaken. J> J> I'm out of the office right now, but when I return, I'd already planned to duplicate the test in a test harness with several multi-core boxes acting as source & sink, and the DUT to include several popular platforms for pfSense running a set of software including running same across -HEAD, 9-STABLE, and 8.3, both with the pfSense patches (as pfSense), and without. J> J> I doubt I'll get this done prior to BSDcan, but I'll get it done, if only for internal reasons. Good news! Thanks, Jim! -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 15:56:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 44D8C79 for ; Sat, 27 Apr 2013 15:56:22 +0000 (UTC) (envelope-from bredehorn@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mx1.freebsd.org (Postfix) with ESMTP id D90591F30 for ; Sat, 27 Apr 2013 15:56:21 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.24]) by mrigmx.server.lan (mrigmx001) with ESMTP (Nemesis) id 0LjP2f-1V3GFH3VD5-00da3N for ; Sat, 27 Apr 2013 17:56:20 +0200 Received: (qmail invoked by alias); 27 Apr 2013 15:56:20 -0000 Received: from p57BD33BB.dip0.t-ipconnect.de (EHLO [192.168.178.30]) [87.189.51.187] by mail.gmx.net (mp024) with SMTP; 27 Apr 2013 17:56:20 +0200 X-Authenticated: #168415 X-Provags-ID: V01U2FsdGVkX19CzgdFU+pC/t5qEsfMJib4rDaTRJq0/EquQxq6Nu AQta5dxXz9KiX1 Message-ID: <517BF523.6010804@gmx.de> Date: Sat, 27 Apr 2013 17:56:19 +0200 From: Rainer Bredehorn User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jason Fesler , freebsd-net@FreeBSD.org Subject: Re: PF IPv6 fragment support References: In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 15:56:22 -0000 Hi Jason! Am 27.04.2013 03:39, schrieb Jason Fesler: > On Fri, Apr 26, 2013 at 1:26 AM, Rainer Bredehorn wrote: >> I've modified the kernel PF implementation to pass IPv6 fragments. >> The first fragment is handled by the PF rules of course ignoring possible checksums. > > Are you checking L4 before passing/not passing? What if the L4 header > is fragmented? Yes, when the L4 header is present it can be checked statefully. A fragment offset of zero indicates the precence off the upper layer header. A fragmented upper layer header is a problem. I think that could only be solved when the packets are reassembled. In my case it is not a big problem because I did some other modification like limiting the allowed number of extension headers. So a fragmented upper layer header should be a rare case. >> All other fragments are passed by PF to the IP stack. >> This can be done state-full but reassembling fragments is not supported. > > Reassembling packets will allow full L4 checking. Correct but it didn't work for IPv6 in FreeBSD 8.3. Reassembling is not my favorite. I don't want to buffer network packets due to performance reasons. Rainer.