From owner-freebsd-ipfw@freebsd.org Mon Apr 18 11:31:07 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A15C9B1149F; Mon, 18 Apr 2016 11:31:07 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport2.cc.swin.edu.au (iport2.cc.swin.edu.au [136.186.0.52]) by mx1.freebsd.org (Postfix) with ESMTP id 17F261D63; Mon, 18 Apr 2016 11:31:06 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.24,502,1454936400"; d="scan'208";a="19047243" Received: from gsp-ex01.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.17]) by iport2.cc.swin.edu.au with ESMTP; 18 Apr 2016 21:29:57 +1000 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.170]) by gsp-ex01.ds.swin.edu.au ([169.254.1.160]) with mapi id 14.03.0294.000; Mon, 18 Apr 2016 21:29:56 +1000 From: Rasool Al-Saadi To: "aqm@ietf.org" , "freebsd-net@freebsd.org" , "freebsd-ipfw@freebsd.org" CC: Grenville Armitage Subject: Dummynet AQM v0.2- CoDel, FQ-CoDel, PIE and FQ-PIE for FreeBSD's ipfw/dummynet Thread-Topic: Dummynet AQM v0.2- CoDel, FQ-CoDel, PIE and FQ-PIE for FreeBSD's ipfw/dummynet Thread-Index: AdGZZMR7qylo1SZqRpSWlM5D+/m1Cw== Date: Mon, 18 Apr 2016 11:29:55 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C206A7B@gsp-ex02.ds.swin.edu.au> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.126.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Apr 2016 11:31:07 -0000 Dear all, I would like to announce that we (myself and Grenville Armitage) released D= ummynet AQM v0.2, which is an independent implementation of CoDel, FQ-CoDel= PIE and FQ-PIE* for FreeBSD's ipfw/dummynet framework, based on the IETF = CoDel [1], FQ-CoDel [2] and PIE [3] Internet-Drafts. * "FlowQueue-PIE" (FQ-PIE) implementation combines FQ-CoDel's FlowQueuing w= ith PIE's individual queue management. In addition to PIE and FQ-PIE, this version includes bugs fixing and improv= ements to CoDel and FQ-CoDel code. We prepared patches for FreeBSD11-CURRENT-r297692 and FreeBSD 10.x-RELEASE= (10.0, 10.1, 10.2, 10.3), and a technical report of our implementation. Patches and documentation can be found in: http://caia.swin.edu.au/freebsd/aqm Technical report: http://caia.swin.edu.au/reports/160418A/CAIA-TR-160418A.pdf [1] "Controlled Delay Active Queue Management", https://tools.ietf.org/htm= l/draft-ietf-aqm-codel-03 [2] "FlowQueue-Codel" , https://tools.ietf.org/html/draft-ietf-aqm-fq-codel= -06 [3] "PIE: A Lightweight Control Scheme To Address the Bufferbloat Problem",= https://tools.ietf.org/html/draft-ietf-aqm-pie-06 Regards, Rasool Al-Saadi From owner-freebsd-ipfw@freebsd.org Tue Apr 19 01:56:32 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50D19B121B4 for ; Tue, 19 Apr 2016 01:56:32 +0000 (UTC) (envelope-from g_amanakis@yahoo.com) Received: from nm25-vm2.bullet.mail.ne1.yahoo.com (nm25-vm2.bullet.mail.ne1.yahoo.com [98.138.91.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E5961299 for ; Tue, 19 Apr 2016 01:56:31 +0000 (UTC) (envelope-from g_amanakis@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1461030860; bh=U/e7gxOQq/CJaUGqXeAl2bY7/xl8BsykGKNclhZg/jA=; h=Subject:From:Date:To:CC:From:Subject; b=OAW8z8UhBBc7LlLtr26/KrWqFjG17Sga/MLRZqY7uaDLRgCYnPpvXT50FkKx8hEUb/+fhTnXtOeG5SgtyFnCUH9Pn3atpWaL3d309TDn94NTAv1b38teyIqDVHx1ubvXFSZcPGLgEN2ckOslmzKgl3lSjWOoB+42OleKL6o07l5KEwwhGVqmfC6EoTlU2sHd+rgiuoCn0ZjGmikCcmEubQsX+XMO3U9rZtObV5ilP95Mb/bndaYx5vKtiyo04v4CWtgHSZV2qUFjWqhr6HRapbqxj/OCT/os14Jb7QxuOmPL25uIbgcPtQ91/hHY6zKPkMl4PPBDYrq6lEKIJVGsrw== Received: from [98.138.226.178] by nm25.bullet.mail.ne1.yahoo.com with NNFMP; 19 Apr 2016 01:54:20 -0000 Received: from [98.138.84.46] by tm13.bullet.mail.ne1.yahoo.com with NNFMP; 19 Apr 2016 01:54:20 -0000 Received: from [127.0.0.1] by smtp114.mail.ne1.yahoo.com with NNFMP; 19 Apr 2016 01:54:20 -0000 X-Yahoo-Newman-Id: 461763.99654.bm@smtp114.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: nkuWYQkVM1nbJ5kJAyEXw.jmzIdlFB.0XacJBqkJKLc2O_G R3DlzUEJ1Js7CvAJZf5HMq62OmDiyPC8Wx4Hsyrk461wZvX_9zOtyQgJPwMC G5.hCZ6wmqn1M6RF_nMx_i4wpY_.utgRIuzy9C0azk6uS_brGHGxbKsy273t DCvZTaAg583zMir9wdwWhOxICBof1lDWiI8Jlh9KrjZuURSIfk_DqL6QgrkB BMT784ojc5YJL.IZO4DIgOxFcT7rdH7dofZNUrtdlcuzz5Hh8RdJcLA_yIYg WQOSPyejJnKhgmeaEIAykp1Bn.2Tlzj4Ls_p_785hVZnJucg92Rrzkm7Pj8V 17recZPup5UA36qPcGBGS_49EoaJjvZZJWlWEwnIITNrbVfobPiFU.Ab9qxe gEUveYoZZJxGj7YCAvPs4pnlv8NB0Tv0OWQhEdr.clqX12vx1JuhCP.FhSto _WChskQJTnxkLZJpXb483fd.we5AmFwRA3IZyUioZI37qfX3sxVoKa0DLdxu AqWBTSEu0ZQr5yiQjOyJGSohD6sQBKbZ3 X-Yahoo-SMTP: 6sUo5IiswBDB2TZm6JKJ6DaI.Rsz4O0- User-Agent: K-9 Mail for Android MIME-Version: 1.0 Subject: Re: Dummynet AQM v0.2- CoDel, FQ-CoDel, PIE and FQ-PIE for FreeBSD's ipfw/dummynet From: "G. Amanakis" Date: Mon, 18 Apr 2016 21:54:18 -0400 To: freebsd-ipfw@freebsd.org CC: garmitage@swin.edu.au,ralsaadi@swin.edu.au Message-ID: <0CBE5C7D-83B0-45AC-82E5-5E49F5E3260A@yahoo.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2016 01:56:32 -0000 Thank you for your efforts, this is much appreciated=2E I am using v0=2E1 o= n 10=2E3-RELEASE on a SoHo server with no problems so far=2E To the develop= ers: any chance of including this in the upcoming 11=2E0-RELEASE? From owner-freebsd-ipfw@freebsd.org Wed Apr 20 06:08:38 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2340B15FF9 for ; Wed, 20 Apr 2016 06:08:38 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id E44CF1961; Wed, 20 Apr 2016 06:08:37 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.24,508,1454936400"; d="scan'208";a="19320593" Received: from gsp-ex01.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.17]) by iport1.cc.swin.edu.au with ESMTP; 20 Apr 2016 16:07:29 +1000 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.170]) by gsp-ex01.ds.swin.edu.au ([169.254.1.160]) with mapi id 14.03.0294.000; Wed, 20 Apr 2016 16:07:28 +1000 From: Rasool Al-Saadi To: Don Lewis CC: "freebsd-ipfw@FreeBSD.org" Subject: RE: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet Thread-Topic: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet Thread-Index: AdFwoEB9RcA1ON/lTEC2HgPx0zqG6gJqmosACB9U/dA= Date: Wed, 20 Apr 2016 06:07:28 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C20AEFC@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C187192@gsp-ex02.ds.swin.edu.au> <201603100828.u2A8Smbt013642@gw.catspoiler.org> In-Reply-To: <201603100828.u2A8Smbt013642@gw.catspoiler.org> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.112.109] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2016 06:08:38 -0000 > -----Original Message----- > From: Don Lewis [mailto:truckman@FreeBSD.org] > Sent: Thursday, March 10, 2016 7:29 PM > To: Rasool Al-Saadi > Cc: aqm@ietf.org; freebsd-net@FreeBSD.org; freebsd-ipfw@FreeBSD.org; > Grenville Armitage > Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's > ipfw/dummynet >=20 > On 26 Feb, Rasool Al-Saadi wrote: > > Dear all, > > > > I would like to announce that we (myself and Grenville Armitage) releas= ed > Dummynet AQM v0.1, which is an independent implementation of CoDel and > FQ-CoDel for FreeBSD's ipfw/dummynet framework, based on the IETF > CoDel [1] and FQ-CoDel [2] Internet-Drafts. > > We prepared patches for FreeBSD11-CURRENT-r295345 and FreeBSD 10.x- > RELEASE (10.0, 10.1, 10.2), and a technical report of our implementation= . > > > > Patches and documentation can be found in: > > http://caia.swin.edu.au/freebsd/aqm > > > > Technical report: > > http://caia.swin.edu.au/reports/160226A/CAIA-TR-160226A.pdf >=20 > I've got some results with running this on my firewall in an attempt to t= ame a > severe bufferbloat problem on my ADSL connection to the outside world. >=20 > The raw speed numbers reported by my ADSL modem are 6016 Kb/s > downstream and 768 Kb/s upstream. I set my MTU to 1492 to avoid > fragmentation from PPPoE overhead. >=20 > Using with things unthrottled, I > observe about 5050 Kb/s downstream and 648Kb/s upstream, with a > bufferbloat rating of F. >=20 > I configured the system to use FQ-CoDel, with separate pipes for each > direction. Because of the slow upstream speed, I increased the target va= lue > for the upstream direction to 25 ms since a maximum size packet will requ= ire > about 20 ms to send. I also set the > net.inet.tcp.experimental.initcwnd10 sysctl value to 0. The latter seeme= d to > help a lot. With this feature enabled, the initial packet blast at the s= tart of > the upload caused a large initial latency spike, and the initial transfer= rate > ended up being very slow and it took a long time to ramp up to its maximu= m > sustained value. >=20 > My current dummynet pipe bandwidth settings are 4800 Kb/s downstream > and > 615 Kb/s upstream. The speedtest results for these settings are about 46= 00 > Kb/s downstream and about 600 Kb/s upstream. I'm somewhat > disappointed in the bandwith loss, but my bufferbloat rating has improved= to > mostly A's with some B's. >=20 > I do still see a large increase in latency at the start of transfers, and= then it > oscillates for a while before settling down at a reasonable value for the > remainder of the transfer. I suspect this is to be expected. >=20 > It would be nice if the implementation was able to account for the PPPOE > and ATM framing overhead like the Linux implementation does. I think tha= t > would help performance when there is a mix of packet sizes. Dave T=E4ht suggests you to try a "quantum 300" for your 600kbit uplink. BTW, if you interested try our FQ-PIE implementation in Dummynet AQM v0.2 i= n your configuration. Regards, Rasool From owner-freebsd-ipfw@freebsd.org Wed Apr 20 10:28:16 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26002B16206 for ; Wed, 20 Apr 2016 10:28:16 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 019F61CA2 for ; Wed, 20 Apr 2016 10:28:15 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 983E03ACDA for ; Wed, 20 Apr 2016 03:21:56 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-ipfw@freebsd.org Subject: How can I find the bandwidth hogs? Date: Wed, 20 Apr 2016 03:21:56 -0700 Message-ID: <162.1461147716@server1.tristatelogic.com> X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2016 10:28:16 -0000 I don't have very much inbound bandwidth... only 6MB/sec. I've noticed, repeatedly, that when visiting various web sites, even if I'm not actively doing anything on these sites, _something_ ends up gobbling up as much as 2/3rds or more of my inbound bandwidth. I suspect that this may perhaps be due to some nasty bandwith hogging banner advertisments associated with the sites I visit. Or maybe not. I really don't know. But I would like to find out which IPs are the main source of this problem so that I can just simply block those in my IPFW rules. So anyway, my question is a simple one: How can I get ipwf... or something else... to keep counts of the numbers of bytes received from a given source port AND also from various source IPs, i.e. over some period of time, e.g. an hour or a day? Please understand. I _do not_ just want ipfw to give me a TOTAL count of all bytes received, over time, from some particular source port (e.g. 80, 443). I already know how to do that simple thing. Rather, I want ipfw... or some other tool... to give me, at the end of the test time period... a nice LIST of how many bytes came in from each and every separate source IP over the given time period. (There could be hundreds or thousands of IPs sending me packets with a source port of 80 or 443 over the given time period, so that list could end up being really long.) Can anybody give me a hint of how to do this? From owner-freebsd-ipfw@freebsd.org Wed Apr 20 17:11:50 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AE53B15CCD for ; Wed, 20 Apr 2016 17:11:50 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47CBB182F for ; Wed, 20 Apr 2016 17:11:50 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u3KHBgnL077532; Wed, 20 Apr 2016 10:11:46 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201604201711.u3KHBgnL077532@gw.catspoiler.org> Date: Wed, 20 Apr 2016 10:11:42 -0700 (PDT) From: Don Lewis Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet To: ralsaadi@swin.edu.au cc: freebsd-ipfw@FreeBSD.org In-Reply-To: <6545444AE21C2749939E637E56594CEA3C20AEFC@gsp-ex02.ds.swin.edu.au> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8BIT X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2016 17:11:50 -0000 On 20 Apr, Rasool Al-Saadi wrote: > > >> -----Original Message----- >> From: Don Lewis [mailto:truckman@FreeBSD.org] >> Sent: Thursday, March 10, 2016 7:29 PM >> To: Rasool Al-Saadi >> Cc: aqm@ietf.org; freebsd-net@FreeBSD.org; freebsd-ipfw@FreeBSD.org; >> Grenville Armitage >> Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's >> ipfw/dummynet >> >> On 26 Feb, Rasool Al-Saadi wrote: >> > Dear all, >> > >> > I would like to announce that we (myself and Grenville Armitage) released >> Dummynet AQM v0.1, which is an independent implementation of CoDel and >> FQ-CoDel for FreeBSD's ipfw/dummynet framework, based on the IETF >> CoDel [1] and FQ-CoDel [2] Internet-Drafts. >> > We prepared patches for FreeBSD11-CURRENT-r295345 and FreeBSD 10.x- >> RELEASE (10.0, 10.1, 10.2), and a technical report of our implementation. >> > >> > Patches and documentation can be found in: >> > http://caia.swin.edu.au/freebsd/aqm >> > >> > Technical report: >> > http://caia.swin.edu.au/reports/160226A/CAIA-TR-160226A.pdf >> >> I've got some results with running this on my firewall in an attempt to tame a >> severe bufferbloat problem on my ADSL connection to the outside world. >> >> The raw speed numbers reported by my ADSL modem are 6016 Kb/s >> downstream and 768 Kb/s upstream. I set my MTU to 1492 to avoid >> fragmentation from PPPoE overhead. >> >> Using with things unthrottled, I >> observe about 5050 Kb/s downstream and 648Kb/s upstream, with a >> bufferbloat rating of F. >> >> I configured the system to use FQ-CoDel, with separate pipes for each >> direction. Because of the slow upstream speed, I increased the target value >> for the upstream direction to 25 ms since a maximum size packet will require >> about 20 ms to send. I also set the >> net.inet.tcp.experimental.initcwnd10 sysctl value to 0. The latter seemed to >> help a lot. With this feature enabled, the initial packet blast at the start of >> the upload caused a large initial latency spike, and the initial transfer rate >> ended up being very slow and it took a long time to ramp up to its maximum >> sustained value. >> >> My current dummynet pipe bandwidth settings are 4800 Kb/s downstream >> and >> 615 Kb/s upstream. The speedtest results for these settings are about 4600 >> Kb/s downstream and about 600 Kb/s upstream. I'm somewhat >> disappointed in the bandwith loss, but my bufferbloat rating has improved to >> mostly A's with some B's. >> >> I do still see a large increase in latency at the start of transfers, and then it >> oscillates for a while before settling down at a reasonable value for the >> remainder of the transfer. I suspect this is to be expected. >> >> It would be nice if the implementation was able to account for the PPPOE >> and ATM framing overhead like the Linux implementation does. I think that >> would help performance when there is a mix of packet sizes. > > Dave Täht suggests you to try a "quantum 300" for your 600kbit uplink. Actually I need to decrease the quantum from 1514 to 1506 in the other direction as well. I decreased the MTU on this path to 1492 to compensate for the PPPOE header that is added by my DSL router. The default 1500 byte MTU will result in fragmentation. If I don't change the quantum to match, then periodically two maximal size packets will be allowed from the same flow. > BTW, if you interested try our FQ-PIE implementation in Dummynet AQM v0.2 in your configuration. I'm planning on trying that when I have the time. I also want to test with and without ECN on the end station. From owner-freebsd-ipfw@freebsd.org Wed Apr 20 21:58:14 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C8F3B16630 for ; Wed, 20 Apr 2016 21:58:14 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB2411359 for ; Wed, 20 Apr 2016 21:58:13 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id n3so103526987wmn.0 for ; Wed, 20 Apr 2016 14:58:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=dstUuKB5qQIcNRp8sRukcHBhORsuVPZ+Xfm74+eRG9I=; b=aP9fRuweLbJPGw4+pB6TpMzEFLo91ykxaC5+9LVnQqcgmD3zGs82OSyG0wkMRVEUcy Fiu7tSwWxgirTjgPnsTkzYGB+GKYJKwm8x8Fgx03roW1L+fs4rv5fBgo2NYxiM5n8AsY 7mzVdSsMkqG49SuF7s5CqiQcXvcslGDOFTXEWtTCb5gK/nMaJZOYgIai79G0k0urX71q KDSLOKoVZoRYb6GeUmgpFDkfjt8/wVFAsAvtivVBcyU0jOguCaBQUYx/6j6BvRNETU6V 9R8cmz7mx0VnO4SG9Jd7yy8m90iloJNt6+ut4KO69P8F5zUL4lpJVOI8g+iKeLIwrBNj mA5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=dstUuKB5qQIcNRp8sRukcHBhORsuVPZ+Xfm74+eRG9I=; b=LCW/w52LCV29sX0AGbcVLQvBCBFRW/33YDfId7ZYkxMuYtfbO6i7NFugL+UdpymqYs NrOAiXKUUikztZ6bxpUNtr4JDFgJTmuF9LWWT49URKA1w9w/s70XBRfiN3a/vr0Olpax q3SRryVTKC/kbo/RIYAQP8Ara+WDqMPr9NbRL/lbJyZDQ+FOCgM/ioJew28cgCo/QrGu jKdQdDzlhA7mt91Uma7bz11i55Q7Bp/vGmZo4UWnc0XpuybWBpWpqFtnCr8aD4E8YVQA C7GgzWFfXuUfuZShD7k/f9LgNiP9lqDEpB2SEiaXB8zUqtgZ8mcft1gS6MHOnVBZCDjr BXcg== X-Gm-Message-State: AOPr4FUAcqG8VPHA5BSISabWcasoiEQKRP12UxJnX3t4JV7dx1FyiSX+R5fV1+WxD6YiXjfZN1S/bi/EZO0asQ== MIME-Version: 1.0 X-Received: by 10.28.165.216 with SMTP id o207mr1199423wme.31.1461189491531; Wed, 20 Apr 2016 14:58:11 -0700 (PDT) Received: by 10.194.42.41 with HTTP; Wed, 20 Apr 2016 14:58:11 -0700 (PDT) In-Reply-To: <162.1461147716@server1.tristatelogic.com> References: <162.1461147716@server1.tristatelogic.com> Date: Wed, 20 Apr 2016 23:58:11 +0200 Message-ID: Subject: Re: How can I find the bandwidth hogs? From: Ben Woods To: "Ronald F. Guilmette" Cc: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2016 21:58:14 -0000 On 20 April 2016 at 12:21, Ronald F. Guilmette wrote: > > Rather, I want ipfw... or some other tool... to give me, at the > end of the test time period... a nice LIST of how many bytes came in > from each and every separate source IP over the given time period. > (There could be hundreds or thousands of IPs sending me packets > with a source port of 80 or 443 over the given time period, so > that list could end up being really long.) > > Can anybody give me a hint of how to do this? Easiest solution I can think of: Use net-mgmt/darkstat. https://unix4lyfe.org/darkstat/ Slightly more complicated solution, but with potentially more power for filtering the data: Netflow/NfDump/NfSen https://forums.freebsd.org/threads/49724/ Good luck! Regards, Ben From owner-freebsd-ipfw@freebsd.org Wed Apr 20 22:18:49 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CBECB16D56 for ; Wed, 20 Apr 2016 22:18:49 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 1A7FB1D9D for ; Wed, 20 Apr 2016 22:18:47 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 0792C3AF79 for ; Wed, 20 Apr 2016 15:18:47 -0700 (PDT) From: "Ronald F. Guilmette" To: freebsd-ipfw@freebsd.org Subject: NEVERMIND! (was: How can I find the bandwidth hogs?) In-Reply-To: Date: Wed, 20 Apr 2016 15:18:47 -0700 Message-ID: <4301.1461190727@server1.tristatelogic.com> X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2016 22:18:49 -0000 Some hours after I posted my question, it dawned on me (duh!) that what I was asking for was some tool to perform what is most probably called "traffic analysis". so I just googled for "FreeBSSD traffic analysis" and of course, that got me boat loads of information about several different tools, all available on FreeBSD, that will do something along the lines of what I want to do. So I'm good. Please disregard my prior question here. In message Ben Woods wrote: >Easiest solution I can think of: Use net-mgmt/darkstat. >https://unix4lyfe.org/darkstat/ Oh! Thanks. That actually is _not_ one of the tools that I found when googling. I'll check that one out too. From owner-freebsd-ipfw@freebsd.org Thu Apr 21 00:08:49 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F68DB155C8 for ; Thu, 21 Apr 2016 00:08:49 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) Received: from iport1.cc.swin.edu.au (iport1.cc.swin.edu.au [136.186.0.49]) by mx1.freebsd.org (Postfix) with ESMTP id 83BDA102C; Thu, 21 Apr 2016 00:08:48 +0000 (UTC) (envelope-from ralsaadi@swin.edu.au) X-IronPort-AV: E=Sophos;i="5.24,511,1454936400"; d="scan'208";a="19342119" Received: from gsp-ex01.ds.swin.edu.au (HELO outlook.swin.edu.au) ([136.186.126.17]) by iport1.cc.swin.edu.au with ESMTP; 21 Apr 2016 10:08:46 +1000 Received: from GSP-EX02.ds.swin.edu.au ([169.254.2.170]) by gsp-ex01.ds.swin.edu.au ([169.254.1.160]) with mapi id 14.03.0294.000; Thu, 21 Apr 2016 10:08:46 +1000 From: Rasool Al-Saadi To: Don Lewis CC: "freebsd-ipfw@FreeBSD.org" Subject: RE: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet Thread-Topic: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet Thread-Index: AdFwoEB9RcA1ON/lTEC2HgPx0zqG6gJqmosACB9U/dAAAvlHAAAjSEZw Date: Thu, 21 Apr 2016 00:08:46 +0000 Message-ID: <6545444AE21C2749939E637E56594CEA3C20C123@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C20AEFC@gsp-ex02.ds.swin.edu.au> <201604201711.u3KHBgnL077532@gw.catspoiler.org> In-Reply-To: <201604201711.u3KHBgnL077532@gw.catspoiler.org> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [136.186.112.108] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Apr 2016 00:08:49 -0000 > -----Original Message----- > From: Don Lewis [mailto:truckman@FreeBSD.org] > Sent: Thursday, 21 April 2016 3:12 AM > To: Rasool Al-Saadi > Cc: freebsd-ipfw@FreeBSD.org > Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's > ipfw/dummynet >=20 > On 20 Apr, Rasool Al-Saadi wrote: > > > > > >> -----Original Message----- > >> From: Don Lewis [mailto:truckman@FreeBSD.org] > >> Sent: Thursday, March 10, 2016 7:29 PM > >> To: Rasool Al-Saadi > >> Cc: aqm@ietf.org; freebsd-net@FreeBSD.org; freebsd- > ipfw@FreeBSD.org; > >> Grenville Armitage > >> Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's > >> ipfw/dummynet > >> > >> On 26 Feb, Rasool Al-Saadi wrote: > >> > Dear all, > >> > > >> > I would like to announce that we (myself and Grenville Armitage) > >> > released > >> Dummynet AQM v0.1, which is an independent implementation of CoDel > >> and FQ-CoDel for FreeBSD's ipfw/dummynet framework, based on the > IETF > >> CoDel [1] and FQ-CoDel [2] Internet-Drafts. > >> > We prepared patches for FreeBSD11-CURRENT-r295345 and FreeBSD > >> > 10.x- > >> RELEASE (10.0, 10.1, 10.2), and a technical report of our implementat= ion. > >> > > >> > Patches and documentation can be found in: > >> > http://caia.swin.edu.au/freebsd/aqm > >> > > >> > Technical report: > >> > http://caia.swin.edu.au/reports/160226A/CAIA-TR-160226A.pdf > >> > >> I've got some results with running this on my firewall in an attempt > >> to tame a severe bufferbloat problem on my ADSL connection to the > outside world. > >> > >> The raw speed numbers reported by my ADSL modem are 6016 Kb/s > >> downstream and 768 Kb/s upstream. I set my MTU to 1492 to avoid > >> fragmentation from PPPoE overhead. > >> > >> Using with things unthrottled, > >> I observe about 5050 Kb/s downstream and 648Kb/s upstream, with a > >> bufferbloat rating of F. > >> > >> I configured the system to use FQ-CoDel, with separate pipes for each > >> direction. Because of the slow upstream speed, I increased the > >> target value for the upstream direction to 25 ms since a maximum size > >> packet will require about 20 ms to send. I also set the > >> net.inet.tcp.experimental.initcwnd10 sysctl value to 0. The latter > >> seemed to help a lot. With this feature enabled, the initial packet > >> blast at the start of the upload caused a large initial latency > >> spike, and the initial transfer rate ended up being very slow and it > >> took a long time to ramp up to its maximum sustained value. > >> > >> My current dummynet pipe bandwidth settings are 4800 Kb/s > downstream > >> and > >> 615 Kb/s upstream. The speedtest results for these settings are > >> about 4600 Kb/s downstream and about 600 Kb/s upstream. I'm > somewhat > >> disappointed in the bandwith loss, but my bufferbloat rating has > >> improved to mostly A's with some B's. > >> > >> I do still see a large increase in latency at the start of transfers, > >> and then it oscillates for a while before settling down at a > >> reasonable value for the remainder of the transfer. I suspect this is= to be > expected. > >> > >> It would be nice if the implementation was able to account for the > >> PPPOE and ATM framing overhead like the Linux implementation does. I > >> think that would help performance when there is a mix of packet sizes. > > > > Dave T=E4ht suggests you to try a "quantum 300" for your 600kbit uplink= . >=20 > Actually I need to decrease the quantum from 1514 to 1506 in the other > direction as well. I decreased the MTU on this path to 1492 to compensat= e > for the PPPOE header that is added by my DSL router. The default 1500 by= te > MTU will result in fragmentation. If I don't change the quantum to match= , > then periodically two maximal size packets will be allowed from the same > flow. >=20 > > BTW, if you interested try our FQ-PIE implementation in Dummynet AQM > v0.2 in your configuration. >=20 > I'm planning on trying that when I have the time. I also want to test wi= th and > without ECN on the end station. If you went to test with ECN, please use Dummynet AQM v0.2 patch as v0.1 ha= s incorrect ECN implementation.=20 Regards, Rasool From owner-freebsd-ipfw@freebsd.org Sat Apr 23 06:46:43 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 327A2B181F0 for ; Sat, 23 Apr 2016 06:46:43 +0000 (UTC) (envelope-from nazari.s11@gmail.com) Received: from mbob.nabble.com (mbob.nabble.com [162.253.133.15]) by mx1.freebsd.org (Postfix) with ESMTP id 240E01E09 for ; Sat, 23 Apr 2016 06:46:43 +0000 (UTC) (envelope-from nazari.s11@gmail.com) Received: from msam.nabble.com (unknown [162.253.133.85]) by mbob.nabble.com (Postfix) with ESMTP id 0694125F3430 for ; Fri, 22 Apr 2016 23:32:51 -0700 (PDT) Date: Fri, 22 Apr 2016 23:46:40 -0700 (MST) From: samira To: freebsd-ipfw@freebsd.org Message-ID: <1461394000058-6093661.post@n5.nabble.com> Subject: Whether IPFW generates " No buffer space available " error ? MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Apr 2016 06:46:43 -0000 Hi everyone, I using FreeBSD9.2 and defining a rule in ipfw that divert tcp packets on port 80 to port 8000 and by suricata will be reviewed. ipfw list: 01901 divert 8000 tcp from any to any dst-port 80 And then the packets is sent by altq to queue defined ipfw list: 03009 skipto 3011 tcp from any to any dst-port 80 03010 skipto 3012 ip from any to any 03011 allow altq http-gbeth3-out ip from any to any via gbeth3 out And we limit bandwidth in pf.conf for http traffic pf.conf: queue http-gbeth3-out bandwidth 50Kb hfsc ( upperlimit 50Kb ) When the transmission of huge amounts of http packets and pf action is to drop packets, suricata crash and the following message appears in the suricata.log file: - [ERRCODE: SC_WARN_IPFW_XMIT(84)] - Write to ipfw divert socket failed: No buffer space available Has anyone dealt with this issue? There is a similar problem: By sending ICMP packets to the queue and send ping from the interface also seen this problem and the following message is displayed: ping: sendto: No buffer space available If the specified bandwidth increased and not drop any packets, this problem does not occur. Thank you for all of your comments and help. -- View this message in context: http://freebsd.1045724.n5.nabble.com/Whether-IPFW-generates-No-buffer-space-available-error-tp6093661.html Sent from the freebsd-ipfw mailing list archive at Nabble.com.