From owner-freebsd-net@FreeBSD.ORG Mon May 5 05:41:33 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1FC91065672 for ; Mon, 5 May 2008 05:41:33 +0000 (UTC) (envelope-from oleksandr@samoylyk.sumy.ua) Received: from mail.telesweet.net (news.telesweet.net [194.110.252.16]) by mx1.freebsd.org (Postfix) with ESMTP id 264CE8FC1B for ; Mon, 5 May 2008 05:41:32 +0000 (UTC) (envelope-from oleksandr@samoylyk.sumy.ua) Received: from localhost (localhost [127.0.0.1]) by mail.telesweet.net (Postfix) with ESMTP id DF0F8B82B; Mon, 5 May 2008 08:41:30 +0300 (EEST) X-Virus-Scanned: by Telesweet Mail Virus Scanner X-Spam-Flag: NO X-Spam-Score: -1.44 X-Spam-Level: X-Spam-Status: No, score=-1.44 tagged_above=-999 required=5 tests=[ALL_TRUSTED=-1.44] Received: from [10.0.0.109] (pigeon-work.telesweet [10.0.0.109]) by mail.telesweet.net (Postfix) with ESMTP id BE58CB813; Mon, 5 May 2008 08:41:26 +0300 (EEST) Message-ID: <481E9E06.3040603@samoylyk.sumy.ua> Date: Mon, 05 May 2008 08:41:26 +0300 From: Oleksandr Samoylyk User-Agent: Thunderbird 2.0.0.12 (X11/20080227) MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <481C84B7.6020205@samoylyk.sumy.ua> <481E338D.6040706@samoylyk.sumy.ua> <2a41acea0805041529j5d4dd2f7x5e07a8d6d2eb89b6@mail.gmail.com> <2a41acea0805041532le60cc9cybf22887e9fdcb3f9@mail.gmail.com> In-Reply-To: <2a41acea0805041532le60cc9cybf22887e9fdcb3f9@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Jack Vogel , mike@sentex.net Subject: Re: Troubles with em on FreeBSD 7 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2008 05:41:33 -0000 Jack Vogel wrote: > Oh, I just had a thought, increase the RX processing limit, > that only allows you to process 100 packets in one pass. > > First change it to 250 and see what it does, you might > also set it to -1 which will allow you to process til you > drain the ring, the risk is that you cause other problems > by doing that, but heck at this point anything is worth > trying, right? > last pid: 55326; load averages: 1.40, 1.24, 1.05 up 0+06:56:52 08:32:41 83 processes: 10 running, 60 sleeping, 13 waiting CPU states: 0.5% user, 0.0% nice, 10.5% system, 1.1% interrupt, 87.9% idle Mem: 42M Active, 265M Inact, 272M Wired, 244K Cache, 214M Buf, 7338M Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 12 root 1 171 ki31 0K 16K CPU6 6 413:05 100.00% idle: cpu6 16 root 1 171 ki31 0K 16K RUN 2 401:28 100.00% idle: cpu2 15 root 1 171 ki31 0K 16K CPU3 3 402:45 96.09% idle: cpu3 17 root 1 171 ki31 0K 16K CPU1 1 398:26 92.68% idle: cpu1 14 root 1 171 ki31 0K 16K CPU4 4 401:24 90.28% idle: cpu4 18 root 1 171 ki31 0K 16K CPU0 0 382:00 89.26% idle: cpu0 13 root 1 171 ki31 0K 16K CPU5 5 392:57 81.40% idle: cpu5 11 root 1 171 ki31 0K 16K CPU7 7 378:44 59.47% idle: cpu7 30 root 1 -68 - 0K 16K CPU7 7 37:24 44.78% em1 taskq 31 root 1 -68 - 0K 16K - 5 23:00 18.26% em2 taskq 19 root 1 -44 - 0K 16K WAIT 3 35:45 14.60% swi1: net 32 root 1 -68 - 0K 16K - 0 19:51 7.08% em3 taskq 722 root 2 5 0 68280K 21652K select 2 0:00 1.46% mpd5 20 root 1 -32 - 0K 16K WAIT 2 3:56 0.00% swi4: clock 29 root 1 -68 - 0K 16K - 6 2:53 0.00% em0 taskq 22 root 1 44 - 0K 16K - 1 0:49 0.00% yarrow I've added more NICs. The thing is that the most busiest interface is em3 (uplink) and it's just 7%. # netstat -I em0 -w 1 (just for the way back traffic) input (em0) output packets errs bytes packets errs bytes colls 13 0 2344 8408 0 4290811 0 20 0 2232 8147 0 4179926 0 12 0 1358 8204 0 3984088 0 33 0 5660 7982 0 3944704 0 25 0 3165 8183 0 4226968 0 10 0 1012 8620 0 4288722 0 25 0 2822 8283 0 4220019 0 ^C # netstat -I em1 -w 1 input (em1) output packets errs bytes packets errs bytes colls 7137 0 2618138 0 0 0 0 6772 0 2565885 0 0 0 0 6982 0 2430458 0 0 0 0 6758 0 2459663 0 0 0 0 7036 0 2665113 0 0 0 0 ^C # netstat -I em2 -w 1 input (em2) output packets errs bytes packets errs bytes colls 1237 0 327276 0 0 0 0 1158 0 328459 0 0 0 0 1243 0 346992 0 0 0 0 1076 0 323918 0 0 0 0 1242 0 363008 0 0 0 0 ^C # netstat -I em3 -w 1 input (em3) output packets errs bytes packets errs bytes colls 5681 0 3822499 6233 0 2443102 0 5958 0 4233009 6692 0 2354643 0 6202 0 4296877 6432 0 2455513 0 6117 0 4146643 6419 0 2327346 0 6324 0 4128040 6695 0 2468243 0 5856 0 4052255 6252 0 2445518 0 ^C Why so? -- Oleksandr Samoylyk OVS-RIPE