From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 13:16:11 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C84B106567A for ; Tue, 24 Jun 2008 13:16:11 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id 8357D8FC14 for ; Tue, 24 Jun 2008 13:16:10 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 5F2D11B10EF4; Tue, 24 Jun 2008 15:16:09 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on malcho.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, J_CHICKENPOX_48 autolearn=no version=3.2.4 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 7B4071B10CB6; Tue, 24 Jun 2008 15:16:06 +0200 (CEST) Message-ID: <4860F395.1010000@moneybookers.com> Date: Tue, 24 Jun 2008 16:16:05 +0300 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.14 (X11/20080616) MIME-Version: 1.0 To: Paul References: <4860EF76.1050807@moneybookers.com> <4860F0FA.1010707@gtcomm.net> In-Reply-To: <4860F0FA.1010707@gtcomm.net> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.93, clamav-milter version 0.93 on blah.cmotd.com X-Virus-Status: Clean Cc: "freebsd-net@freebsd.org" Subject: Re: jboss4 on freebsd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2008 13:16:11 -0000 Paul wrote: > kern.ipc.nmbclusters=128000 changed - no effect > > Check output from netstat -m, this shows network buffers. 770/8200/8970 mbufs in use (current/cache/total) 768/5426/6194/128000 mbuf clusters in use (current/cache/total/max) 768/5248 mbuf+clusters out of packet secondary zone in use (current/cache) 0/677/677/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1728K/15610K/17338K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 73 requests for I/O initiated by sendfile 0 calls to protocol drain routines This output is in the same second as I see no buffer space available .. isn't this weird? > > > Stefan Lambrev wrote: >> Greetings, >> >> I'm experimenting with jboss4 cluster under freebsd 7 (amd64). >> In my configuration I have 2 jboss instances which are in cluster and >> they communicate via separate network (used only for shared data) >> When I create some load on the application sometimes I see this error: >> >> 2008-06-24 14:46:21,602 ERROR [org.jgroups.protocols.UDP] failed >> sending message to 10.50.1.1:57680 (59800 bytes) >> java.io.IOException: No buffer space available >> >> It looks very much, that jboss can't handle properly such error as on >> linux there is no such thing as no network buffers ;) - >> http://wiki.freebsd.org/AvoidingLinuxisms >> >> But what really bothers me is that I see "No buffer space available" >> on very low network IO - >> >> input (em2) output >> packets errs bytes packets errs bytes colls >> 144 0 2203390 292 0 2072771 0 >> 1568 0 2329764 63 0 9099 0 >> 76 0 231562 34 0 148306 0 >> 563 0 1152531 1009 0 1768748 0 >> 1625 0 2601502 104 0 229728 0 >> 65 0 467296 85 0 441566 0 >> 464 0 680082 973 0 1439442 0 >> 357 0 1940361 55 0 222484 0 >> 1651 0 2827932 145 0 450265 0 >> >> E.g. traffic between 1-3MB/s. >> >> I'm using: >> em2: flags=8843 metric 0 mtu >> 9000 >> >> options=19b >> ether 00:15:17:60:04:c8 >> inet 10.3.3.117 netmask 0xffffff00 broadcast 10.3.3.255 >> media: Ethernet autoselect (1000baseTX ) >> status: active >> >> em2: port 0x2020-0x203f >> mem 0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 >> on pci5 >> em2: Using MSI interrupt >> em2: [FILTER] >> >> and my sysctl.conf is: >> kern.maxfiles=65000 >> kern.ipc.shmmax=67108864 >> kern.fallback_elf_brand=3 >> kern.threads.max_threads_per_proc=6000 >> kern.ipc.somaxconn=512 >> #jboss extra >> net.inet.udp.maxdgram=73728 >> kern.ipc.maxsockbuf=1048576 >> net.inet.udp.recvspace=147456 >> kern.ipc.maxsockets=49312 >> >> Any ideas how I can improve things? >> > -- Best Wishes, Stefan Lambrev ICQ# 24134177