From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 17:46:32 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCFAD1065670 for ; Tue, 24 Jun 2008 17:46:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outO.internet-mail-service.net (outo.internet-mail-service.net [216.240.47.238]) by mx1.freebsd.org (Postfix) with ESMTP id CE3208FC0A for ; Tue, 24 Jun 2008 17:46:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 7DC522482; Tue, 24 Jun 2008 10:46:32 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 4AEA22D6150; Tue, 24 Jun 2008 10:46:32 -0700 (PDT) Message-ID: <48613304.7090204@elischer.org> Date: Tue, 24 Jun 2008 10:46:44 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Stefan Lambrev References: <4860EF76.1050807@moneybookers.com> <4860F0FA.1010707@gtcomm.net> <4860F395.1010000@moneybookers.com> In-Reply-To: <4860F395.1010000@moneybookers.com> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Paul Subject: Re: jboss4 on freebsd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2008 17:46:32 -0000 Stefan Lambrev wrote: > > > Paul wrote: >> kern.ipc.nmbclusters=128000 > changed - no effect >> if this is udp or some datagram traffic then it is telling you that the interface queue has filled up.. >> Check output from netstat -m, this shows network buffers. > 770/8200/8970 mbufs in use (current/cache/total) > 768/5426/6194/128000 mbuf clusters in use (current/cache/total/max) > 768/5248 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/677/677/12800 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 1728K/15610K/17338K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 73 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > This output is in the same second as I see no buffer space available .. > isn't this weird? >> >> >> Stefan Lambrev wrote: >>> Greetings, >>> >>> I'm experimenting with jboss4 cluster under freebsd 7 (amd64). >>> In my configuration I have 2 jboss instances which are in cluster and >>> they communicate via separate network (used only for shared data) >>> When I create some load on the application sometimes I see this error: >>> >>> 2008-06-24 14:46:21,602 ERROR [org.jgroups.protocols.UDP] failed >>> sending message to 10.50.1.1:57680 (59800 bytes) >>> java.io.IOException: No buffer space available >>> >>> It looks very much, that jboss can't handle properly such error as on >>> linux there is no such thing as no network buffers ;) - >>> http://wiki.freebsd.org/AvoidingLinuxisms >>> >>> But what really bothers me is that I see "No buffer space available" >>> on very low network IO - >>> >>> input (em2) output >>> packets errs bytes packets errs bytes colls >>> 144 0 2203390 292 0 2072771 0 >>> 1568 0 2329764 63 0 9099 0 >>> 76 0 231562 34 0 148306 0 >>> 563 0 1152531 1009 0 1768748 0 >>> 1625 0 2601502 104 0 229728 0 >>> 65 0 467296 85 0 441566 0 >>> 464 0 680082 973 0 1439442 0 >>> 357 0 1940361 55 0 222484 0 >>> 1651 0 2827932 145 0 450265 0 >>> >>> E.g. traffic between 1-3MB/s. >>> >>> I'm using: >>> em2: flags=8843 metric 0 mtu >>> 9000 >>> >>> options=19b >>> ether 00:15:17:60:04:c8 >>> inet 10.3.3.117 netmask 0xffffff00 broadcast 10.3.3.255 >>> media: Ethernet autoselect (1000baseTX ) >>> status: active >>> >>> em2: port 0x2020-0x203f >>> mem 0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 >>> on pci5 >>> em2: Using MSI interrupt >>> em2: [FILTER] >>> >>> and my sysctl.conf is: >>> kern.maxfiles=65000 >>> kern.ipc.shmmax=67108864 >>> kern.fallback_elf_brand=3 >>> kern.threads.max_threads_per_proc=6000 >>> kern.ipc.somaxconn=512 >>> #jboss extra >>> net.inet.udp.maxdgram=73728 >>> kern.ipc.maxsockbuf=1048576 >>> net.inet.udp.recvspace=147456 >>> kern.ipc.maxsockets=49312 >>> >>> Any ideas how I can improve things? >>> >> >