Date: Tue, 24 Jun 2008 18:46:36 +0400 From: Dmitriy <_pppp@mail.ru> To: Stefan Lambrev <stefan.lambrev@moneybookers.com> Cc: freebsd-net@FreeBSD.org Subject: Re: jboss4 on freebsd Message-ID: <E1KB9n2-0002X3-00._pppp-mail-ru@f7.mail.ru> In-Reply-To: <4860EF76.1050807@moneybookers.com> References: <4860EF76.1050807@moneybookers.com>
next in thread | previous in thread | raw e-mail | index | archive | help
-----Original Message----- From: Stefan Lambrev <stefan.lambrev@moneybookers.com> To: freebsd-net@FreeBSD.org Date: Tue, 24 Jun 2008 15:58:30 +0300 Subject: jboss4 on freebsd > > Greetings, > > I'm experimenting with jboss4 cluster under freebsd 7 (amd64). > In my configuration I have 2 jboss instances which are in cluster and > they communicate via separate network (used only for shared data) > When I create some load on the application sometimes I see this error: > > 2008-06-24 14:46:21,602 ERROR [org.jgroups.protocols.UDP] failed sending > message to 10.50.1.1:57680 (59800 bytes) > java.io.IOException: No buffer space available > > It looks very much, that jboss can't handle properly such error as on > linux there is no such thing as no network buffers ;) - > http://wiki.freebsd.org/AvoidingLinuxisms > > But what really bothers me is that I see "No buffer space available" on > very low network IO - > > input (em2) output > packets errs bytes packets errs bytes colls > 144 0 2203390 292 0 2072771 0 > 1568 0 2329764 63 0 9099 0 > 76 0 231562 34 0 148306 0 > 563 0 1152531 1009 0 1768748 0 > 1625 0 2601502 104 0 229728 0 > 65 0 467296 85 0 441566 0 > 464 0 680082 973 0 1439442 0 > 357 0 1940361 55 0 222484 0 > 1651 0 2827932 145 0 450265 0 > > E.g. traffic between 1-3MB/s. > > I'm using: > em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 > options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4> > ether 00:15:17:60:04:c8 > inet 10.3.3.117 netmask 0xffffff00 broadcast 10.3.3.255 > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > > em2: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0x2020-0x203f mem > 0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 on pci5 > em2: Using MSI interrupt > em2: [FILTER] > > and my sysctl.conf is: > kern.maxfiles=65000 > kern.ipc.shmmax=67108864 > kern.fallback_elf_brand=3 > kern.threads.max_threads_per_proc=6000 > kern.ipc.somaxconn=512 > #jboss extra > net.inet.udp.maxdgram=73728 > kern.ipc.maxsockbuf=1048576 > net.inet.udp.recvspace=147456 > kern.ipc.maxsockets=49312 > > Any ideas how I can improve things? > I guess you get the error from the NIC driver. There 2 conditions which can trigger it: 1. if( adapter->num_tx_desc_avail < EM_TX_OP_THRESHOLD ) 2. if( nsegs > ( adapter->num_tx_desc_avail - 2 ) ) I have 2 suggestions how to fix your error: 1. try to disable TXCSUM ( #ifconfig em2 -txcsum ) 2. increase the number of transfer descriptors available if your hardware supports it: [man em(4) excerpt] Tunables can be set at the loader(8) prompt before booting the kernel or stored in loader.conf(5). hw.em.txd Number of transmit descriptors allocated by the driver. The default value is 256. The 82542 and 82543-based adapters can handle up to 256 descriptors, while others can have up to 4096. hw.em.tx_int_delay This value delays the generation of transmit interrupts in units of 1.024 microseconds. The default value is 64. hw.em.tx_abs_int_delay If hw.em.tx_int_delay is non-zero, this tunable limits the maxi- mum delay in which a transmit interrupt is generated. [/man] Regards, Dmitriy.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1KB9n2-0002X3-00._pppp-mail-ru>