Date: Wed, 7 Mar 2012 12:29:14 -0800 From: YongHyeon PYUN <pyunyh@gmail.com> To: Eugene Grosbein <egrosbein@rdtc.ru> Cc: marius@freebsd.org, yongari@freebsd.org, "net@freebsd.org" <net@freebsd.org> Subject: Re: suboptimal bge(4) BCM5704 performance in RELENG_8 Message-ID: <20120307202914.GB9436@michelle.cdnetworks.com> In-Reply-To: <4F5608EA.6080705@rdtc.ru> References: <4F5608EA.6080705@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 06, 2012 at 07:54:02PM +0700, Eugene Grosbein wrote: > Hi! > > Yesterday I've updated old HP ProLiant DL360 G4p to 8.3-PRELELEASE/amd64 > running busy icecast2 server in hope it can saturate 1G bge(4) link. > > This server has PCI-X connected HP NC7782 Gigabit Server Adapter (BCM5704). Would you show me the output of dmesg(bge(4) and brgphy(4) related ones)? > Is it supposed to emit more than 540Mbit/s with average packet size equal to 1300? > Yes. > netstat shows increasing "Drop" counters: > > # netstat -id > Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop > bge0 1500 <Link#1> 00:18:fe:86:ab:f4 635577217 0 0 1047897138 0 0 454509 > > vmstat shows relatively high but pretty sane interrupt rate: > > # vmstat -i | grep bge > irq25: bge0 232949783 10149 > > Input traffic is less than 20Mbit/s meantime, output does not go above 540Mbit/s. > Its mbufs usage is normal (server has 2G RAM): > > # netstat -m > 13459/15986/29445 mbufs in use (current/cache/total) > 13322/13112/26434/262144 mbuf clusters in use (current/cache/total/max) > 13322/13046 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/104/104/131072 4k (page size) jumbo clusters in use (current/cache/total/max) > 0/0/0/65536 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/32768 16k jumbo clusters in use (current/cache/total/max) > 30008K/30636K/60645K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > I've glanced at bge(4) code and see it uses lots of hardcoded constants > for rx/tx descriptor rings, for interrupt moderation and for interface FIFO queue. Firmware has a fixed number of descriptors so increasing them wouldn't give better numbers. > And no loader tunnables/sysctls like em/igb have. > Show me the output of "sysctl dev.bge.0.stats". > Should I try to play with constants in the code and if so, what are limits of this chip? > Or it will never be capable of utilizing full gigabit speed? BCM5704 is old controller but it should have no problems to saturate the link. I suspect it could be related with DMA configuration but needs more information. > > Eugene Grosbein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120307202914.GB9436>