From owner-freebsd-net@FreeBSD.ORG Tue Mar 6 12:54:06 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 12813106566C; Tue, 6 Mar 2012 12:54:06 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 74F458FC1D; Tue, 6 Mar 2012 12:54:04 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id q26Cs2r4031181; Tue, 6 Mar 2012 19:54:02 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4F5608EA.6080705@rdtc.ru> Date: Tue, 06 Mar 2012 19:54:02 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: "net@freebsd.org" Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: marius@freebsd.org, yongari@freebsd.org Subject: suboptimal bge(4) BCM5704 performance in RELENG_8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Mar 2012 12:54:06 -0000 Hi! Yesterday I've updated old HP ProLiant DL360 G4p to 8.3-PRELELEASE/amd64 running busy icecast2 server in hope it can saturate 1G bge(4) link. This server has PCI-X connected HP NC7782 Gigabit Server Adapter (BCM5704). Is it supposed to emit more than 540Mbit/s with average packet size equal to 1300? netstat shows increasing "Drop" counters: # netstat -id Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop bge0 1500 00:18:fe:86:ab:f4 635577217 0 0 1047897138 0 0 454509 vmstat shows relatively high but pretty sane interrupt rate: # vmstat -i | grep bge irq25: bge0 232949783 10149 Input traffic is less than 20Mbit/s meantime, output does not go above 540Mbit/s. Its mbufs usage is normal (server has 2G RAM): # netstat -m 13459/15986/29445 mbufs in use (current/cache/total) 13322/13112/26434/262144 mbuf clusters in use (current/cache/total/max) 13322/13046 mbuf+clusters out of packet secondary zone in use (current/cache) 0/104/104/131072 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/65536 9k jumbo clusters in use (current/cache/total/max) 0/0/0/32768 16k jumbo clusters in use (current/cache/total/max) 30008K/30636K/60645K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines I've glanced at bge(4) code and see it uses lots of hardcoded constants for rx/tx descriptor rings, for interrupt moderation and for interface FIFO queue. And no loader tunnables/sysctls like em/igb have. Should I try to play with constants in the code and if so, what are limits of this chip? Or it will never be capable of utilizing full gigabit speed? Eugene Grosbein