Date: Wed, 07 Mar 2012 15:12:32 +0700 From: Eugene Grosbein <egrosbein@rdtc.ru> To: pyunyh@gmail.com Cc: marius@freebsd.org, yongari@freebsd.org, "net@freebsd.org" <net@freebsd.org> Subject: Re: suboptimal bge(4) BCM5704 performance in RELENG_8 Message-ID: <4F571870.3090902@rdtc.ru> In-Reply-To: <20120307202914.GB9436@michelle.cdnetworks.com> References: <4F5608EA.6080705@rdtc.ru> <20120307202914.GB9436@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
08.03.2012 03:29, YongHyeon PYUN пишет: > On Tue, Mar 06, 2012 at 07:54:02PM +0700, Eugene Grosbein wrote: >> Hi! >> >> Yesterday I've updated old HP ProLiant DL360 G4p to 8.3-PRELELEASE/amd64 >> running busy icecast2 server in hope it can saturate 1G bge(4) link. >> >> This server has PCI-X connected HP NC7782 Gigabit Server Adapter (BCM5704). > > Would you show me the output of dmesg(bge(4) and brgphy(4) related > ones)? # egrep 'bge|brgphy' /var/run/dmesg.boot bge0: <HP NC7782 Gigabit Server Adapter, ASIC rev. 0x002100> mem 0xfdf70000-0xfdf7ffff irq 25 at device 2.0 on pci2 bge0: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X miibus0: <MII bus> on bge0 brgphy0: <BCM5704 10/100/1000baseTX PHY> PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:18:fe:86:ab:f4 bge0: [ITHREAD] bge1: <HP NC7782 Gigabit Server Adapter, ASIC rev. 0x002100> mem 0xfdf60000-0xfdf6ffff irq 26 at device 2.1 on pci2 bge1: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X miibus1: <MII bus> on bge1 brgphy1: <BCM5704 10/100/1000baseTX PHY> PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:18:fe:86:ab:f3 bge1: [ITHREAD] There are bge0 and bge1 but bge1 is not used and is down. >> Is it supposed to emit more than 540Mbit/s with average packet size equal to 1300? >> > > Yes. [skip] >> I've glanced at bge(4) code and see it uses lots of hardcoded constants >> for rx/tx descriptor rings, for interrupt moderation and for interface FIFO queue. > > Firmware has a fixed number of descriptors so increasing them > wouldn't give better numbers. What it the limit? >> And no loader tunnables/sysctls like em/igb have. > > Show me the output of "sysctl dev.bge.0.stats". # sysctl dev.bge.0.stats dev.bge.0.stats.FramesDroppedDueToFilters: 0 dev.bge.0.stats.DmaWriteQueueFull: 84072 dev.bge.0.stats.DmaWriteHighPriQueueFull: 0 dev.bge.0.stats.NoMoreRxBDs: 0 dev.bge.0.stats.InputDiscards: 0 dev.bge.0.stats.InputErrors: 30 dev.bge.0.stats.RecvThresholdHit: 745400662 dev.bge.0.stats.DmaReadQueueFull: 2020586592 dev.bge.0.stats.DmaReadHighPriQueueFull: 0 dev.bge.0.stats.SendDataCompQueueFull: 0 dev.bge.0.stats.RingSetSendProdIndex: 2832885493 dev.bge.0.stats.RingStatusUpdate: 899990835 dev.bge.0.stats.Interrupts: 899990835 dev.bge.0.stats.AvoidedInterrupts: 0 dev.bge.0.stats.SendThresholdHit: 0 dev.bge.0.stats.rx.ifHCInOctets: 491268800 dev.bge.0.stats.rx.Fragments: 234 dev.bge.0.stats.rx.UnicastPkts: 1977202324 dev.bge.0.stats.rx.MulticastPkts: 0 dev.bge.0.stats.rx.FCSErrors: 341 dev.bge.0.stats.rx.AlignmentErrors: 0 dev.bge.0.stats.rx.xonPauseFramesReceived: 0 dev.bge.0.stats.rx.xoffPauseFramesReceived: 0 dev.bge.0.stats.rx.ControlFramesReceived: 0 dev.bge.0.stats.rx.xoffStateEntered: 0 dev.bge.0.stats.rx.FramesTooLong: 0 dev.bge.0.stats.rx.Jabbers: 0 dev.bge.0.stats.rx.UndersizePkts: 0 dev.bge.0.stats.rx.inRangeLengthError: 0 dev.bge.0.stats.rx.outRangeLengthError: 0 dev.bge.0.stats.tx.ifHCOutOctets: 683592718 dev.bge.0.stats.tx.Collisions: 0 dev.bge.0.stats.tx.XonSent: 0 dev.bge.0.stats.tx.XoffSent: 0 dev.bge.0.stats.tx.flowControlDone: 0 dev.bge.0.stats.tx.InternalMacTransmitErrors: 0 dev.bge.0.stats.tx.SingleCollisionFrames: 0 dev.bge.0.stats.tx.MultipleCollisionFrames: 0 dev.bge.0.stats.tx.DeferredTransmissions: 0 dev.bge.0.stats.tx.ExcessiveCollisions: 0 dev.bge.0.stats.tx.LateCollisions: 0 dev.bge.0.stats.tx.UnicastPkts: 3292778353 dev.bge.0.stats.tx.MulticastPkts: 0 dev.bge.0.stats.tx.BroadcastPkts: 147 dev.bge.0.stats.tx.CarrierSenseErrors: 0 dev.bge.0.stats.tx.Discards: 0 dev.bge.0.stats.tx.Errors: 0 >> Should I try to play with constants in the code and if so, what are limits of this chip? >> Or it will never be capable of utilizing full gigabit speed? > > BCM5704 is old controller but it should have no problems to > saturate the link. I suspect it could be related with DMA > configuration but needs more information. I'll supply any information. It's also possible to try kernel patches, if any :-) Eugene Grosbein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F571870.3090902>