Date: Tue, 9 Mar 2010 14:04:57 -0800 From: "David Christensen" <davidch@broadcom.com> To: "pyunyh@gmail.com" <pyunyh@gmail.com> Cc: Ian FREISLICH <ianf@clue.co.za>, "current@freebsd.org" <current@freebsd.org>, David Christensen <davidch@freebsd.org> Subject: RE: dev.bce.X.com_no_buffers increasing and packet loss Message-ID: <5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7@IRVEXCHCCR01.corp.ad.broadcom.com> In-Reply-To: <20100309214012.GQ1311@michelle.cdnetworks.com> References: <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <E1NnVaT-0003Ft-3p@clue.co.za> <E1Nnc4d-0003mB-6e@clue.co.za> <E1Nne0Q-0003uZ-OR@clue.co.za> <E1Noulp-0007Rc-Ro@clue.co.za> <20100309212139.GO1311@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EED69@IRVEXCHCCR01.corp.ad.broadcom.com> <20100309214012.GQ1311@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > > It's been running for about 1:23 on the patched driver. > I'm still > > > > seeing the com_no_buffers increase: > > > > > > > > [firewall2.jnb1] ~ # sysctl dev.bce |grep com_no_buffers > > > > dev.bce.0.com_no_buffers: 5642 > > > > dev.bce.1.com_no_buffers: 497 > > > > dev.bce.2.com_no_buffers: 6260612 > > > > dev.bce.3.com_no_buffers: 4871338 > > > > > > > > > > Still have no idea why these counters are increasing here. > > > Actually the counter is read from scratch pad of completion > > > processor. The datasheet does not even document the counter. > > > Maybe david know better what's happening here(CCed). > > > > > > > Interupt rate is down now, at about 3500 per second per > interface. > > > > > > > > Interestingly setting net.inet.ip.fastforwarding=0 reduces CPU > > > > consumption from 25% to 9% and less packet loss. > > > > The com_no_buffers statistic comes from firmware and indicates how > > many times a valid frame was received but could not be > placed into the > > receive chain because there were no available RX buffers. The > > firmware will then drop the frame but that dropped frame won't be > > reflected in any of the hardware based statistics. > > > > Yeah, but the question is why bce(4) has no available RX buffers. > The system has a lot of available mbufs so I don't see the > root cause here. What's the traffic look like? Jumbo, standard, short frames? Any good ideas on profiling the code? I haven't figured out how to use the CPU TSC but there is a free running timer on the device that might be usable to calculate where the driver's time is spent. Dave
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7>