Date: Thu, 27 Jan 2011 09:37:23 -0800 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Damien Fleuriot <ml@my.gd> Cc: freebsd-stable@freebsd.org, "Vogel, Jack" <jack.vogel@intel.com>, freebsd-pf@freebsd.org Subject: Re: High interrupt rate on a PF box + performance Message-ID: <20110127173723.GA36846@icarus.home.lan> In-Reply-To: <4D41ABF1.1010405@my.gd> References: <4D41417A.20904@my.gd> <20110127172724.GA36587@icarus.home.lan> <4D41ABF1.1010405@my.gd>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jan 27, 2011 at 06:31:29PM +0100, Damien Fleuriot wrote:
>
>
> On 1/27/11 6:27 PM, Jeremy Chadwick wrote:
> > On Thu, Jan 27, 2011 at 10:57:14AM +0100, Damien Fleuriot wrote:
> >> Hello list,
> >>
> >> I have a problem with interrupts, network cards, and PF performance.
> >>
> >> We have 2 firewalls running FreeBSD 8.0 for the current master and
> >> FreeBSD 8.1 for the backup host, which I upgraded just yesterday.
> >>
> >> [...]
> >>
> >> vmstat -i
> >> ---
> >> interrupt total rate
> >> irq14: ata0 36 0
> >> irq16: mfi0 353244 1
> >> irq21: uhci0 uhci+ 461504 1
> >> cpu0: timer 615183815 1996
> >> irq256: bce0 1015412475 3295
> >> irq257: igb0 1067318584 3464
> >> irq258: igb0 695648752 2258
> >> irq259: igb0 2 0
> >> irq260: igb1 11503857 37
> >> irq261: igb1 506598 1
> >> irq262: igb1 69 0
> >> irq269: bce1 790820 2
> >> cpu1: timer 615183757 1996
> >> cpu2: timer 615197165 1996
> >> cpu3: timer 615197165 1996
> >> Total 5252757843 17050
> >
> > There are changes to the igb(4) driver which are in RELENG_8 (8-STABLE),
> > and some which will be in the upcoming 8.2-RELEASE, which may address
> > this. Jack Vogel of Intel would be able to confirm for sure; CC'ing him
> > here.
> >
> > Could you please provide output from the following commands?
> >
> > * pciconf -lvcb (only include igbX entries, thanks)
> > * sysctl -a | grep msi
> >
> > Thanks.
> >
> > I can't help with the CARP-related issues or other stuff you're
> > experiencing. These issues may all be separate problems, hard to say.
> >
>
>
> igb0@pci0:14:0:0: class=0x020000 card=0x145a8086 chip=0x10d68086
> rev=0x02 hdr=0x00
> igb1@pci0:14:0:1: class=0x020000 card=0x145a8086 chip=0x10d68086
> rev=0x02 hdr=0x00
> igb2@pci0:15:0:0: class=0x020000 card=0x145a8086 chip=0x10d68086
> rev=0x02 hdr=0x00
> igb3@pci0:15:0:1: class=0x020000 card=0x145a8086 chip=0x10d68086
> rev=0x02 hdr=0x00
What you did here was "pciconf -lvcb | grep igb", or something
equivalent. There is output after each of these entries which is highly
relevant. Example for an emX device:
em1@pci0:15:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = 'Intel PRO/1000 PL Network Adaptor (82573L)'
class = network
subclass = ethernet
bar [10] = type Memory, range 32, base 0xdc300000, size 131072, enabled
bar [18] = type I/O Port, range 32, base 0x3000, size 32, enabled
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
This is the sort of output we're looking for.
> hw.bce.msi_enable: 1
> hw.pci.honor_msi_blacklist: 1
> hw.pci.enable_msix: 1
> hw.pci.enable_msi: 1
>
>
> igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
> 0 mtu 1500
> options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
> ether 00:1b:21:12:ec:38
> inet [snip] netmask 0xffffffc0 broadcast [snip]
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
> ether 00:1b:21:12:ec:39
> inet 10.0.0.252 netmask 0xffffff00 broadcast 10.0.0.255
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
>
> Here you go :)
>
> Note that the igb2 and 3 interfaces are unused, unplugged, unconfigured
--
| Jeremy Chadwick jdc@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110127173723.GA36846>