Date: Wed, 27 Feb 2008 20:50:58 +0100 From: Holger Kipp <hk@alogis.com> To: Mike Tancsa <mike@sentex.net> Cc: freebsd-stable@freebsd.org Subject: [solved/workaround?] Re: em very slow, shared irq... on 6.3p8 Message-ID: <20080227195058.GA27997@intserv.int1.b.intern> In-Reply-To: <200802271452.m1REqJ6H059687@lava.sentex.ca> References: <20080227104942.GA14530@intserv.int1.b.intern> <200802271452.m1REqJ6H059687@lava.sentex.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Feb 27, 2008 at 09:50:16AM -0500, Mike Tancsa wrote: more details below. as it currently is, polling seems to do the trick, however handling several em-interfaces with the same irq (mind you, it is pci) shouldn't cause delays of up to 1.5 seconds for a simple ping... Therefore I consider using polling for a nearly idle system more a workaround than a solution to this problem :-( > At 05:49 AM 2/27/2008, Holger Kipp wrote: > > >I therefore assume that the problem is between receiving the > >irq from em<x> and getting the data from the interface on the firewall > >itself. > > I would try upgrading to 6.3R (there are several em driver bug fixes) done. system is now 6.3-RELEASE-p1 which also gave me -c option for pciconf and msi syscontrols (were missing in the old 6.2). > and then try the box with > % cat /boot/loader.conf > hw.pci.enable_msi=1 > > ...if the cards support msi. > > I think pciconf -lvc should tell you if the cards and slots support it or > not. pciconf -lvc says for all em<x>: cap 05[d0] = MSI supports 1 message, 64 bit so I assume they do support MSI. with msi disabled I get 38 packets transmitted, 38 packets received, 0% packet loss round-trip min/avg/max/stddev = 4.833/228.022/1539.337/339.768 ms with msi enabled (via sysctl) I get 33 packets transmitted, 33 packets received, 0% packet loss round-trip min/avg/max/stddev = 1.865/156.421/1339.841/239.375 ms so looks equally bad (I don't consider 30-40 packets a meaningful sample). I don't know if it makes any differences if switched on directly in loader.conf, though. enabling polling (withous MSI) gives 30 packets transmitted, 30 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.366/0.790/1.339/0.290 ms (maybe I should have used HZ=2000 to keep it below 0.6ms ;-) > Also, if you dont need IPV6, use FAST_IPSEC. It does not need > mpsafe. If you do need IPSEC and IPV6, 7.0R got rid of that restriction. I think this enough changes in one go for a production system ;-) many thanks for the recommendations! Best regards, Holger
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080227195058.GA27997>