Date: Thu, 06 Oct 2005 09:22:17 +1000 From: Dave+Seddon <dave-dated-1128986539.c8b6a2@seddon.ca> To: ferdinand.goldmann@jku.at Cc: net@freebsd.org Subject: Re: dummynet, em driver, device polling issues :-(( Message-ID: <1128554539.19942.TMDA@seddon.ca> In-Reply-To: <4343C559.5000000@jku.at> References: <4341089F.7010504@jku.at> <20051003104548.GB70355@cell.sick.ru> <4341242F.9060602@jku.at> <20051003123210.GF70355@cell.sick.ru> <43426EF3.3020404@jku.at> <9CD8C672-1EF2-42FE-A61E-83DC684C893D@dragondata.com> <43429157.90606@jku.at> <4342987D.7000200@benswebs.com> <20051004161217.GB43195@obiwan.tataz.chchile.org> <1128470191.75484.TMDA@seddon.ca> <979B163D-7078-4558-9095-DC329707A5B4@dragondata.com> <4343C559.5000000@jku.at>
next in thread | previous in thread | raw e-mail | index | archive | help
Greetings, The default values are based on 100 MB/s fxp driver. Luigi did heaps of work a few years ago on this, and arrived at these values after lots of testing (i think). (I also remember reading some interesting stuff where he had fxp and a 3com card and was testing to see how many frames he could push out of each differnet card - fxp won!). Given we're now running 1000MB/s em cards, it might be safe to say you can increase the defaults by 10. You have to edit the source to change some of the defaults: /usr/src/sys/kern/kern_poll.c #define MIN_POLL_BURST_MAX 100 #define MAX_POLL_BURST_MAX 10000 I tried doing this, but encountered the problems with the throughput somehow related to the em cards and gave up. Maybe you're results will be better. Regards, Dave Seddon Ferdinand Goldmann writes: > Kevin Day wrote: > >> In one case, we had a system acting as a router. It was a Dell PowerEdge >> 2650, with two dual "server" adapters. each were on separate PCI busses. >> 3 were "lan" links, and one was a "wan" link. The lan links were >> receiving about 300mbps each, all going out the "wan" link at near >> 900mbps at peak. We were never able to get above 944mbps, but I never >> cared enough to figure out where the bottleneck was there. > > 944mbps is a very good value, anyway. What we see in our setup are > throuput rates around 300mbps or below. When testing with tcpspray, > throughput hardly exceeded 13MB/s. > > Are you running vlans on your interface? Our em0-card connects several > sites together, which are all sitting on separate vlan interfaces for > which the em0 acts as parent interface. > >> This was with PCI-X, and a pretty stripped config on the server side. > > Maybe this makes a difference, too. We only have a quite old xSeries 330 > with PCI and a 1.2GHz CPU. > >> >> Nothing fancy on polling, i think we set HZ to 10000 > > Ten-thousand? Or is this a typo, and did you mean thousand? > > This is weird. :-( Please, is there any good documentation on tuning > device polling? The man page does not give any useful pointers about > values to use for Gbit cards. I have already read things about people > using 2000, 4000HZ ... Gaaah! > > I tried with 1000 and 2000 so far, without good results. It seems like > everybody makes wild assumptions on what values to use for polling. > >> , turned on idle_poll, and set user_frac to 10 because we had some cpu >> hungry tasks that were not a high priority. > > I think I red somewhere about problems with idle_poll. How high is your > burst_max value? Are you seeing a lot of ierrs? > > *sigh* :-( confusing. > > -- > >> Ferdinand Goldmann //// | | > >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at |--00 | UNIX | > >> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397 C ^ | | > >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at \ ~/ ~~~|~~~~~~~~ > >> PGP D4CF 8AA4 4B2A 7B88 65CA 5EDC 0A9B FA9A 13EA B993| |-----3 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1128554539.19942.TMDA>