Date: Wed, 02 Dec 2009 18:05:42 +0200 From: "Yuriy A. Korobko" <administrator@shtorm.com> To: Barney Cordoba <barney_cordoba@yahoo.com> Cc: freebsd-net@freebsd.org Subject: Re: Reducing number of interrupts from intel pro 1000 et adapter Message-ID: <1259769942.6739.33.camel@stormi-desktop> In-Reply-To: <913529.24196.qm@web63906.mail.re1.yahoo.com> References: <913529.24196.qm@web63906.mail.re1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > > I'd like to know a way to control tx interrupts on intel > > pro 1000 et > > adapter with igb driver. Just installed one in the router > > and systat > > shows 8-9k rx interrupts and 20k tx interrupts from igb0 > > and igb1 > > adapters. Box is a router running freebsd 7.2 release, I've > > tried > > default driver from kernel source and latest from intel > > site, effect is > > the same with automatic interrupt moderation enabled and > > disabled. I > > have the same box with intel pro 1000 pt adapter which > > have > > tx(rx)_int_delay sysctls in em driver, I was able to reduce > > number of > > tx/rx interrupts to 7-8k per interface and got much more > > cpu idle > > because of less context switches with same pps. > > > I'm curious as to why you are doing a load test on a single core system > with a part that is clearly designed to be used on a multicore system? > > Barney Box have core quad cpu: 87 processes: 7 running, 58 sleeping, 22 waiting CPU 0: 0.0% user, 0.0% nice, 3.1% system, 36.4% interrupt, 60.5% idle CPU 1: 0.0% user, 0.0% nice, 15.5% system, 41.1% interrupt, 43.4% idle CPU 2: 0.0% user, 0.0% nice, 11.6% system, 27.1% interrupt, 61.2% idle CPU 3: 0.0% user, 0.0% nice, 2.3% system, 41.5% interrupt, 56.2% idle Mem: 192M Active, 206M Inact, 246M Wired, 716K Cache, 112M Buf, 1356M Anyway, high interrupt rate fixed by border# cat /boot/loader.conf | grep igb if_igb_load="YES" hw.igb.rxd=4096 hw.igb.txd=4096 hw.igb.num_queues=1 hw.igb.enable_aim=1 hw.igb.low_latency=1000 hw.igb.ave_latency=2000 hw.igb.bulk_latency=4000 hw.igb.rx_process_limit=200 hw.igb.fc_setting=0 hw.igb.lro=0 border# Have no idea why, but it seems like setting dev.igb. sysctls at runtime just ignored by the driver. Now I have 1000-1200 rx and around 2000 tx interrupts per second from interface with 100 kpps flow in each direction, main cpu eater on the router is dummynet right now, it will be awesome if one day we can have it multithreaded. I'm using latest igb driver from intel site, default driver from kernel may not have this issue. Thanks.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1259769942.6739.33.camel>