Date: Fri, 29 Oct 2010 11:11:18 -0700 From: Pyun YongHyeon <pyunyh@gmail.com> To: =?utf-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?= <kes-kes@yandex.ru> Cc: freebsd-net@freebsd.org Subject: Re: Polling slows down bandwidth Message-ID: <20101029181118.GB19479@michelle.cdnetworks.com> In-Reply-To: <1519248747.20101028232111@yandex.ru> References: <1519248747.20101028232111@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Oct 28, 2010 at 11:21:11PM +0300, ?????????????? ?????????????? wrote: > Hello, > w/0 polling: > > > serv1# ifconfig nfe0 > nfe0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=10b<RXCSUM,TXCSUM,VLAN_MTU,TSO4> > ether 00:13:d4:ce:82:16 > inet 10.11.8.17 netmask 0xfffffc00 broadcast 10.11.11.255 > inet 10.11.15.15 netmask 0xffffff00 broadcast 10.11.15.255 > inet 10.11.8.15 netmask 0xfffffc00 broadcast 10.11.11.255 > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > > serv2# ifconfig re0 > re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC> > ether 00:1c:c0:c8:5a:4e > inet 192.168.255.254 netmask 0xffffffff broadcast 192.168.255.254 > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > > > serv1# systat -v > 2 users Load 0.38 0.49 0.38 Oct 28 23:10 > > Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER > Tot Share Tot Share Free in out in out > Act 324452 20196 490400 25112 938576 count > All 346596 21060 1074286k 28104 pages > Proc: Interrupts > r p d s w Csw Trp Sys Int Sof Flt cow 29953 total > 1 47 71k 18 5022 27k 2029 1 zfod atkbd0 1 > ozfod 7 ata0 irq14 > 62.4%Sys 18.7%Intr 0.4%User 0.0%Nice 18.4%Idle %ozfod 27944 nfe0 irq23 > | | | | | | | | | | | daefr 2001 cpu0: time > ===============================++++++++++ prcfr igb0 256 > 1 dtbuf 6 totfr 1 igb0 257 > Namei Name-cache Dir-cache 100000 desvn react igb0 258 > Calls hits % hits % 84106 numvn pdwak > 5 5 100 24824 frevn pdpgs > intrn > Disks ad0 373808 wire > KB/t 16.00 292088 act > tps 7 417568 inact > MB/s 0.10 200 cache > %busy 0 938376 free > > iperf result: > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.3 sec 450 MBytes 368 Mbits/sec > > > after enable POLLING: > serv1# ifconfig nfe0 polling > 2 users Load 0.32 0.39 0.35 Oct 28 23:13 > > Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER > Tot Share Tot Share Free in out in out > Act 324464 20196 490656 25112 938428 count > All 346624 21060 1074286k 28104 pages > Proc: Interrupts > r p d s w Csw Trp Sys Int Sof Flt cow 2006 total > 1 47 28k 19 2691 6 2036 2 zfod atkbd0 1 > ozfod 4 ata0 irq14 > 24.7%Sys 18.6%Intr 0.7%User 0.0%Nice 55.9%Idle %ozfod nfe0 irq23 > | | | | | | | | | | | daefr 2001 cpu0: time > ============++++++++++ prcfr igb0 256 > 10 dtbuf 2 totfr 1 igb0 257 > Namei Name-cache Dir-cache 100000 desvn react igb0 258 > Calls hits % hits % 84106 numvn pdwak > 20 20 100 24824 frevn pdpgs > intrn > Disks ad0 373944 wire > KB/t 16.00 292104 act > tps 4 417564 inact > MB/s 0.07 200 cache > %busy 0 938228 free > > I get bad results ((( > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.3 sec 180 MBytes 147 Mbits/sec > nfe(4) controllers are one of rare gigabit controllers that lacks efficient interrupt moderation mechanism. So it's normal to see high number of interrupts under load. hz controls how frequently checks controller's RX/TX queue so you may have to increase hz to get reasonable performance under high network load with polling(4). One of important performance factor for NIC is how many frames should be processed for given network load pattern. If you have to process more frames you have to increase hz in polling(4). I don't know what nfe(4) controller you have but it seems it's somewhat low-end model because MSI/MSIX is not used at all.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101029181118.GB19479>