Date: Sat, 21 Nov 2020 13:55:42 -0800 From: Michael Sierchio <kudzu@tenebras.com> To: "Sean C. Farley" <scf@freebsd.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Determining cause of transfer limit Message-ID: <CAHu1Y73dxUeAA1cMboU4TBsHWYDsa-C9F7P2iACbrWQN2f%2BygA@mail.gmail.com> In-Reply-To: <9d7b39fb-7c1-fe7b-fa9a-ab1aa89cb96a@FreeBSD.org> References: <9d7b39fb-7c1-fe7b-fa9a-ab1aa89cb96a@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Sorry for the top post. Have you tried device polling? From /usr/src/sys/amd64/conf/NOTES: ##################################################################### # NETWORKING OPTIONS # # DEVICE_POLLING adds support for mixed interrupt-polling handling # of network device drivers, which has significant benefits in terms # of robustness to overloads and responsivity, as well as permitting # accurate scheduling of the CPU time between kernel network processing # and other activities. The drawback is a moderate (up to 1/HZ seconds) # potential increase in response times. # It is strongly recommended to use HZ=3D1000 or 2000 with DEVICE_POLLING # to achieve smoother behaviour. # Additionally, you can enable/disable polling at runtime with help of # the ifconfig(8) utility, and select the CPU fraction reserved to # userland with the sysctl variable kern.polling.user_frac # (default 50, range 0..100). # # Not all device drivers support this mode of operation at the time of # this writing. See polling(4) for more details. options DEVICE_POLLING On Sat, Nov 21, 2020 at 10:24 AM Sean C. Farley <scf@freebsd.org> wrote: > I have recently upped my Internet service and have now noticed a limit > being reached, but I am not certain which limit and best option to > resolve it. > > I am using a circa 2007 system as a multi-purpose router running FreeBSD > 12-STABLE (r367740). The issue is that it maxes out around 400Mb/s when > running a speed test through it between my workstation and various test > sites (i.e., DSL Reports and Speedtest). There are two NIC's (both are > Intel 82541PI) in use with one to the ISP and one to my workstation. > > At first, I saw one of them apparently hitting an interrupt rate of just > over 8000, so I bumped their rate limits higher with little to no > improvement. > > What makes me believe I can theoretically get faster speeds is that I > can use the onboard NIC (Marvell 88E8056) to replace one of the NIC's > and nearly double the speed. The difference is that it is on the PCI-E > bus and has MSI support. > > irq16: em0:irq0+ > irq17: em1:irq0 > irq20: hpet0 > irq258: mskc0 > > I have many network settings, but changing them did nothing. Here are > the settings I am trying now that seem to squeak a little extra > performance. The commented-out lines are ones I tried without seeing > any change. I have also tested without these settings. > > /boot/loader.conf > hw.em.rx_process_limit=3D"-1" > # dev.em.0.iflib.override_nrxds=3D"2048" > # dev.em.1.iflib.override_nrxds=3D"2048" > # dev.em.2.iflib.override_nrxds=3D"2048" > # dev.em.0.iflib.override_ntxds=3D"2048" > # net.link.ifqmaxlen=3D"2048" > hw.em.max_interrupt_rate=3D"32000" > # net.isr.maxthreads=3D"-1" > # net.isr.bindthreads=3D"1" > > /etc/sysctl.conf > kern.random.harvest.mask=3D351 > dev.em.0.fc=3D0 > dev.em.1.fc=3D0 > dev.em.0.itr=3D122 # Allow past 8000 interrupts/second. > dev.em.1.itr=3D122 > net.inet.ip.redirect=3D0 > net.inet6.ip6.redirect=3D0 > > Increasing these from 66 to 250 did not help: > hw.em.rx_abs_int_delay: 66 > hw.em.tx_abs_int_delay: 66 > hw.em.tx_int_delay: 66 > > I am utilizing pf, but I doubt it is the issue since using the same > rules with the msk driver would have held the speed down to 400Mb/s. > > Am I hitting the limit of the PCI bus (memory or interrupt) or something > else? I can buy a new PCI-E NIC for the internal network, but I rather > fully utilize the Intel NIC's I have, if possible. > > Sean > -- > scf@FreeBSD.org > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 "Well," Brahm=C4=81 said, "even after ten thousand explanations, a fool is = no wiser, but an intelligent person requires only two thousand five hundred." - The Mah=C4=81bh=C4=81rata
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHu1Y73dxUeAA1cMboU4TBsHWYDsa-C9F7P2iACbrWQN2f%2BygA>