Date: Sat, 21 Nov 2020 13:24:23 -0500 (EST) From: "Sean C. Farley" <scf@FreeBSD.org> To: freebsd-net@FreeBSD.org Subject: Determining cause of transfer limit Message-ID: <9d7b39fb-7c1-fe7b-fa9a-ab1aa89cb96a@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
I have recently upped my Internet service and have now noticed a limit being reached, but I am not certain which limit and best option to resolve it. I am using a circa 2007 system as a multi-purpose router running FreeBSD 12-STABLE (r367740). The issue is that it maxes out around 400Mb/s when running a speed test through it between my workstation and various test sites (i.e., DSL Reports and Speedtest). There are two NIC's (both are Intel 82541PI) in use with one to the ISP and one to my workstation. At first, I saw one of them apparently hitting an interrupt rate of just over 8000, so I bumped their rate limits higher with little to no improvement. What makes me believe I can theoretically get faster speeds is that I can use the onboard NIC (Marvell 88E8056) to replace one of the NIC's and nearly double the speed. The difference is that it is on the PCI-E bus and has MSI support. irq16: em0:irq0+ irq17: em1:irq0 irq20: hpet0 irq258: mskc0 I have many network settings, but changing them did nothing. Here are the settings I am trying now that seem to squeak a little extra performance. The commented-out lines are ones I tried without seeing any change. I have also tested without these settings. /boot/loader.conf hw.em.rx_process_limit="-1" # dev.em.0.iflib.override_nrxds="2048" # dev.em.1.iflib.override_nrxds="2048" # dev.em.2.iflib.override_nrxds="2048" # dev.em.0.iflib.override_ntxds="2048" # net.link.ifqmaxlen="2048" hw.em.max_interrupt_rate="32000" # net.isr.maxthreads="-1" # net.isr.bindthreads="1" /etc/sysctl.conf kern.random.harvest.mask=351 dev.em.0.fc=0 dev.em.1.fc=0 dev.em.0.itr=122 # Allow past 8000 interrupts/second. dev.em.1.itr=122 net.inet.ip.redirect=0 net.inet6.ip6.redirect=0 Increasing these from 66 to 250 did not help: hw.em.rx_abs_int_delay: 66 hw.em.tx_abs_int_delay: 66 hw.em.tx_int_delay: 66 I am utilizing pf, but I doubt it is the issue since using the same rules with the msk driver would have held the speed down to 400Mb/s. Am I hitting the limit of the PCI bus (memory or interrupt) or something else? I can buy a new PCI-E NIC for the internal network, but I rather fully utilize the Intel NIC's I have, if possible. Sean -- scf@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9d7b39fb-7c1-fe7b-fa9a-ab1aa89cb96a>