Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 21 Nov 2020 22:22:00 +0000
From:      "Saad, Mark" <Mark.Saad@lucera.com>
To:        Michael Sierchio <kudzu@tenebras.com>, "Sean C. Farley" <scf@freebsd.org>
Cc:        "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject:   Re: Determining cause of transfer limit
Message-ID:  <BL0PR12MB4756681FCB179E3011C233B095FE0@BL0PR12MB4756.namprd12.prod.outlook.com>
In-Reply-To: <CAHu1Y73dxUeAA1cMboU4TBsHWYDsa-C9F7P2iACbrWQN2f%2BygA@mail.gmail.com>
References:  <9d7b39fb-7c1-fe7b-fa9a-ab1aa89cb96a@FreeBSD.org>, <CAHu1Y73dxUeAA1cMboU4TBsHWYDsa-C9F7P2iACbrWQN2f%2BygA@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
scf
  can I use polling on sfxge ? Also if I want to use polling on say ix or s=
fxge
does this break things like pf, bird or nsd ?

---
Mark Saad
Lucera Financial Infrastructures, LLC
msaad@lucera.com


________________________________________
From: owner-freebsd-net@freebsd.org <owner-freebsd-net@freebsd.org> on beha=
lf of Michael Sierchio <kudzu@tenebras.com>
Sent: Saturday, November 21, 2020 4:55 PM
To: Sean C. Farley
Cc: freebsd-net@freebsd.org
Subject: Re: Determining cause of transfer limit

Sorry for the top post.  Have you tried device polling? From
/usr/src/sys/amd64/conf/NOTES:

#####################################################################

# NETWORKING OPTIONS


#

# DEVICE_POLLING adds support for mixed interrupt-polling handling

# of network device drivers, which has significant benefits in terms

# of robustness to overloads and responsivity, as well as permitting

# accurate scheduling of the CPU time between kernel network processing

# and other activities.  The drawback is a moderate (up to 1/HZ seconds)

# potential increase in response times.

# It is strongly recommended to use HZ=3D1000 or 2000 with DEVICE_POLLING

# to achieve smoother behaviour.

# Additionally, you can enable/disable polling at runtime with help of

# the ifconfig(8) utility, and select the CPU fraction reserved to

# userland with the sysctl variable kern.polling.user_frac

# (default 50, range 0..100).

#

# Not all device drivers support this mode of operation at the time of

# this writing.  See polling(4) for more details.


options         DEVICE_POLLING

On Sat, Nov 21, 2020 at 10:24 AM Sean C. Farley <scf@freebsd.org> wrote:

> I have recently upped my Internet service and have now noticed a limit
> being reached, but I am not certain which limit and best option to
> resolve it.
>
> I am using a circa 2007 system as a multi-purpose router running FreeBSD
> 12-STABLE (r367740).  The issue is that it maxes out around 400Mb/s when
> running a speed test through it between my workstation and various test
> sites (i.e., DSL Reports and Speedtest).  There are two NIC's (both are
> Intel 82541PI) in use with one to the ISP and one to my workstation.
>
> At first, I saw one of them apparently hitting an interrupt rate of just
> over 8000, so I bumped their rate limits higher with little to no
> improvement.
>
> What makes me believe I can theoretically get faster speeds is that I
> can use the onboard NIC (Marvell 88E8056) to replace one of the NIC's
> and nearly double the speed.  The difference is that it is on the PCI-E
> bus and has MSI support.
>
> irq16: em0:irq0+
> irq17: em1:irq0
> irq20: hpet0
> irq258: mskc0
>
> I have many network settings, but changing them did nothing.  Here are
> the settings I am trying now that seem to squeak a little extra
> performance.  The commented-out lines are ones I tried without seeing
> any change.  I have also tested without these settings.
>
> /boot/loader.conf
> hw.em.rx_process_limit=3D"-1"
> # dev.em.0.iflib.override_nrxds=3D"2048"
> # dev.em.1.iflib.override_nrxds=3D"2048"
> # dev.em.2.iflib.override_nrxds=3D"2048"
> # dev.em.0.iflib.override_ntxds=3D"2048"
> # net.link.ifqmaxlen=3D"2048"
> hw.em.max_interrupt_rate=3D"32000"
> # net.isr.maxthreads=3D"-1"
> # net.isr.bindthreads=3D"1"
>
> /etc/sysctl.conf
> kern.random.harvest.mask=3D351
> dev.em.0.fc=3D0
> dev.em.1.fc=3D0
> dev.em.0.itr=3D122        # Allow past 8000 interrupts/second.
> dev.em.1.itr=3D122
> net.inet.ip.redirect=3D0
> net.inet6.ip6.redirect=3D0
>
> Increasing these from 66 to 250 did not help:
> hw.em.rx_abs_int_delay: 66
> hw.em.tx_abs_int_delay: 66
> hw.em.tx_int_delay: 66
>
> I am utilizing pf, but I doubt it is the issue since using the same
> rules with the msk driver would have held the speed down to 400Mb/s.
>
> Am I hitting the limit of the PCI bus (memory or interrupt) or something
> else?  I can buy a new PCI-E NIC for the internal network, but I rather
> fully utilize the Intel NIC's I have, if possible.
>
> Sean
> --
> scf@FreeBSD.org
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>


--

"Well," Brahm=E0 said, "even after ten thousand explanations, a fool is no
wiser, but an intelligent person requires only two thousand five hundred."

- The Mah=E0bh=E0rata
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BL0PR12MB4756681FCB179E3011C233B095FE0>