Date: Sat, 26 Apr 2025 12:34:22 -0600 From: Brett Glass <brett@lariat.net> To: Paul Procacci <pprocacci@gmail.com> Cc: questions@freebsd.org Subject: Re: Avoiding the lagg(4) bottleneck Message-ID: <202504261834.MAA20118@mail.lariat.net> In-Reply-To: <CAFbbPujJzc_VB4E1YM%2BW_-s6z6irzUshVe37vGRUouLqicOpsQ@mail.g mail.com> References: <202504210148.TAA26419@mail.lariat.net> <CAFbbPujJzc_VB4E1YM%2BW_-s6z6irzUshVe37vGRUouLqicOpsQ@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
All:
The LAG configuration is about as simple as it can be:
ifconfig_igb2="up -tso4 -tso6 -lro -vlanhwtso media 1000baseTX
mediaopt full-duplex"
ifconfig_igb3="up -tso4 -tso6 -lro -vlanhwtso media 1000baseTX
mediaopt full-duplex"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport igb2 laggport igb3 lacp_fast_timeout"
Other sysctls of possible interest (from /boot/loader.conf):
# Disable hyperthreading
machdep.hyperthreading_allowed="0"
# Allowing multithreaded networking yields major load reduction
net.isr.maxthreads="-1"
# Optimizations for NICs
# This is claimed to provide improvement with em/igb driver
hw.em.rx_process_limit="-1"
# Don't be overwhelmed by interrupts
hw.em.max_interrupt_rate=10000
# Don't try to save energy on em/igb interface
hw.em.eee_setting=0
We have tried
net.link.lagg.default_use_flowid=0
but it does not seem to matter.
Our concern is that the lagg(4) driver may be limited by single
threading and/or locks.
We are testing with real life loads, attempting to saturate a pair of gigabit
upstream connections. Try as we might, we can't get 2 Gbps out of
it; in fact, we
have trouble getting 1 Gbps out of the pair.
--Brett Glass
At 10:05 PM 4/20/2025, Paul Procacci wrote:
>What is your lagg configuration?
>How many different ip/port source tuples are involved with your testing?
>How are you testing?
>
>There's no known deficiency with FreeBSD's lagg driver.
>
>~Paul
>
>--
>__________________
>
>:(){ :|:& };:
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202504261834.MAA20118>