Date: Tue, 31 Jan 2023 16:32:20 -0500 From: Paul Mather <paul@gromit.dlib.vt.edu> To: Matt Garber <matt.garber@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: Slow WAN traffic to FreeBSD hosts but not to Linux hosts---how to debug/fix? Message-ID: <8A1B625C-D948-43C4-B787-DFB8A25BBF7A@gromit.dlib.vt.edu> In-Reply-To: <CANwXMPNBkWQHzT4f%2Bjy0RV1gK7R6nJOWYsG5NZTf9RadRDNfXg@mail.gmail.com> References: <95EDCFCA-7E3F-458F-85A6-856D606B9D98@gromit.dlib.vt.edu> <tr9g44$d3l$1@ciao.gmane.io> <CANwXMPNBkWQHzT4f%2Bjy0RV1gK7R6nJOWYsG5NZTf9RadRDNfXg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_AE7EC00B-3170-4B90-9FE3-DF492002C429 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On Jan 30, 2023, at 6:30 PM, Matt Garber <matt.garber@gmail.com> wrote: >=20 >> > Any help/insight is gratefully appreciated. >> >=20 >> > Cheers, >> >=20 >> > Paul. >> >=20 >>=20 >> sysctl net.inet.tcp.cc.algorithm=3Dhtcp >>=20 >> I would set "htcp" on the server and home computer to improve through = in=20 >> your type of situation. >=20 >=20 > There may be other FreeBSD sysctls that have bad defaults in this = scenario and could be better tuned, but I doubt changing the CC = algorithm at this point is the problem =E2=80=94 at least not so much a = problem that=E2=80=99s causing throughput to be reduced so drastically. = Happy to be wrong if that does help things quickly and easily, though. Changing the CC algorithm doesn't affect speeds greatly in my case. For = example, changing to HTCP on one of the remote FreeBSD systems improves = speeds somewhat, but not to put them on a par with what I get from the = Linux systems. I can easily get 19--20 MB/s on a 100 MB download from = Linux but I've never managed to get above 10 MB/s at all from the remote = FreeBSD systems, even when switching to HTCP. In addition, the Linux = download quickly gets up to max speed but FreeBSD is slower to increase = speeds and is more variable in the speeds throughout the entire = download. > (Since OP mentioned that FreeBSD CC was set to CUBIC, that would match = what the Linux boxes are using by default, too, unless they=E2=80=99ve = been changed to something newer like BBR=E2=80=A6 so seems like CUBIC = *should* be performing fine on this WAN link, and the difference is = something else.) I suspect the same thing, too, i.e., it is implementation/specific = configuration details that differ between FreeBSD and Linux, especially = when they are both set to use CUBIC. I checked on one of the Linux systems (Rocky Linux 9) and verified it = defaults to CUBIC (net.ipv4.tcp_congestion_control =3D cubic). Cheers, Paul. --Apple-Mail=_AE7EC00B-3170-4B90-9FE3-DF492002C429 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;">On Jan 30, = 2023, at 6:30 PM, Matt Garber <matt.garber@gmail.com> = wrote:<div><br><div><blockquote = type=3D"cite"><div><div><div><div><br></div><div dir=3D"auto"><div = class=3D"gmail_quote"></div></div></div><div><blockquote = class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;borde= r-left-color:rgb(204,204,204)"> > Any help/insight is gratefully appreciated.<br> > <br> > Cheers,<br> > <br> > Paul.<br> > <br> <br> sysctl net.inet.tcp.cc.algorithm=3Dhtcp<br> <br> I would set "htcp" on the server and home computer to improve through in = <br> your type of situation.</blockquote><div dir=3D"auto"><br></div><div = dir=3D"auto"><br></div></div></div><div><div><div dir=3D"auto">There may = be other FreeBSD sysctls that have bad defaults in this scenario and = could be better tuned, but I doubt changing the CC algorithm at this = point is the problem =E2=80=94 at least not so much a problem that=E2=80=99= s causing throughput to be reduced so drastically. Happy to be wrong if = that does help things quickly and easily, = though.</div></div></div></div></blockquote><div><br></div><div><br></div>= <div>Changing the CC algorithm doesn't affect speeds greatly in my case. = For example, changing to HTCP on one of the remote FreeBSD systems = improves speeds somewhat, but not to put them on a par with what I get = from the Linux systems. I can easily get 19--20 MB/s on a 100 MB = download from Linux but I've never managed to get above 10 MB/s at all = from the remote FreeBSD systems, even when switching to HTCP. In = addition, the Linux download quickly gets up to max speed but FreeBSD is = slower to increase speeds and is more variable in the speeds throughout = the entire download.</div><div><br></div><br><blockquote = type=3D"cite"><div><div><div dir=3D"auto">(Since OP mentioned that = FreeBSD CC was set to CUBIC, that would match what the Linux boxes are = using by default, too, unless they=E2=80=99ve been changed to something = newer like BBR=E2=80=A6 so seems like CUBIC *should* be performing fine = on this WAN link, and the difference is something = else.)</div></div></div></blockquote><br></div><div><br></div><div>I = suspect the same thing, too, i.e., it is implementation/specific = configuration details that differ between FreeBSD and Linux, especially = when they are both set to use CUBIC.</div><div><br></div><div>I checked = on one of the Linux systems (Rocky Linux 9) and verified it defaults to = CUBIC (net.ipv4.tcp_congestion_control =3D = cubic).</div><div><br></div><div>Cheers,</div><div><br></div><div>Paul.</d= iv><br></div></body></html>= --Apple-Mail=_AE7EC00B-3170-4B90-9FE3-DF492002C429--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8A1B625C-D948-43C4-B787-DFB8A25BBF7A>