Date: Fri, 2 Feb 2024 19:15:39 -0800 From: Rick Macklem <rick.macklem@gmail.com> To: Drew Gallatin <gallatin@freebsd.org> Cc: Richard Scheffenegger <rscheff@freebsd.org>, "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org>, FreeBSD Transport <freebsd-transport@freebsd.org>, rmacklem@freebsd.org, kp@freebsd.org Subject: Re: Increasing TCP TSO size support Message-ID: <CAM5tNy61PapZ-KtyOPzc_O=8wMzNr6tX69Nq8tPoaf0QJ7utyA@mail.gmail.com> In-Reply-To: <2fac0ac3-ba3a-4bca-b0d4-fafb0c5b75fd@app.fastmail.com> References: <2c31ac44-b34b-469c-a6de-fdd927ec2f9e@freebsd.org> <CAM5tNy6TbvXqrRRD=XpDBRGk81rzW5k38AzXeKFKLDL01fOYQQ@mail.gmail.com> <e5df5725-ac9c-4e88-ade5-b0a561bfacd6@app.fastmail.com> <CAM5tNy7pSDGQK-JzceB1S-nX1xy8dz5j5m_jwXt5uwr7WN-q0w@mail.gmail.com> <2fac0ac3-ba3a-4bca-b0d4-fafb0c5b75fd@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Feb 2, 2024 at 6:20=E2=80=AFPM Drew Gallatin <gallatin@freebsd.org>= wrote: > > > > On Fri, Feb 2, 2024, at 9:05 PM, Rick Macklem wrote: > > > But the page size is only 4K on most platforms. So while an M_EXTPGS m= buf can hold 5 pages (..from memory, too lazy to do the math right now) and= reduces socket buffer mbuf chain lengths by a factor of 10 or so (2k vs 20= k per mbuf), the S/G list that a NIC will need to consume would likely decr= ease only by a factor of 2. And even then only if the busdma code to map m= bufs for DMA is not coalescing adjacent mbufs. I know busdma does some coa= lescing, but I can't recall if it coalesces physcally adjacent mbufs. > > I'm guessing the factor of 2 comes from the fact that each page is a > contiguous segment? > > > Actually, no, I'm being dumb. I was thinking that pages would be split u= p, but that's wrong. Without M_EXTPGS, each mbuf generated by sendfile (or= nfs) would be an M_EXT with a wrapper around a single 4K page. So the sca= tter/gather list would be exactly the same. > > The win would be if the pages themselves were contiguous (which they ofte= n are), and if the bus_dma mbuf mapping code coalesced those segments, and = if the device could handle DMA across a 4K boundary. That's what would get= you shorter s/g lists. > > I think tcp_m_copy() can handle this now, as if_hw_tsomaxsegsize is set b= y the driver to express how long the max contiguous segment they can handle= is. Sounds good. I'll give it a try someday soon (April maybe). Thanks for all the good info, rick > > BTW, I really hate the mixing of bus dma restrictions with the hw_tsomax = stuff. It always makes my head explode.. > > Drew >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy61PapZ-KtyOPzc_O=8wMzNr6tX69Nq8tPoaf0QJ7utyA>