Date: Fri, 2 Feb 2024 19:15:39 -0800 From: Rick Macklem <rick.macklem@gmail.com> To: Drew Gallatin <gallatin@freebsd.org> Cc: Richard Scheffenegger <rscheff@freebsd.org>, "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org>, FreeBSD Transport <freebsd-transport@freebsd.org>, rmacklem@freebsd.org, kp@freebsd.org Subject: Re: Increasing TCP TSO size support Message-ID: <CAM5tNy61PapZ-KtyOPzc_O=8wMzNr6tX69Nq8tPoaf0QJ7utyA@mail.gmail.com> In-Reply-To: <2fac0ac3-ba3a-4bca-b0d4-fafb0c5b75fd@app.fastmail.com> References: <2c31ac44-b34b-469c-a6de-fdd927ec2f9e@freebsd.org> <CAM5tNy6TbvXqrRRD=XpDBRGk81rzW5k38AzXeKFKLDL01fOYQQ@mail.gmail.com> <e5df5725-ac9c-4e88-ade5-b0a561bfacd6@app.fastmail.com> <CAM5tNy7pSDGQK-JzceB1S-nX1xy8dz5j5m_jwXt5uwr7WN-q0w@mail.gmail.com> <2fac0ac3-ba3a-4bca-b0d4-fafb0c5b75fd@app.fastmail.com>
index | next in thread | previous in thread | raw e-mail
On Fri, Feb 2, 2024 at 6:20 PM Drew Gallatin <gallatin@freebsd.org> wrote: > > > > On Fri, Feb 2, 2024, at 9:05 PM, Rick Macklem wrote: > > > But the page size is only 4K on most platforms. So while an M_EXTPGS mbuf can hold 5 pages (..from memory, too lazy to do the math right now) and reduces socket buffer mbuf chain lengths by a factor of 10 or so (2k vs 20k per mbuf), the S/G list that a NIC will need to consume would likely decrease only by a factor of 2. And even then only if the busdma code to map mbufs for DMA is not coalescing adjacent mbufs. I know busdma does some coalescing, but I can't recall if it coalesces physcally adjacent mbufs. > > I'm guessing the factor of 2 comes from the fact that each page is a > contiguous segment? > > > Actually, no, I'm being dumb. I was thinking that pages would be split up, but that's wrong. Without M_EXTPGS, each mbuf generated by sendfile (or nfs) would be an M_EXT with a wrapper around a single 4K page. So the scatter/gather list would be exactly the same. > > The win would be if the pages themselves were contiguous (which they often are), and if the bus_dma mbuf mapping code coalesced those segments, and if the device could handle DMA across a 4K boundary. That's what would get you shorter s/g lists. > > I think tcp_m_copy() can handle this now, as if_hw_tsomaxsegsize is set by the driver to express how long the max contiguous segment they can handle is. Sounds good. I'll give it a try someday soon (April maybe). Thanks for all the good info, rick > > BTW, I really hate the mixing of bus dma restrictions with the hw_tsomax stuff. It always makes my head explode.. > > Drew >help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy61PapZ-KtyOPzc_O=8wMzNr6tX69Nq8tPoaf0QJ7utyA>