Date: Mon, 18 Mar 2024 21:41:47 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Drew Gallatin <gallatin@freebsd.org> Cc: Mike Karels <mike@karels.net>, tuexen <tuexen@freebsd.org>, Nuno Teixeira <eduardo@freebsd.org>, garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart <rrs@freebsd.org> Subject: Re: Request for Testing: TCP RACK Message-ID: <ZfiY-xUUM3wrBEz_@kib.kiev.ua> In-Reply-To: <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> References: <CAFDf7U%2BAjfeY%2Bqjq%2B-R71w5i1pRoxQdOmqJ9w4s1U13AA8-duA@mail.gmail.com> <C5D50314-4B0C-42F6-AA67-B5A32A4BA335@freebsd.org> <CAFDf7UKL6vtKo1Mn9Vw_5OD9Xubuw%2BdgS83WKwsiTUaXHs8D6Q@mail.gmail.com> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <CAFDf7UKDWSnhm%2BTwP=ZZ9dkk0jmAgjGKPLpkX-CKuw3yH233gQ@mail.gmail.com> <CAFDf7UJq9SCnU-QYmS3t6EknP369w2LR0dNkQAc-NaRLvwVfoQ@mail.gmail.com> <A3F1FC0C-C199-4565-8E07-B233ED9E7B2E@freebsd.org> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> <ZfiI7GcbTwSG8kkO@kib.kiev.ua> <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin wrote: > I got the idea from > https://people.mpi-sws.org/~druschel/publications/soft-timers-tocs.pdf > The gist is that the TCP pacing stuff needs to run frequently, and > rather than run it out of a clock interrupt, its more efficient to run > it out of a system call context at just the point where we return to > userspace and the cache is trashed anyway. The current implementation > is fine for our workload, but probably not idea for a generic system. > Especially one where something is banging on system calls. > > Ast's could be the right tool for this, but I'm super unfamiliar with > them, and I can't find any docs on them. > > Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to > what's happening here? This call would need some AST number added, and then it registers the ast to run on next return to userspace, for the current thread. Is it enough? > > Drew > > On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira <eduardo@freebsd.org> wrote: > > > >> > > > >> Hello all! > > > >> > > > >> It works just fine! > > > >> System performance is OK. > > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > > >> > > > >> --- > > > >> net.inet.tcp.functions_available: > > > >> Stack D Alias PCB count > > > >> freebsd freebsd 0 > > > >> rack * rack 38 > > > >> --- > > > >> > > > >> It would be so nice that we can have a sysctl tunnable for this patch > > > >> so we could do more tests without recompiling kernel. > > > > Thanks for testing! > > > > > > > > @gallatin: can you come up with a patch that is acceptable for Netflix > > > > and allows to mitigate the performance regression. > > > > > > Ideally, tcphpts could enable this automatically when it starts to be > > > used (enough?), but a sysctl could select auto/on/off. > > There is already a well-known mechanism to request execution of the > > specific function on return to userspace, namely AST. The difference > > with the current hack is that the execution is requested for one callback > > in the context of the specific thread. > > > > Still, it might be worth a try to use it; what is the reason to hit a thread > > that does not do networking, with TCP processing? > > > > > > > > Mike > > > > > > > Best regards > > > > Michael > > > >> > > > >> Thanks all! > > > >> Really happy here :) > > > >> > > > >> Cheers, > > > >> > > > >> Nuno Teixeira <eduardo@freebsd.org> escreveu (domingo, 17/03/2024 à(s) 20:26): > > > >>> > > > >>> Hello, > > > >>> > > > >>>> I don't have the full context, but it seems like the complaint is a performance regression in bonnie++ and perhaps other things when tcp_hpts is loaded, even when it is not used. Is that correct? > > > >>>> > > > >>>> If so, I suspect its because we drive the tcp_hpts_softclock() routine from userret(), in order to avoid tons of timer interrupts and context switches. To test this theory, you could apply a patch like: > > > >>> > > > >>> It's affecting overall system performance, bonnie was just a way to > > > >>> get some numbers to compare. > > > >>> > > > >>> Tomorrow I will test patch. > > > >>> > > > >>> Thanks! > > > >>> > > > >>> -- > > > >>> Nuno Teixeira > > > >>> FreeBSD Committer (ports) > > > >> > > > >> > > > >> > > > >> -- > > > >> Nuno Teixeira > > > >> FreeBSD Committer (ports) > > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ZfiY-xUUM3wrBEz_>