Date: Mon, 19 Jan 2015 21:22:42 -0800 From: Adrian Chadd <adrian@freebsd.org> To: Hans Petter Selasky <hps@selasky.org> Cc: "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org> Subject: Re: svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys Message-ID: <CAJ-VmomHUxRLfsBLNj_TBYS0sfZq0LgMM59hhy84E8pPwh3KpA@mail.gmail.com> In-Reply-To: <54BDE58F.3070909@selasky.org> References: <201501151532.t0FFWV2Y037455@svn.freebsd.org> <CAJ-Vmok0GXZoojyi=jE=b5D-d338APztaf3Pw0_AAQ-173XSWw@mail.gmail.com> <54BDD9E1.6090505@selasky.org> <CAJ-VmokUJPzHqwjQT%2BRPh5afYbQnCAr1FdEUf_Z=_-OCg14ApQ@mail.gmail.com> <54BDE58F.3070909@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 19 January 2015 at 21:20, Hans Petter Selasky <hps@selasky.org> wrote: > On 01/20/15 06:04, Adrian Chadd wrote: >> >> On 19 January 2015 at 20:30, Hans Petter Selasky <hps@selasky.org> wrote: >>> >>> On 01/19/15 22:59, Adrian Chadd wrote: >>>> >>>> >>>> Hi, >>>> >>>> Would you please check what the results of this are with CPU specific >>>> callwheels? >>>> >>>> I'm doing some 10+ gig traffic testing on -HEAD with RSS enabled (on >>>> ixgbe) and with this setup, the per-CPU TCP callwheel stuff is >>>> enabled. But all the callwheels are now back on clock(0) and so is the >>>> lock contention. :( >>>> >>>> Thanks, >>>> >>> >>> Hi, >>> >>> Like stated in the manual page, callout_reset_curcpu/on() does not work >>> with >>> MPSAFE callouts any more! >> >> >> Hm! >> > > Hi Adrian, > >> How many places in the kernel did you leave like this? :P > > > :-) > >> >> I mean, I'm glad to have stuff be forced to be cleaned up, but you >> didn't even leave a KASSERT or a debug warning that something >> unsupported is being done. I'm sure I'm not going to be the first >> person to be caught out like this. > > > MPSAFE is still valid and fully useable and can be used with > callout_reset_curcpu/on(), but the callout CPU will remain at zero. > There is no need for a KASSERT() yet. Right, but people won't know that their callout won't be scheduled on the CPU they've asked for and there's no way to get notified of that. So something debug-y may be useful, even to make it easy to track down situations where sometimes it succeeds and sometimes it fails (because well, there's lots of places in the kernel where locking is .. suboptimal.) >> >>> You need to use callout_init_{mtx,rm,rw} and remove the custom locking >>> inside the callback in the TCP stack to get it working like before! >> >> >> Would you please give me a hand with this? I've sunk a lot of (unpaid, >> personal) spare time into getting the RSS stuff into shape and now a >> lot of it just plainly doesn't do anything. :( > > > I'll send you a patch in an hours time from now for 11-current. This should > be fairly trivial and then you can test and review it! Sweet, thanks. I'l test it, but anything that changes the locking to TCP is going to need a more thorough review. The "there be dragons" disclaimer is appropriate. :) -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmomHUxRLfsBLNj_TBYS0sfZq0LgMM59hhy84E8pPwh3KpA>