Date: Fri, 7 Feb 2014 17:56:29 -0800 From: Adrian Chadd <adrian@freebsd.org> To: Ryan Stone <rysto32@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: flowtable, collisions, locking and CPU affinity Message-ID: <CAJ-VmonFOGf7AyUNgwroOmjJWfQL4RXp2wZSpj-=wnCTKVoo8g@mail.gmail.com> In-Reply-To: <CAFMmRNyYFte0Q5Jhw0A3Fkya9d8hQ-22pRHYZE62pB_wKb9Caw@mail.gmail.com> References: <CAJ-VmonNCzFED=20_C2fV1g1jvFNRE=N-H%2B09Wb2OdxdzHp9JQ@mail.gmail.com> <CAFMmRNyYFte0Q5Jhw0A3Fkya9d8hQ-22pRHYZE62pB_wKb9Caw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 7 February 2014 17:05, Ryan Stone <rysto32@gmail.com> wrote:
> On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>> In any case - the reason it's happening above is because there's no
>> actual lock held over the whole lookup/insert path. It's a per-CPU
>> critical enter/exit path, so the only way to guarantee consistency is
>> to use sched_pin() for the entirety of the function.
>
> sched_pin seems like a very heavy hammer for what has to be a very
> rare and mostly harmless race. Why not redo the lookup after you have
> reacquired the lock, and if you don't have to do the insert anymore
> then don't and move on?
You say "rare and harmless race"; I can trick the system to do this:
flowtable for IPv4:
57448 lookups
39543 hits
17905 misses
17820 collisions
382 free checks
82 frees
.. and it gets stuck in a loop of never quite correctly updating/using
the flowtable entries.
So, it is mostly harmless, except exactly when it bites you in the ass.
Yes, we could just reacquire the lock and insert if required, but I
still have to be absolutely sure that the thread isn't preempted and
migrated to another core. Otherwie we'd have teh same issue again.
I'll keep tickling it.
Thanks,
-a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmonFOGf7AyUNgwroOmjJWfQL4RXp2wZSpj-=wnCTKVoo8g>