Date: Mon, 15 Sep 2014 12:04:51 +0300 From: Alexander Motin <mav@FreeBSD.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r271604 - head/sys/kern Message-ID: <5416ABB3.50808@FreeBSD.org> In-Reply-To: <20140915085122.GF2737@kib.kiev.ua> References: <201409142213.s8EMDJdM065051@svn.freebsd.org> <20140915085122.GF2737@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 15.09.2014 11:51, Konstantin Belousov wrote: > On Sun, Sep 14, 2014 at 10:13:19PM +0000, Alexander Motin wrote: >> Author: mav Date: Sun Sep 14 22:13:19 2014 New Revision: 271604 >> URL: http://svnweb.freebsd.org/changeset/base/271604 >> >> Log: Add couple memory barries to serialize tdq_cpu_idle and >> tdq_load accesses. > Serialize what against what ? > > It seems what you do is just ensuring that the write to > tdq_cpu_idle in sched_idletd() become visible faster than it was > before your patch. Memory barrier after the assignment of > dq->tdq_cpu_idle = 1 flushes write buffers, which makes the > tdq_notify() to see the write immediately. Am I right ? Right. My understanding of the problem is that tdq_notify() does not see updated tdq_cpu_idle in time, while cpu_idle() does not see updated tdq_load in time. From my experiments any one of those two barriers appear enough to fix the problem, but I tried to be safe. > If true, this must be commented to explain the stray barriers > appearing in the code. > >> >> This change fixes transient performance drops in some of my >> benchmarks, vanishing as soon as I am trying to collect any stats >> from the scheduler. It looks like reordered access to those >> variables sometimes caused loss of IPI_PREEMPT, that delayed >> thread execution until some later interrupt. >> >> MFC after: 3 days >> >> Modified: head/sys/kern/sched_ule.c >> >> Modified: head/sys/kern/sched_ule.c >> ============================================================================== >> >> - --- head/sys/kern/sched_ule.c Sun Sep 14 22:10:35 2014 (r271603) >> +++ head/sys/kern/sched_ule.c Sun Sep 14 22:13:19 2014 (r271604) >> @@ -1037,6 +1037,7 @@ tdq_notify(struct tdq *tdq, struct threa >> ctd = pcpu_find(cpu)->pc_curthread; if (!sched_shouldpreempt(pri, >> ctd->td_priority, 1)) return; + mb(); if (TD_IS_IDLETHREAD(ctd)) >> { /* * If the MD code has an idle wakeup routine try that before >> @@ -2640,6 +2641,7 @@ sched_idletd(void *dummy) >> >> /* Run main MD idle handler. */ tdq->tdq_cpu_idle = 1; + mb(); >> cpu_idle(switchcnt * 4 > sched_idlespinthresh); tdq->tdq_cpu_idle >> = 0; >> > I suspect that what you are trying to do could be achieved by > using the FreeBSD API, instead of compat shims, in the following > way. I was really trying to do it native way originally, but haven't found how. If I understand semantics right, I would need atomic_load_rel() and atomic_store_acq(), but there is no such ones. Second one you successfully replaced by atomic_add_rel(). But I am not sure about the first -- on x86 your idea should work since uses LOCK CMPXCHG, but on other platforms barrier is provided after the load, not before. > diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index > 0a63c01..1ffac22 100644 --- a/sys/kern/sched_ule.c +++ > b/sys/kern/sched_ule.c @@ -1042,7 +1042,8 @@ tdq_notify(struct tdq > *tdq, struct thread *td) * If the MD code has an idle wakeup > routine try that before * falling back to IPI. */ - if > (!tdq->tdq_cpu_idle || cpu_idle_wakeup(cpu)) + if > (!atomic_load_acq_int(&tdq->tdq_cpu_idle) || + > cpu_idle_wakeup(cpu)) return; } tdq->tdq_ipipending = 1; @@ -2639,7 > +2640,7 @@ sched_idletd(void *dummy) continue; > > /* Run main MD idle handler. */ - tdq->tdq_cpu_idle = 1; + > atomic_add_rel_int(&tdq->tdq_cpu_idle, 1); cpu_idle(switchcnt * 4 > > sched_idlespinthresh); tdq->tdq_cpu_idle = 0; > > - -- Alexander Motin -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQF8BAEBCgBmBQJUFquzXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4 MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/3LUIAMizmXrjgfoQBLhD25fKhP0w XulDxS4L0xGE17xDW0MTL7BJ8A/xz/nolY8SQFbHe5/7jlueqqG2PoSokrQMHpYw IZ0aYy5rQw+ZgR40zRFbTY1iNdZ7K9QsB6qweWQztSnBvLvlQ0Nx+/YgeDNuHNe1 jWfJjmJLp3gz+/VQOUnHIBdslPS6YplOc5vQmc4NqFGhMplnhzWARJbLqxmYu7JK HmeIP1uVxDEiG82TU+x4n21HehUi2POXAnygLrv0B4xqN4n8pYNyEtknesJiQO0t lyrcXE2vWsDrLMrqCsaZc6pZEXqk8PJ4Og/v6S/XztV/FtrI5Ilqjz/i6e45b8Y= =tfah -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5416ABB3.50808>