From owner-svn-src-all@FreeBSD.ORG Mon Sep 15 09:05:02 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C6147D4; Mon, 15 Sep 2014 09:05:02 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB943261; Mon, 15 Sep 2014 09:05:01 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id c11so4022041lbj.37 for ; Mon, 15 Sep 2014 02:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=gkEzwT+1EI88nn609ONKH3PKrud5hTDZluh6NOwOrIc=; b=nIEAP3zcEWhiV6WrSv2V7md34JakYmdVzjR3i6O+Tg4wIYQ+QvV+0LSBNBKq+PYfu7 4Mi27Buerfd/iwbj1fm8bWDDe9V3eOcWGP86FXQZpjgTKXmQFuwThvC7yYUTx4pr1nyl SnUWNoHnTtegyWnFwKQee16JaG+Q+NnAl3bjF711rB9IDOdqW84JjgWYS2vQTc4SiJxw YpECW2CnLIZ0WbOtta+cvdNQARbJAVwr61zeAnc5r7MW0MH5HY50/V5oAbJe5i8vg9CF 58K5OkjS2EKCuGbqw5Zfb1F0Ju9l0yaLpkpCqxaTidHTznz7e7GmHxleIJ6McerGddLQ nkmg== X-Received: by 10.152.18.130 with SMTP id w2mr1590238lad.89.1410771899309; Mon, 15 Sep 2014 02:04:59 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id pr7sm4029047lbc.18.2014.09.15.02.04.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Sep 2014 02:04:58 -0700 (PDT) Sender: Alexander Motin Message-ID: <5416ABB3.50808@FreeBSD.org> Date: Mon, 15 Sep 2014 12:04:51 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: svn commit: r271604 - head/sys/kern References: <201409142213.s8EMDJdM065051@svn.freebsd.org> <20140915085122.GF2737@kib.kiev.ua> In-Reply-To: <20140915085122.GF2737@kib.kiev.ua> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Sep 2014 09:05:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 15.09.2014 11:51, Konstantin Belousov wrote: > On Sun, Sep 14, 2014 at 10:13:19PM +0000, Alexander Motin wrote: >> Author: mav Date: Sun Sep 14 22:13:19 2014 New Revision: 271604 >> URL: http://svnweb.freebsd.org/changeset/base/271604 >> >> Log: Add couple memory barries to serialize tdq_cpu_idle and >> tdq_load accesses. > Serialize what against what ? > > It seems what you do is just ensuring that the write to > tdq_cpu_idle in sched_idletd() become visible faster than it was > before your patch. Memory barrier after the assignment of > dq->tdq_cpu_idle = 1 flushes write buffers, which makes the > tdq_notify() to see the write immediately. Am I right ? Right. My understanding of the problem is that tdq_notify() does not see updated tdq_cpu_idle in time, while cpu_idle() does not see updated tdq_load in time. From my experiments any one of those two barriers appear enough to fix the problem, but I tried to be safe. > If true, this must be commented to explain the stray barriers > appearing in the code. > >> >> This change fixes transient performance drops in some of my >> benchmarks, vanishing as soon as I am trying to collect any stats >> from the scheduler. It looks like reordered access to those >> variables sometimes caused loss of IPI_PREEMPT, that delayed >> thread execution until some later interrupt. >> >> MFC after: 3 days >> >> Modified: head/sys/kern/sched_ule.c >> >> Modified: head/sys/kern/sched_ule.c >> ============================================================================== >> >> - --- head/sys/kern/sched_ule.c Sun Sep 14 22:10:35 2014 (r271603) >> +++ head/sys/kern/sched_ule.c Sun Sep 14 22:13:19 2014 (r271604) >> @@ -1037,6 +1037,7 @@ tdq_notify(struct tdq *tdq, struct threa >> ctd = pcpu_find(cpu)->pc_curthread; if (!sched_shouldpreempt(pri, >> ctd->td_priority, 1)) return; + mb(); if (TD_IS_IDLETHREAD(ctd)) >> { /* * If the MD code has an idle wakeup routine try that before >> @@ -2640,6 +2641,7 @@ sched_idletd(void *dummy) >> >> /* Run main MD idle handler. */ tdq->tdq_cpu_idle = 1; + mb(); >> cpu_idle(switchcnt * 4 > sched_idlespinthresh); tdq->tdq_cpu_idle >> = 0; >> > I suspect that what you are trying to do could be achieved by > using the FreeBSD API, instead of compat shims, in the following > way. I was really trying to do it native way originally, but haven't found how. If I understand semantics right, I would need atomic_load_rel() and atomic_store_acq(), but there is no such ones. Second one you successfully replaced by atomic_add_rel(). But I am not sure about the first -- on x86 your idea should work since uses LOCK CMPXCHG, but on other platforms barrier is provided after the load, not before. > diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index > 0a63c01..1ffac22 100644 --- a/sys/kern/sched_ule.c +++ > b/sys/kern/sched_ule.c @@ -1042,7 +1042,8 @@ tdq_notify(struct tdq > *tdq, struct thread *td) * If the MD code has an idle wakeup > routine try that before * falling back to IPI. */ - if > (!tdq->tdq_cpu_idle || cpu_idle_wakeup(cpu)) + if > (!atomic_load_acq_int(&tdq->tdq_cpu_idle) || + > cpu_idle_wakeup(cpu)) return; } tdq->tdq_ipipending = 1; @@ -2639,7 > +2640,7 @@ sched_idletd(void *dummy) continue; > > /* Run main MD idle handler. */ - tdq->tdq_cpu_idle = 1; + > atomic_add_rel_int(&tdq->tdq_cpu_idle, 1); cpu_idle(switchcnt * 4 > > sched_idlespinthresh); tdq->tdq_cpu_idle = 0; > > - -- Alexander Motin -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQF8BAEBCgBmBQJUFquzXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFOThDRjNDNEU2OUNDM0NEMEU1NzlENTU4 MzE4QzM5NTVCQUIyMjdGAAoJEIMYw5VbqyJ/3LUIAMizmXrjgfoQBLhD25fKhP0w XulDxS4L0xGE17xDW0MTL7BJ8A/xz/nolY8SQFbHe5/7jlueqqG2PoSokrQMHpYw IZ0aYy5rQw+ZgR40zRFbTY1iNdZ7K9QsB6qweWQztSnBvLvlQ0Nx+/YgeDNuHNe1 jWfJjmJLp3gz+/VQOUnHIBdslPS6YplOc5vQmc4NqFGhMplnhzWARJbLqxmYu7JK HmeIP1uVxDEiG82TU+x4n21HehUi2POXAnygLrv0B4xqN4n8pYNyEtknesJiQO0t lyrcXE2vWsDrLMrqCsaZc6pZEXqk8PJ4Og/v6S/XztV/FtrI5Ilqjz/i6e45b8Y= =tfah -----END PGP SIGNATURE-----