From owner-svn-src-stable-10@FreeBSD.ORG Wed Sep 17 14:06:21 2014 Return-Path: Delivered-To: svn-src-stable-10@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC2D2314; Wed, 17 Sep 2014 14:06:21 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6D4C1AC; Wed, 17 Sep 2014 14:06:21 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s8HE6LTH071462; Wed, 17 Sep 2014 14:06:21 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s8HE6LNu071461; Wed, 17 Sep 2014 14:06:21 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201409171406.s8HE6LNu071461@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 17 Sep 2014 14:06:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r271707 - stable/10/sys/kern X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-10@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for only the 10-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 14:06:21 -0000 Author: mav Date: Wed Sep 17 14:06:21 2014 New Revision: 271707 URL: http://svnweb.freebsd.org/changeset/base/271707 Log: MFC r271604, r271616: Add couple memory barriers to order tdq_cpu_idle and tdq_load accesses. This change fixes transient performance drops in some of my benchmarks, vanishing as soon as I am trying to collect any stats from the scheduler. It looks like reordered access to those variables sometimes caused loss of IPI_PREEMPT, that delayed thread execution until some later interrupt. Approved by: re (marius) Modified: stable/10/sys/kern/sched_ule.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/kern/sched_ule.c ============================================================================== --- stable/10/sys/kern/sched_ule.c Wed Sep 17 08:47:58 2014 (r271706) +++ stable/10/sys/kern/sched_ule.c Wed Sep 17 14:06:21 2014 (r271707) @@ -1037,6 +1037,14 @@ tdq_notify(struct tdq *tdq, struct threa ctd = pcpu_find(cpu)->pc_curthread; if (!sched_shouldpreempt(pri, ctd->td_priority, 1)) return; + + /* + * Make sure that tdq_load updated before calling this function + * is globally visible before we read tdq_cpu_idle. Idle thread + * accesses both of them without locks, and the order is important. + */ + mb(); + if (TD_IS_IDLETHREAD(ctd)) { /* * If the MD code has an idle wakeup routine try that before @@ -2645,6 +2653,12 @@ sched_idletd(void *dummy) /* Run main MD idle handler. */ tdq->tdq_cpu_idle = 1; + /* + * Make sure that tdq_cpu_idle update is globally visible + * before cpu_idle() read tdq_load. The order is important + * to avoid race with tdq_notify. + */ + mb(); cpu_idle(switchcnt * 4 > sched_idlespinthresh); tdq->tdq_cpu_idle = 0;