From owner-svn-src-all@FreeBSD.ORG Tue Mar 10 13:55:07 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40B95DCE; Tue, 10 Mar 2015 13:55:07 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 171AAE40; Tue, 10 Mar 2015 13:55:07 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 717CAB93A; Tue, 10 Mar 2015 09:55:05 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Subject: Re: svn commit: r279699 - in head/sys: amd64/amd64 i386/i386 Date: Tue, 10 Mar 2015 09:54:43 -0400 Message-ID: <158511036.uSYxIlCjo8@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <20150309174252.GO2379@kib.kiev.ua> References: <201503062034.t26KYSP2063973@svn.freebsd.org> <20150309174252.GO2379@kib.kiev.ua> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 10 Mar 2015 09:55:05 -0400 (EDT) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 13:55:07 -0000 On Monday, March 09, 2015 07:42:52 PM Konstantin Belousov wrote: > On Fri, Mar 06, 2015 at 08:34:28PM +0000, John Baldwin wrote: > > Author: jhb > > Date: Fri Mar 6 20:34:28 2015 > > New Revision: 279699 > > URL: https://svnweb.freebsd.org/changeset/base/279699 > > > > Log: > > Only schedule interrupts on a single hyperthread of a modern Intel CPU > > core > > by default. Previously we used a single hyperthread on Pentium4-era > > cores but used both hyperthreads on more recent CPUs. > > > > MFC after: 2 weeks > > > > Modified: > > head/sys/amd64/amd64/mp_machdep.c > > head/sys/i386/i386/mp_machdep.c > > > > Modified: head/sys/amd64/amd64/mp_machdep.c > > ========================================================================== > > ==== --- head/sys/amd64/amd64/mp_machdep.c Fri Mar 6 16:43:54 > > 2015 (r279698) +++ head/sys/amd64/amd64/mp_machdep.c Fri Mar 6 20:34:28 > > 2015 (r279699) @@ -828,8 +828,8 @@ set_interrupt_apic_ids(void) > > > > continue; > > > > /* Don't let hyperthreads service interrupts. */ > > > > - if (hyperthreading_cpus > 1 && > > - apic_id % hyperthreading_cpus != 0) > > + if (cpu_logical > 1 && > > + apic_id % cpu_logical != 0) > > > > continue; > > > > intr_add_cpu(i); > > > > Modified: head/sys/i386/i386/mp_machdep.c > > ========================================================================== > > ==== --- head/sys/i386/i386/mp_machdep.c Fri Mar 6 16:43:54 > > 2015 (r279698) +++ head/sys/i386/i386/mp_machdep.c Fri Mar 6 20:34:28 > > 2015 (r279699) @@ -842,8 +842,8 @@ set_interrupt_apic_ids(void) > > > > continue; > > > > /* Don't let hyperthreads service interrupts. */ > > > > - if (hyperthreading_cpus > 1 && > > - apic_id % hyperthreading_cpus != 0) > > + if (cpu_logical > 1 && > > + apic_id % cpu_logical != 0) > > > > continue; > > > > intr_add_cpu(i); > > BTW, this sounds somewhat backward from the intention of the HTT/SMT. > The feature was aimed to reduce latency of memory or device registers > reads by allowing several contexts to stuck on the cache line fill > or waiting for the device response to read request. > > Since typical interrupt handler just EOIs the source or does nothing at > all to the source, what is the reasoning behind the change ? It also affects where ithreads are run. SCHED_ULE does a soft binding of ithreads to the CPU that scheduled them (i.e. the CPU that received the interrupt). It also matters for bus_get_cpus(9) as this will affect the set of CPUs that returns so that by default multi queue NICs will only schedule one interrupt + bound ithread (and in some cases bound taskqueue thread) per- core rather than one per-thread. I have thought about having a separate tunable to control whether or not this is in effect if there are any workloads where having interrupts on all threads in a core helps (for all the cases I am aware of, people end up disabling HTT in the BIOS to get this effect instead). Ultimately I'd like to make it possible to manipulate the set of CPUs used for interrupts directly, but that's a larger change. -- John Baldwin