From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 28 18:31:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05E31A8C; Sat, 28 Mar 2015 18:31:50 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AEFA8692; Sat, 28 Mar 2015 18:31:49 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YbvWN-000P9t-Fn; Sat, 28 Mar 2015 21:31:47 +0300 Date: Sat, 28 Mar 2015 21:31:47 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150328183147.GC23643@zxy.spb.ru> References: <20150328112035.GZ23643@zxy.spb.ru> <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2015 18:31:50 -0000 On Sat, Mar 28, 2015 at 11:23:56AM -0700, Adrian Chadd wrote: > Ah, I think that's because the taskqueues in the driver for deferred > handling aren't also being pinned. > > I've talked to John about this - the problem is that all the > taskqueues for all the drivers run under one kernel process. Find out > their threadids and pin them too. > > Eg: > > # procstat -ta | grep em0 > 0 100024 kernel em0 que -1 8 sleep - > 0 100025 kernel em0 txq -1 8 sleep - # procstat -ta | grep ix0 0 100065 kernel ix0 que 6 8 run - 0 100067 kernel ix0 que 7 8 run - 0 100069 kernel ix0 que 8 8 run - 0 100071 kernel ix0 linkq 7 8 sleep - 12 100064 intr irq270: ix0:que 6 8 run - 12 100066 intr irq271: ix0:que 7 8 run - 12 100068 intr irq272: ix0:que 8 8 run - 12 100070 intr irq273: ix0:link 0 8 wait - As you see -- I am already pined all. # cpuset -g -x 270 irq 270 mask: 6 # cpuset -g -t 100065 tid 100065 mask: 6 # cpuset -g -t 100064 tid 100064 mask: 6 > # vmstat -ia | grep em > irq256: em0 68465 1 > # cpuset -g -x 256 > irq 256 mask: 0, 1 > # cpuset -g -t 100024 > tid 100024 mask: 0, 1 > # cpuset -g -t 100025 > tid 100025 mask: 0, 1 > > So you'd have to manually do that - there's no generic interface at > the moment to be able to ask a device driver to re-mask its taskqueue > thread(s) for a given queue and rewire its interrupt(s) for that > queue. > > (That would be a nice smallish project to prototype, btw.)