From owner-freebsd-arch@freebsd.org Fri Jul 24 10:29:21 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A74479A9079 for ; Fri, 24 Jul 2015 10:29:21 +0000 (UTC) (envelope-from meloun@miracle.cz) Received: from mail.miracle.cz (mail.miracle.cz [193.84.128.19]) by mx1.freebsd.org (Postfix) with ESMTP id 69EF015D8 for ; Fri, 24 Jul 2015 10:29:21 +0000 (UTC) (envelope-from meloun@miracle.cz) Received: from [193.84.128.50] (meloun.ad.miracle.cz [193.84.128.50]) by mail.miracle.cz (Postfix) with ESMTPSA id 5ECB6ACABEB for ; Fri, 24 Jul 2015 12:22:33 +0200 (CEST) Message-ID: <55B211FA.3010700@miracle.cz> Date: Fri, 24 Jul 2015 12:22:50 +0200 From: Michal Meloun Organization: Miracle Group User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: Interrupts on all CPUs References: <20150723182449.0ecc7ea9@bender> In-Reply-To: <20150723182449.0ecc7ea9@bender> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.miracle.cz); Fri, 24 Jul 2015 12:22:33 +0200 (CEST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jul 2015 10:29:21 -0000 > We have a problem on ARM where some interrupts need to be unmasked on > all CPUs in an SMP environment. The main place we need this is with the > timer driver as the hardware can send interrupts to the requested CPU. > > Until now we have hacked around this by unmasking the three interrupts > the timer hardware uses when we enable the non-boot CPU. Luckily these > interrupts values have been common across all SoCs, however there is no > requirement for this to be the case. > > To fix this I'm proposing adding a flag to bus_setup_intr to add > support for per-cpu interrupts. The architecture code is then expected > to perform whatever is needed to handle this. > > I have attached a proof of concept patch that adds support to arm64 for > this. When we setup an interrupt with the flag set it will record this > so when the non-boot CPUs are enabled they can unmask the interrupt. It > will then signal to any running CPUs to unmask the given interrupt. > > I would expect this could be extended to other architectures, however I > only know the arm and arm64 code, and am only able to test there. > > Andrew I don’t think that this patch solves the problem. In some case, we must initialize per core hardware before we can enable interrupt for it. This is for example, case of the qcom Krait per core devices. I'm thinking about something like that config_intrhook for registering “secondary_attach()” functions from each appropriate driver. The list can be executed as part of secondary_init(), in same order as registered. This, of course implies existence of specific SGI and PPI variant of bus_setup_interrupt() that have core ID as argument and allows multiple activation. Moreover, this approach removed necessary of direct calls for initialization of secondary cores in gic and timer drivers. What do you think? Michal