From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 31 23:14:06 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF844A5A for ; Tue, 31 Mar 2015 23:14:05 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE64C8D8 for ; Tue, 31 Mar 2015 23:14:05 +0000 (UTC) Received: by iedm5 with SMTP id m5so29631043ied.3 for ; Tue, 31 Mar 2015 16:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hc1tcvzMxkrs/3IywrCkP0FmiJ4QKui14w+46T0W3uM=; b=PZoCiDUNfJAUInqa+eQWRwKs9osihtWYzTLguUVvqcQJ1TK30TsBhNzoAsd6mUCIPi 2blqHoOcIs3ytQG/zVlFzDhbenxwUN4Tf9sleZwUGuPBk/hMY1hFriY7Ln7BNuhtkdZT 8CDWbPAfR5y24eppP5cQkwJpdx3HUo4htoTm0DGSWW1gnIXxHin/Bi1acToNG5zjBA3V AVwqFi6CNFIEJdu1+2yd9mgqymV8GQTVYNUh4wo9PxB3aQ/lv0h3yQAjdeIrh+tuKERA qa6PC6lRzJcRTvIVfYqzqt5bnmyFN1GzgoMtKf0rzM+hEj0G7piGHP1PdjbkMFoYN9Rb AlGA== MIME-Version: 1.0 X-Received: by 10.107.39.72 with SMTP id n69mr44465639ion.8.1427843644983; Tue, 31 Mar 2015 16:14:04 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Tue, 31 Mar 2015 16:14:04 -0700 (PDT) In-Reply-To: <20150331225947.GC23643@zxy.spb.ru> References: <20150328112035.GZ23643@zxy.spb.ru> <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> Date: Tue, 31 Mar 2015 16:14:04 -0700 X-Google-Sender-Auth: CVgL8MyGBQP3YyD7U6qQL0X8ktQ Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 23:14:06 -0000 You also have to do the taskqueue threads. (I mean, pmc could also be broken...) -a On 31 March 2015 at 15:59, Slawa Olhovchenkov wrote: > On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote: > >> That's done deferred by the bus interrupt wiring. That's something >> John's been looking into as part of the general NUMA work (and I'm >> trying to debug right now, on dual-socket boxes with ixgbe. :-) >> >> Look at bus_bind_intr() and the twisty path to intr_event_bind(), then >> x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at >> boot, versus what happens via calls to pic_assign_cpu to setup the >> wiring. > > I am do simple, ugle hack ixgbe driver for let start cpu binding. > I am still see ixgbe in pmc output. > > What may be wrong? > What may be miss? > > ===== > static int ixgbe_start_cpu = 0; > TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu); > SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0, > "Start CPU for next IRQ binding"); > > [...] > if (adapter->num_queues > 1) > bus_bind_intr(dev, que->res, i+ixgbe_start_cpu); > > #ifndef IXGBE_LEGACY_TX > TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr); > #endif > TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que); > que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT, > taskqueue_thread_enqueue, &que->tq); > taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", > device_get_nameunit(adapter->dev)); > } > ixgbe_start_cpu += adapter->num_queues; >