From owner-freebsd-arch@freebsd.org  Tue May 30 11:12:43 2017
Return-Path: <owner-freebsd-arch@freebsd.org>
Delivered-To: freebsd-arch@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BBABB79C16
 for <freebsd-arch@mailman.ysv.freebsd.org>;
 Tue, 30 May 2017 11:12:43 +0000 (UTC)
 (envelope-from emeric.poupon@stormshield.eu)
Received: from work.stormshield.eu (gwlille.netasq.com [91.212.116.1])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id D4F2B81783
 for <freebsd-arch@freebsd.org>; Tue, 30 May 2017 11:12:41 +0000 (UTC)
 (envelope-from emeric.poupon@stormshield.eu)
Received: from work.stormshield.eu (localhost [127.0.0.1])
 by work.stormshield.eu (Postfix) with ESMTPS id 706893760B1E;
 Tue, 30 May 2017 12:56:57 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by work.stormshield.eu (Postfix) with ESMTP id 630133760A0B;
 Tue, 30 May 2017 12:56:57 +0200 (CEST)
Received: from work.stormshield.eu ([127.0.0.1])
 by localhost (work.stormshield.eu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id WuoVfhcIi03v; Tue, 30 May 2017 12:56:57 +0200 (CEST)
Received: from work.stormshield.eu (localhost [127.0.0.1])
 by work.stormshield.eu (Postfix) with ESMTP id 4E680376068D;
 Tue, 30 May 2017 12:56:57 +0200 (CEST)
Date: Tue, 30 May 2017 12:56:56 +0200 (CEST)
From: Emeric POUPON <emeric.poupon@stormshield.eu>
To: Adrian Chadd <adrian.chadd@gmail.com>
Cc: freebsd-arch <freebsd-arch@freebsd.org>
Message-ID: <816581118.55670987.1496141816904.JavaMail.zimbra@stormshield.eu>
In-Reply-To: <CAJ-Vmo=6bpo1Yu6XosN3BiYOakjeXS8J7wenfubzkWz2SxXR1g@mail.gmail.com>
References: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu>
 <CAJ-Vmo=6bpo1Yu6XosN3BiYOakjeXS8J7wenfubzkWz2SxXR1g@mail.gmail.com>
Subject: Re: numa and taskqueues
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Thread-Topic: numa and taskqueues
Thread-Index: 1prrS86VmRWPDnLN63KY5Yag7cknDQ==
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch/>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 30 May 2017 11:12:43 -0000

Hi,

> Anyway - I think it'd be nice to have domain aware and pcpu aware
> taskqueues so we can eventually migrate to a taskqueue group model of
> "one top level things for net processing" for devices to share, etc,
> etc. But for the short term just prototype it with some thin API in
> crypto that wraps the taskqueue / kproc work so it gets done, then
> push that work out for review/evaluation. if it does indeed work the
> way you intend, we can try to use it as a template for a higher level,
> shared taskqueue thing.

It looks like it is somewhat mandatory to modify the taskqueue API to pin threads to the
correct CPUs. The logic to define which CPU need to be set is another story that indeed can first
be implemented in crypto(9).

By the way:
1/ do you have some pointers on domain enumeration and other numa related code?
2/ about https://reviews.freebsd.org/D10680, I think it would be great to have this commited as a first step. 
Since it seems to be stuck, maybe I can add more people on this. Any suggestion?

Emeric