From owner-freebsd-arch@freebsd.org Tue May 30 14:46:32 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17436B7E9CB for ; Tue, 30 May 2017 14:46:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A00EC64C40 for ; Tue, 30 May 2017 14:46:31 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wm0-x22e.google.com with SMTP id d127so100013423wmf.0 for ; Tue, 30 May 2017 07:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=TeVkayfMb/qeYu4rLId9n20vvn72FhSFdvDcLwlXWzs=; b=iudwF1Jes6EH9hCMdVrq13DHD0RdQSSOgHZRBwz8yYS27CRPgsgiubU469kufje44b N/Kq/FfLvtH6Q8wdNfcBjWqdA9dtQfHCDXWRnCcidMrqC03j2efpduEpRbi3i4QMRGpq 27I9traUal4e9YNJxwyFgnvnPLTexBojS1y7z3ykqx0biAI6ug+agwo6QaRkWFROSbTP 91UC87qWMVFANh1+jhhAxGebAZYb5U+YLJq08EznqZ5Qu3qufqEKosX7DrWoN02Jnxxh 06QTgIje5SF1hq0xjqPdRdz5oTStM0USsIuQd3Rbdkbs5QBPrjND/RsqKKDIkBhgy4Q7 LW9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=TeVkayfMb/qeYu4rLId9n20vvn72FhSFdvDcLwlXWzs=; b=qkMzsjZGkKgcPEB8u4DWiUp0nNmZcRX1nY1BUeNrFDwwTBR5JgmzDIpsbwrMt9HFIe qxWaWr00SzVy5+0k/TwyjkGRGgRoNH6zPi9hefPZwQ+EURwLQftmhni80nwAkybT7Kc1 QD9ijqhNJAkDsc+XFRKcWgTD1lPNuuGBnn9yK4ms4N9Qk4sGdwKoOyrqpbCUy9+b+1Ak Y0NtOnmNJKcFKdzJcelNCoOPo0RRNgUenrF4l8D+JGOONLOlyjDtrk5sjioyqbcv9HBt 4+pLEfZszXPGtZQyv2kkcxz1ppq+ILO17E60X5NYqfx07eIYphYnAGV54jCFJIb+9zie bcTA== X-Gm-Message-State: AODbwcAZYpByO7ityBxi4pKIQgVJKxfenKKG6IcizH3lbKx/VhVjdmXH 4NgvDvneewvgMjmyzwY6PQJNucToFdkw X-Received: by 10.28.50.65 with SMTP id y62mr2140495wmy.5.1496155590104; Tue, 30 May 2017 07:46:30 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.28.193.134 with HTTP; Tue, 30 May 2017 07:46:29 -0700 (PDT) In-Reply-To: <816581118.55670987.1496141816904.JavaMail.zimbra@stormshield.eu> References: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu> <816581118.55670987.1496141816904.JavaMail.zimbra@stormshield.eu> From: Adrian Chadd Date: Tue, 30 May 2017 07:46:29 -0700 X-Google-Sender-Auth: a9CxfDR6fkNdmQK9VgDVY63UZvs Message-ID: Subject: Re: numa and taskqueues To: Emeric POUPON Cc: freebsd-arch Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 May 2017 14:46:32 -0000 On 30 May 2017 at 03:56, Emeric POUPON wrote: > Hi, > >> Anyway - I think it'd be nice to have domain aware and pcpu aware >> taskqueues so we can eventually migrate to a taskqueue group model of >> "one top level things for net processing" for devices to share, etc, >> etc. But for the short term just prototype it with some thin API in >> crypto that wraps the taskqueue / kproc work so it gets done, then >> push that work out for review/evaluation. if it does indeed work the >> way you intend, we can try to use it as a template for a higher level, >> shared taskqueue thing. > > It looks like it is somewhat mandatory to modify the taskqueue API to pin threads to the > correct CPUs. The logic to define which CPU need to be set is another story that indeed can first > be implemented in crypto(9). > > By the way: > 1/ do you have some pointers on domain enumeration and other numa related code? Sorry, I'm a bit too busy with other things to dive in right now :( > 2/ about https://reviews.freebsd.org/D10680, I think it would be great to have this commited as a first step. > Since it seems to be stuck, maybe I can add more people on this. Any suggestion? Well, what's with the ~ 8% performance decrease? Do you know what's going on? For a "we're parallelising IPSEC operations", seeing it get slower with more flows is a bit concerning. Thanks, -adrian