From owner-freebsd-arch@freebsd.org Thu May 18 00:46:17 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2A1CD7045F; Thu, 18 May 2017 00:46:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C2A61B0D; Thu, 18 May 2017 00:46:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id v15so34635735wmv.1; Wed, 17 May 2017 17:46:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=gfVq3TojoDqkMiOWcRtpXZVwS62ksm8yNdKeS0rwaes=; b=rU/2EDXRzi1E6qDY4Yl4Vf2i8qOcyuYTMMhvBXXjfCN6+ujf9cAUEVs79KmncZV7lZ ywHhh4UZSj6mH+9RSSra9ojdjgwGZpRrGl/ZmNjJyQbbGFvlXY5J/MJ8tvJBdL+9NDSm slm5Ub7e0TAXPXfMhWKChC5BuOakqN2RUWjexUlbpIg0r0NP4smR4Szs+zQ33FlTwbsf q1GGCGKgZCvweSMzJBEQKdVOMdkUMJabClY4ajv6Ii7Blu1IiRBGu4trzSViTKtlrBN/ Ef3VXGWGvxGExO8Y625FuNceZkWfcm9ZH5mcmsqOpmRu5HkYPTUDYf5rWAIvAHZoa/Zh ft1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=gfVq3TojoDqkMiOWcRtpXZVwS62ksm8yNdKeS0rwaes=; b=bw6KjIOUK+cZPjI2kLttRJOHp+ZXM5lETbbExed5vZy9U3eEXz3EHMisM1K0TAzQHT oe7/ejClofLGVCs+qbMnavqac8vQkm91MB4pl68z3JzOdfrdp7UBtKu7UmxsuUiKry9w ovZYRIy4/jUBznj/3CveaEDzEY80vYAvwy0hPpDT3/wTwvaYomjsR7d9J4GTJUEkMsAS 7MAwbDfrQnKHNLED7S3Aa3PaOp4N5UFcrN9pPBMeDpuTD+mfHB1qTt0kmm1VKZOlF3za FF4lDud+AZZpxHxbVZO0eV1+6nXBnt9siaJt5HOU6aTbckUxiIAjy8KfAV0NBSJd9c6n HH5A== X-Gm-Message-State: AODbwcA1/eKTD/7X8fg/P7yhGtcak3nhPCHGs/+IsST7sOJlVBloC41C Eiv6mMijxER+hbT0ewazE3QPdbNM8W5A X-Received: by 10.28.214.211 with SMTP id n202mr845740wmg.105.1495068374783; Wed, 17 May 2017 17:46:14 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.28.193.134 with HTTP; Wed, 17 May 2017 17:46:14 -0700 (PDT) From: Adrian Chadd Date: Wed, 17 May 2017 17:46:14 -0700 X-Google-Sender-Auth: kc8mwDQHwQq0pTukRJ5tFF9oqWA Message-ID: Subject: make concurrency kit a module To: freebsd-current , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 May 2017 00:46:17 -0000 hi, this is a quick change that makes concurrency_kit a module. Right now the only thing using it is linuxkpi so it's all dead code on non-linuxkpi platforms. -adrian From owner-freebsd-arch@freebsd.org Thu May 18 01:04:12 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C27D5D70BEC; Thu, 18 May 2017 01:04:12 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wr0-x232.google.com (mail-wr0-x232.google.com [IPv6:2a00:1450:400c:c0c::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1E61930; Thu, 18 May 2017 01:04:12 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wr0-x232.google.com with SMTP id l9so22399181wre.1; Wed, 17 May 2017 18:04:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=zbi9Xv0fbzE66OaYG9LoiZZb4+qG9aPTQSz4IB1LLRI=; b=fJqdWfToEdB3+rv7+H7g8EuhSfgYUE/YIefk+7kOEx+5dLzOdj+U3Mu/Aw0nER5ur0 8T6IA+Rbo0GLHtCK1HKeJYKUPvV6gzaqBQ1DK/c/XA8G3u7XgaNlDfkbudMXaT0KvYmD H97KNEZ6QSAVK1atgP0LDAo3wj2KPxyoqXgJWA/SSBz4eLdcNfLBi92RIxGWVdt9onIZ 93KxRSr9XJI+agT1OFlngAdX6IDlyOpP5SjRfbRjQJ05WozapyMV6gIcDHYkwC/cfwdh vCOxPwcEp7BWnfBT56suN8qBPTj+DmMwINJZuH5KMlHDujQywF7iYJKvxFTXFdxAaYhc rqkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=zbi9Xv0fbzE66OaYG9LoiZZb4+qG9aPTQSz4IB1LLRI=; b=XihynrA3TfHQ1e+KEyIBp63mA478mLVEts7tE7Ax95evpLwx2situBMLlyn4c7EXYs LbJrYvK7pCQ6+A6JS8vZ3XDX9bB+E4OWa9YU6pyqRZa+uCvNgMJE/vOJr+V5+ekOd3EF wGjHXCj4HXwAqTGymXElqrJEKE5KmB/V4Jv1+2gbqahc5Cydh9nuJx+cHXAPgdDTEm3Q nJLtRC+lol89bmSbDB54Mxokys/nK1SU2IQ9upV4hZZNAKR2QK1770MiiIKNh62ntXub nwLL7hYoViVQqdGmGr72g3vEoxbY3rI/d1HdlhmOx3S4DuOxbZdtYpKVFtKeevn6TK1I B0rg== X-Gm-Message-State: AODbwcB3YPF03wYbH8l8sM5lN9uUT3bygh09anai1FunMtPYPRZkvYxh xkvw60XTqfhxLP7oX40yA18+UwsJKXmIWG4= X-Received: by 10.223.152.6 with SMTP id v6mr756661wrb.60.1495069450476; Wed, 17 May 2017 18:04:10 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.28.193.134 with HTTP; Wed, 17 May 2017 18:04:09 -0700 (PDT) In-Reply-To: References: From: Adrian Chadd Date: Wed, 17 May 2017 18:04:09 -0700 X-Google-Sender-Auth: vecR6FgiaCtiptN4C-DsGVWc31w Message-ID: Subject: Re: make concurrency kit a module To: freebsd-current , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 May 2017 01:04:12 -0000 https://reviews.freebsd.org/D10778 -a On 17 May 2017 at 17:46, Adrian Chadd wrote: > hi, > > this is a quick change that makes concurrency_kit a module. Right now > the only thing using it is linuxkpi so it's all dead code on > non-linuxkpi platforms. > > > > -adrian From owner-freebsd-arch@freebsd.org Thu May 18 05:53:53 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3E32D72F5A; Thu, 18 May 2017 05:53:53 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9446C153B; Thu, 18 May 2017 05:53:53 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: by freefall.freebsd.org (Postfix, from userid 1235) id BEB001E4A; Thu, 18 May 2017 05:53:52 +0000 (UTC) Date: Thu, 18 May 2017 07:53:52 +0200 From: Baptiste Daroussin To: Adrian Chadd Cc: freebsd-current , "freebsd-arch@freebsd.org" Subject: Re: make concurrency kit a module Message-ID: <20170518055352.bflapm6mmfhgl4y4@ivaldir.net> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="uymr6tqxazdnlunm" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20170428 (1.8.2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 May 2017 05:53:53 -0000 --uymr6tqxazdnlunm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 17, 2017 at 06:04:09PM -0700, Adrian Chadd wrote: > https://reviews.freebsd.org/D10778 >=20 Except there are plans to use it elsewhere. Many areas may be improved usin= g it. Having it as a module would mean some devs might refrain from using it beca= use there is no waranty for it to be there Areas like VFS and network stack could have a good benefice from using it. Out of curiousity what size is saved? Bapt --uymr6tqxazdnlunm Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEgOTj3suS2urGXVU3Y4mL3PG3PloFAlkdNuUACgkQY4mL3PG3 PlrhBw//ULYy43sP+b/Tq+1cJ20UzEc0+V1B+CBMaLkjftS7tUy9kN7d6egzCAYC jSC3Yr0CiMkH2QwlVPYcSqWirxIeBp5ceGBFDOkAOHE0litUlusdYmm4id3LkiZO VNSWGFMEUMzdONKl6DmLfUds3HUtTIwQhC6qWtapq7yBw6a8jCrhcI2qpMoteaO6 0qWBDRMNTZTNL0I5QzgyxBZYUj+jlj1lP+z4BuQvL/oOIyjQpp/D7udtXALD69te vRN4D7ETycUWfZvzdYUUmxSMU8DzCyS2QHtbGTJPo1ecqlXM3T5tcSqWWUYNnB5u j1w/ymn53focH3sx1N1MOKEGAVqw8UTBjZWqx/D2w/2znyYGTB5nYwpv5VuY0FGs eYYpa6mpPP0etLXhsKlte388VoGxA6ilF/aRE8O/6sBsUwwO7pHhDvTPiatfCIwV nEG7UWTYt81i0UEyTM8sVewF/IZ0+y1OvfDDycJw4sTUi4oiZZIzO5zdraIK5wGX rC1LT1DqLHqzvik9GATFZHKwuQCuxvREjpjGIGh/Pfwr+4wXcAOd2mZXkjJoo5/g fWXA7pjfWzGAd3rhHrKcPgRdBRM3VFZEjTrDyyG+oDOljQY6QrI53Z1FGU2EZZ7F 3ZA/4RRqtSEy481ALeXUdiPrapZIoy5RpaB8ax068a32lWmYRW8= =aHG9 -----END PGP SIGNATURE----- --uymr6tqxazdnlunm-- From owner-freebsd-arch@freebsd.org Thu May 18 06:37:26 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25EF6D72F78 for ; Thu, 18 May 2017 06:37:26 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1F041E03 for ; Thu, 18 May 2017 06:37:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x234.google.com with SMTP id f102so22716198ioi.2 for ; Wed, 17 May 2017 23:37:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=AKk4Tl1GkTLbvhAADMKYGgQPzA+4YXUx+Q/oqYkhew8=; b=eecAOJ4s1VyAnxiOBMpFArH8rFrseupBNJgIjUNk5u3HQXDR/AbYnBFQRrxNvVXnUS +30bDAMVqWE7SDnon91Xw4UR+CvTtHd4LdQo+EedJPlprRmx+dbr9w+UO4+CVvRvqf6o tqjRQf1wJcVICSEHwiJrxdhWwFa4pqxPXkWbvF56VmHLOWL/lpg088tvWoGOJifBFQvV ronTqFbEpdy4ITT4pR5MTa6JZCYQWxPhqbNoo2098l2AFGxPSmxt+/mAfOo4PRkvdUxA yCR0eF1FVm8dM5Rpqz4mdiWnqIJl/EnmtR/Ob8SUaEJqkeZoEJBjlazYxYAUlmkCk1yl zeZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=AKk4Tl1GkTLbvhAADMKYGgQPzA+4YXUx+Q/oqYkhew8=; b=APVWCyHCwK6RD/L2U0KOxqTbewwOBmHaqSQ6C10+s7CLi6g7usK+fLxitTx70IQ5uX oegxSdNUvOOCM5eJYssEc+S2N7HgMnANsATtDuIs+pmgfCY06W1abZ+6MuWQD0E/ZGhm RPQuL7DSYqLjTMiburViuQAYhzLPZC4SHSNU2H9+v5+i/+EsRyADlRXkXIHXJHwGnOat G5q8LN4YugOQnFbiOgz7RDMF0PqzRBOZpFrA7Kd6Z3D8c4hJ6RlW2BhecM12P/vK1Z0B dtcfZdktIvM2hoG17K/qTI7MjqmrnvdamgHs7yvdLJzcDI4begmAiV85JTnkobcv+mdA uCHQ== X-Gm-Message-State: AODbwcDe8DP4O8SKLBp7IqFVpauNeQx9KXDqjA7F+7ybi/QOsca4/vSh pFllXMcdLExobvWMUl2DyrEFDBUVGjPa X-Received: by 10.107.188.132 with SMTP id m126mr3190458iof.148.1495089445063; Wed, 17 May 2017 23:37:25 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.126.6 with HTTP; Wed, 17 May 2017 23:37:24 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:cdd4:9b25:be09:9f7d] In-Reply-To: <20170518055352.bflapm6mmfhgl4y4@ivaldir.net> References: <20170518055352.bflapm6mmfhgl4y4@ivaldir.net> From: Warner Losh Date: Thu, 18 May 2017 00:37:24 -0600 X-Google-Sender-Auth: ayzyUdRe3l1S8tzHXH9yHZce0ME Message-ID: Subject: Re: make concurrency kit a module To: Baptiste Daroussin Cc: Adrian Chadd , freebsd-current , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 May 2017 06:37:26 -0000 On Wed, May 17, 2017 at 11:53 PM, Baptiste Daroussin wrote: > On Wed, May 17, 2017 at 06:04:09PM -0700, Adrian Chadd wrote: >> https://reviews.freebsd.org/D10778 >> > > Except there are plans to use it elsewhere. Many areas may be improved using it. > > Having it as a module would mean some devs might refrain from using it because > there is no waranty for it to be there > > Areas like VFS and network stack could have a good benefice from using it. > > Out of curiousity what size is saved? I'd planned on using it newbus to solve the lifetime issues we have with device_t's.... Warner From owner-freebsd-arch@freebsd.org Fri May 19 07:21:14 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9F8AD73658 for ; Fri, 19 May 2017 07:21:14 +0000 (UTC) (envelope-from emeric.poupon@stormshield.eu) Received: from work.stormshield.eu (gwlille.netasq.com [91.212.116.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9433D1C02 for ; Fri, 19 May 2017 07:21:13 +0000 (UTC) (envelope-from emeric.poupon@stormshield.eu) Received: from work.stormshield.eu (localhost [127.0.0.1]) by work.stormshield.eu (Postfix) with ESMTPS id 9BACA37610B3 for ; Fri, 19 May 2017 09:13:51 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by work.stormshield.eu (Postfix) with ESMTP id 8E43C3761387 for ; Fri, 19 May 2017 09:13:51 +0200 (CEST) Received: from work.stormshield.eu ([127.0.0.1]) by localhost (work.stormshield.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yZihUApLJy7e for ; Fri, 19 May 2017 09:13:51 +0200 (CEST) Received: from work.stormshield.eu (localhost [127.0.0.1]) by work.stormshield.eu (Postfix) with ESMTP id 7B91A37610B3 for ; Fri, 19 May 2017 09:13:51 +0200 (CEST) Date: Fri, 19 May 2017 09:13:51 +0200 (CEST) From: Emeric POUPON To: freebsd-arch Message-ID: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu> Subject: numa and taskqueues MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Thread-Topic: numa and taskqueues Thread-Index: OmwNiBQRFi7BTXWKFdrri3vfOJZN5w== X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 May 2017 07:21:14 -0000 Hello, I have made a review to boost ipsec performance when very few flows are involved: https://reviews.freebsd.org/D10680 (reviews would be appreciated btw!) The idea is to dispatch the crypto jobs using a taskqueue (with nb threads = nbcpus), details are in the review. However, this does not scale well on multi socket architectures (ex: 2*6 cores), a lot of time is wasted in the locks. For testing purposes, I created as many taskqueues as domains and I modified the taskqueue_start_threads function to specify a cpuset_t mask. The idea here is to stay on the same domain to dispatch the crypto jobs and to notify back the crypto users. This gives quite good performance so it seems to be an promising way. Now the question is: how can I make the taskqueues "domain aware"? Do I have to add some logic in crypto(9) or could this be abstracted in some other part of the kernel? Another annoying part is the kprocs used by the return queues. We would also have to bind them to a single domain. How? What do you think? Emeric From owner-freebsd-arch@freebsd.org Fri May 19 15:34:33 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7081FD73F05; Fri, 19 May 2017 15:34:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wr0-x244.google.com (mail-wr0-x244.google.com [IPv6:2a00:1450:400c:c0c::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE9AD1CE5; Fri, 19 May 2017 15:34:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wr0-x244.google.com with SMTP id v42so3313197wrc.3; Fri, 19 May 2017 08:34:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=qTPSeb6VJikBzJZt6MVUfYj6q5Ij10UcnZd+0Aefra4=; b=Cd+hy7SzpRLOvcEYylp1wltqgEAP1iSkPN9WtUO1yp/4M8sz9RkdSc75xZdSn+T0Jf yWRDXktGl8vob8bISReEpbpucQG+jWHGG61q9mkmokQb4UG0ek0Jm/daPWrgbQRaLYC4 sZw6iIVUd7ek4+hYi7EtQkG1xJUz6JAEplyU1JllG1mTw0nsoTvRHgQnzoWmhi72Ku6+ CozD3TC83vTYpetkqM1gQ8e3Z7ZKebK3STQrWm7Kbyi+Qnwbf/Z7KH+M+rsQHZGJF6DJ i2AFdoTEnFoL97fEpIa8MbKBtBRAkG7EiC7H/R87iqurgOfiYJzr4CQkD5WyyuWFTeLt CN1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=qTPSeb6VJikBzJZt6MVUfYj6q5Ij10UcnZd+0Aefra4=; b=nt90J3lQjl0XY5NgzItlO5NCC6Inzwmo4BmqniOSP3FWH2KqVVrxXp6Q8K6laMBQL7 Asv0klmckYGFxAG4rcy9tYs/P67CiG3qiEdjVT2e9xi84TKaCWxgE195rh0eOqccRuYs BBekJxOfDYPgQWEMuIQukvB2yV1zquQxTn2bqqQD8PFW5Gwg9FSBCvMEJ3PEiLjS6PQ1 mGQEBVfTT3r2gSYrfDhqHdyUcZcDAJMjVritzgOM2nn76vl+iO98aYLZwtVvHdE29cHh UxXjYArQmY8T2BpgA49+kWPUE0MIi1BUm9dNeLcvitId5VW9TFCtDSscZnnFO7oVN18s kU1g== X-Gm-Message-State: AODbwcBPRwJaa4Nx8zpBeOPTnpbkm1Fp2BJzM2fJZVt2xc/ffRqrPqbd fYjdMLh4/rekFN9EUrxNrs/k6cJ1oA== X-Received: by 10.223.152.6 with SMTP id v6mr3369123wrb.60.1495208071402; Fri, 19 May 2017 08:34:31 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.28.193.134 with HTTP; Fri, 19 May 2017 08:34:30 -0700 (PDT) In-Reply-To: References: <20170518055352.bflapm6mmfhgl4y4@ivaldir.net> From: Adrian Chadd Date: Fri, 19 May 2017 08:34:30 -0700 X-Google-Sender-Auth: nhkRr2IcIG19ygdWsTbGPiWnkFg Message-ID: Subject: Re: make concurrency kit a module To: Warner Losh Cc: Baptiste Daroussin , freebsd-current , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 May 2017 15:34:33 -0000 On 17 May 2017 at 23:37, Warner Losh wrote: > On Wed, May 17, 2017 at 11:53 PM, Baptiste Daroussin wrote: >> On Wed, May 17, 2017 at 06:04:09PM -0700, Adrian Chadd wrote: >>> https://reviews.freebsd.org/D10778 >>> >> >> Except there are plans to use it elsewhere. Many areas may be improved using it. >> >> Having it as a module would mean some devs might refrain from using it because >> there is no waranty for it to be there >> >> Areas like VFS and network stack could have a good benefice from using it. >> >> Out of curiousity what size is saved? > > I'd planned on using it newbus to solve the lifetime issues we have > with device_t's.... I'm happy with things using it in base outside of the linuxkpi. I'm just trying to push back on the "death by a thousand cuts" that the IOT platforms face for size constraints. There's plenty of stuff in the base kernel that storage challenged platforms don't need but they're not introduced or kept as modules. It's 2017 and people /are still/ making embedded boards with 8MB of NOR flash. -adrian From owner-freebsd-arch@freebsd.org Fri May 19 15:41:16 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09A46D748D0 for ; Fri, 19 May 2017 15:41:16 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wr0-x234.google.com (mail-wr0-x234.google.com [IPv6:2a00:1450:400c:c0c::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 946A512DB for ; Fri, 19 May 2017 15:41:15 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wr0-x234.google.com with SMTP id l50so19466799wrc.3 for ; Fri, 19 May 2017 08:41:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=2J1B2j75/riAsiT3zP6FsMCXii2xkDSnlPeiSdco7RA=; b=eT9HyhJ8rcL/ZppW8I4XEuFraxcdpPd8k74ZFrRQMKjBtYrTaUA2sRFdkYzOXEnWu1 MCUh7+alaZAKOe1bcqVVdAPmWN981zZoYl64i1BOaOF50EnEAlmGnIbzSDhuTg76jDs8 ifZ7+Z+uQHUaM5RHiQPWuvG3qdDUbV/f/PYCnh72DmECRbHWmRk0iL29wbUeIWrUT5h9 Q3/v+wW4XiNkB+9rxK5Q/DygDdXUDkFvqWEg9KZunSrzJXVkZQOo0q2O9SmB/7l5WNew cUNF59IQuxYDaZp0gCwJT66Io6vqntIB/jzVEGAavULp/alBUZ3vCB7CftaLb7VWK2s7 Dh0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=2J1B2j75/riAsiT3zP6FsMCXii2xkDSnlPeiSdco7RA=; b=oGMKdbzsrdJ54JWiV/Mr2j1uwSrb6pPhcUC7yPvZ8MGE9Rc55T5dePdXQRaAbcGLfg WxUFqQ9P58UJi7SRUvHbg9CxVvMo+bC8TaycsIbQzxeRQ1nxdO77SF+HMVxeapZBKm7j djCM4AvufVy2XtA9FBUNydOzuaOsYzmyeRvXR7f2+BdnSB/lsVjVhIOOB/+WNA+ScTeY oP2jeL1oV4T+w24ICFlOkbiJuX8UlIoJ4gbCdVuDhsnj6dekVUkvlzFEOZ040Zob6NOy zaC9NJPakg4lD0eIEVsdpVMSaCeXzxWbVGqTycfr4h0blFUPiZ59VAbmhFndMVtGz2d/ N+iQ== X-Gm-Message-State: AODbwcCz0PDT0306ygabXTIxb9eRDb6u34PvB069v7MfnT3gXq4jRL1+ x1NL2a2XO3P7M0cbXrC5zIEd5WlAH2dc X-Received: by 10.223.134.80 with SMTP id 16mr4215670wrw.62.1495208473666; Fri, 19 May 2017 08:41:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.193.134 with HTTP; Fri, 19 May 2017 08:41:13 -0700 (PDT) In-Reply-To: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu> References: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu> From: Adrian Chadd Date: Fri, 19 May 2017 08:41:13 -0700 Message-ID: Subject: Re: numa and taskqueues To: Emeric POUPON Cc: freebsd-arch Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 May 2017 15:41:16 -0000 Hi, I've been worried about the trend to create ncpu*taskqueue or ndomain*taskqueue for things unless we really need the priority / preemption behaviour. Otherwise we will just end up with a lot of pcpu/pdomain taskqueues that sit idly and/or compete infficiently. Anyway - I think it'd be nice to have domain aware and pcpu aware taskqueues so we can eventually migrate to a taskqueue group model of "one top level things for net processing" for devices to share, etc, etc. But for the short term just prototype it with some thin API in crypto that wraps the taskqueue / kproc work so it gets done, then push that work out for review/evaluation. if it does indeed work the way you intend, we can try to use it as a template for a higher level, shared taskqueue thing. Thanks, -adrian On 19 May 2017 at 00:13, Emeric POUPON wrote: > Hello, > > I have made a review to boost ipsec performance when very few flows are involved: https://reviews.freebsd.org/D10680 (reviews would be appreciated btw!) > The idea is to dispatch the crypto jobs using a taskqueue (with nb threads = nbcpus), details are in the review. > > However, this does not scale well on multi socket architectures (ex: 2*6 cores), a lot of time is wasted in the locks. > > For testing purposes, I created as many taskqueues as domains and I modified the taskqueue_start_threads function to specify a cpuset_t mask. > The idea here is to stay on the same domain to dispatch the crypto jobs and to notify back the crypto users. > This gives quite good performance so it seems to be an promising way. > > Now the question is: how can I make the taskqueues "domain aware"? > Do I have to add some logic in crypto(9) or could this be abstracted in some other part of the kernel? > Another annoying part is the kprocs used by the return queues. We would also have to bind them to a single domain. How? > > What do you think? > > Emeric > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@freebsd.org Fri May 19 17:27:37 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8EDAD73579; Fri, 19 May 2017 17:27:37 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 917821647; Fri, 19 May 2017 17:27:36 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id F042D2615C0; Fri, 19 May 2017 19:27:33 +0200 (CEST) Subject: Re: make concurrency kit a module To: Adrian Chadd , Warner Losh Cc: Baptiste Daroussin , freebsd-current , "freebsd-arch@freebsd.org" References: <20170518055352.bflapm6mmfhgl4y4@ivaldir.net> From: Hans Petter Selasky Message-ID: <3c4c6496-9a80-a45b-b3a1-9cdfb094c0a2@selasky.org> Date: Fri, 19 May 2017 19:25:34 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 May 2017 17:27:37 -0000 On 05/19/17 17:34, Adrian Chadd wrote: > On 17 May 2017 at 23:37, Warner Losh wrote: >> On Wed, May 17, 2017 at 11:53 PM, Baptiste Daroussin wrote: >>> On Wed, May 17, 2017 at 06:04:09PM -0700, Adrian Chadd wrote: >>>> https://reviews.freebsd.org/D10778 >>>> >>> >>> Except there are plans to use it elsewhere. Many areas may be improved using it. >>> >>> Having it as a module would mean some devs might refrain from using it because >>> there is no waranty for it to be there >>> >>> Areas like VFS and network stack could have a good benefice from using it. >>> >>> Out of curiousity what size is saved? >> >> I'd planned on using it newbus to solve the lifetime issues we have >> with device_t's.... > > I'm happy with things using it in base outside of the linuxkpi. > > I'm just trying to push back on the "death by a thousand cuts" that > the IOT platforms face for size constraints. There's plenty of stuff > in the base kernel that storage challenged platforms don't need but > they're not introduced or kept as modules. > > It's 2017 and people /are still/ making embedded boards with 8MB of NOR flash. Hi, Please make sure that the CK can still be built as part of the kernel, if you plan to make it a module by default. --HPS