From owner-freebsd-arch@FreeBSD.ORG Fri May 9 20:05:54 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FBEE805; Fri, 9 May 2014 20:05:54 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFBD6DEE; Fri, 9 May 2014 20:05:53 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so5066351qcx.41 for ; Fri, 09 May 2014 13:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=K4fROt0upQbhbNwC4VXqitoBtOrFfrEAiFREHIC5Bqg=; b=jwXoyclbhc+MbxwUTTrG4F3OVPcO6bER7Yr/BqIPcqpxn58O0Z313RhcPqQCXHwq8E 99dz8ihiHUMdTOeX5Gh2Fxul3/O9JyX8DZVCv6uX9W9vT8q/9NaKiQYc86L+tprp8CBi zBue6E0TV3cXkYa3o6w7aIRwtT2uU1Be0yhu9UJa8wK1PznqwjYKOuVtaBTZu5qZ36UA c3xV3vii73gT7xRRiT1WLQssfjSJSY3ASkrXYYcsnAFq4qYk4NZrlojGwpkkWWT0PjNT OWJ5DsOpOQuuZmPQwbITumkENLhX4f6nl+xLgf9up5yQm0HxQYu3EJyJkz1Sq2v2LO4z LpYA== MIME-Version: 1.0 X-Received: by 10.140.42.85 with SMTP id b79mr16760249qga.87.1399665952387; Fri, 09 May 2014 13:05:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 13:05:52 -0700 (PDT) In-Reply-To: <536D3184.9070302@freebsd.org> References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> <536D3184.9070302@freebsd.org> Date: Fri, 9 May 2014 13:05:52 -0700 X-Google-Sender-Auth: n8CryHSl9hlTy00b1sv-8jkOliA Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 20:05:54 -0000 On 9 May 2014 12:50, Peter Grehan wrote: >> How about i instead do the comprimise: >> >> * i'll pin all other swi's >> * default swi isn't pinned by default, but one can flip on a sysctl at >> boot time to pin it >> >> How's that sound? > > > And also please a sysctl that disables any swi pinning. > > It is sometimes useful to change the default cpuset, for instance to > allocate a subset of CPUs to some particular applications and not FreeBSD. > Having kernel threads pinned prevents this from happening since they are in > the default set. > > (Note that some network drivers are also culprits here, though disabling > MSI-x in them is a workaround). Yup. I've just done that. http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff Which workloads are you thinking about? Maybe we could introduce some higher level description of which CPU(s) at boot time to do "freebsd stuff" on, and then don't start things like pcpu swi's and NIC threads on those CPUs. Can you think of situations where we'd want to have per-cpu swi's even _running_ for CPUs that you want to dedicate to other things? There's nothing stopping you from scheduling a callout on a different target CPU. (It'd require some work, but it likely should be done in some form as part of overall RSS framework hacking.) -a