From owner-freebsd-hackers@freebsd.org Mon Nov 30 15:49:53 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 440584A11E1 for ; Mon, 30 Nov 2020 15:49:53 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cl8nD3SjKz3pn8 for ; Mon, 30 Nov 2020 15:49:52 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-qt1-x831.google.com with SMTP id g17so8444162qts.5 for ; Mon, 30 Nov 2020 07:49:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=4Jdp4gx5zbvS+IYd0QB0OJDiJVdmg2CIyL91d+ryHKw=; b=I42Erk0rNyk1ZFptvZbA8N0+IIqAAqCLqcqsxdLNCZJVrwJoYodwMNAgVYsnde7Ci1 qnncPPpQFE/Z0FlytLge5d+yJRjx7KbBadTUBJktDAg8Cjky5SJtVbLH/58vsJ1uTUhq xb0Xqdh8R7AO/rovCrdNPqUfTyffnXZqA9VQJa1b/72UD0vOww4HYyIa/tptw4I6n4mp 3hWt0cbGGfEleAgKa4ioKX9YEEcxqhnw45D8s09/Q6h9Uflbx0ceUoK+eeHNHRWW7LJl 7jWGYQBZj/OydC9XdyljcDBSYKG8XDDQ2SRwFeI6GLIvfly6yI3q/vD5Wy7d7/vFa+gL 6RgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4Jdp4gx5zbvS+IYd0QB0OJDiJVdmg2CIyL91d+ryHKw=; b=aj/cEv3JIyz88VHbCKH77A8X2LsGU7+bqMftdf8hwVFTTaE34UDp1oWDEYEyF7q3II HTpiwki8Q8ubiN8UjJTWeEk0FzEAIf0RUzffH+Jr4Tl7iUJ9lyqLs5uTE8l6YY7ZsIOF thIrWcOzePBCXmoR0UJaGhcieFJlXOFud8ZFIP4fDP2f1tML2IE2vC7N3wjfJ3BwWcdY LLXip0c8SpXaT23lBFzmkkBGKGSk1WpJvrWKQoGd3gjznhcYpchJyTVWFsknYGZfeu8Q cM33l/Jrf3pA3cNwPfmUTC4Q0ZRz9Gyjif//uzAOnnAJBfKel7RVt31HcK8U8IHMsCJz rejA== X-Gm-Message-State: AOAM532yi8nIT51Qvq5LzRMqqwqKBz7kGx/i9O4n+l5vrteDE1D/kvBE mqYwlNrin94TA/ET+pF6l4ECr8RxG1MinRkXE2BwIQ== X-Google-Smtp-Source: ABdhPJye0FP0Fud9eS5WbPbuBP/f+4vQ8Mr/4rDGh+UTbCTPupL0djiOUEGBc0UA7wibQyyRUSJnUp7Ciyci598nLKQ= X-Received: by 2002:ac8:4e4c:: with SMTP id e12mr22580890qtw.164.1606751391622; Mon, 30 Nov 2020 07:49:51 -0800 (PST) MIME-Version: 1.0 From: Mark Saad Date: Mon, 30 Nov 2020 10:49:41 -0500 Message-ID: Subject: Pinning Threads To: freebsd-net@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Cl8nD3SjKz3pn8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=I42Erk0r; dmarc=none; spf=none (mx1.freebsd.org: domain of nonesuch@longcount.org has no SPF policy when checking 2607:f8b0:4864:20::831) smtp.mailfrom=nonesuch@longcount.org X-Spamd-Result: default: False [-3.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::831:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::831:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::831:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 15:49:53 -0000 Hello All I was curious about what you think of this solution to not having enough cores. I have an 8 core server with a solarflare nic. The Nic is a two port card so it will have 8 queues per port. Next I have netisr set to start 8 threads as well. Now to get the better performance out of pf and this box I was playing around with using cpuset to pin the pf, isr and sfxge threads on particular cpus. But I think I am running into some weird contention. So two questions. Has anyone tried to pin netisr threads to cpu sets , eg cpuset -t 100233 -l 3-7 , also on the same note have you ever tried this with the device queue threads ? Am I playing with fire ? to be clear this is what I used to get the device threads # procstat -ta |awk '/sfxge/ {print $2}' and what I did to get the isr threads procstat -ta |awk '/isr/ {print $2}' -- mark saad | nonesuch@longcount.org