From owner-freebsd-bugs@freebsd.org Fri May 7 10:38:41 2021 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2954262855A for ; Fri, 7 May 2021 10:38:41 +0000 (UTC) (envelope-from orca@tira.com.ua) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fc6PD2NB9z4SC8 for ; Fri, 7 May 2021 10:38:39 +0000 (UTC) (envelope-from orca@tira.com.ua) Received: by mail-ej1-x636.google.com with SMTP id w3so12882821ejc.4 for ; Fri, 07 May 2021 03:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tira-com-ua.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=ymYmwBuJIiijYIV1hTdHSj4w/vW7s8megMv9SKmipQ4=; b=V8ZU794tWb5KWxa6PVTgDYbYIlZCnmle4+K/Bt1hLm1t0fyoadtPo5ktIPxLJBATxA Lpl3cA2bgu9gjXS0Bz5He/6iORxkqDA3aqyav7aAPnoWT24XdT1sZ2ghMTv0FvatbKCv Ombl5gAtRpevbBadq4FbjsQb5sWkZq/IaJ+F64ZlG6UNKm5Pd2esjSPx36yA220e7/pq FAFIbXppZcjVcdpXVhGQeTJXPXI4lvUrEQk/bK0XXwhWSqPAmP4wMMSpYbi8cAKRERTZ o31YZDU4KquKCtvOx/zZh/wOwc7FkOSCh2KdVX4BjX8htR0bcB+8k7+I34qf9SbgqFtg mvTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ymYmwBuJIiijYIV1hTdHSj4w/vW7s8megMv9SKmipQ4=; b=XzY2o4vby0D8EHHs+XpV4Hrrt5bNgyAVKbWDOdhHLiXA7RHM7RnlYZ+N70xvu6PJri TfsGxtNxzC/gAb2isia9rwpnm24j7UGfzxyhomMebwRfxabLgPPyft0bmujCsyWyAgE0 ic6dw68+81hvJtvCntdpOuYp4UarMOrL83S7qf3tKmvkirSWQhjrGPdyB344UOrZX9nN P1kuqSFa4CIrhh2U7u9RC+vNAZW5x/cz6aMnGF6QIW5DCHeyXdeNjVkakpKen4L3XkCk razqyHQytanDuIcz4JKcSweSu//LJwKWyMOZyNjX3QTnLCZ6mhfjo6hKr0z798L3aMzk 0mbg== X-Gm-Message-State: AOAM531dTJ/uFmwmokTc2+9RHUveBelUeyx1fpAjm8quyTmfeLQAn/Zy fwRskamWIX7kgvy3NPaNqL9D72rGbNkzuQCmWvDXYflh1YH020VS X-Google-Smtp-Source: ABdhPJxtuu4P51UrR1KUs/z2gw6WkbyLR1Zrgf8pzK2clO0RmfleOiFCpzvOuIV91GVtihG3/kSlV/qy/mrtCOPzck8= X-Received: by 2002:a17:906:724d:: with SMTP id n13mr9575315ejk.413.1620383918147; Fri, 07 May 2021 03:38:38 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?B?0JDQvdC00YDQtdC5INCT0LXRgNCw0YHQuNC80L7Qsg==?= Date: Fri, 7 May 2021 13:38:30 +0300 Message-ID: Subject: FreeBSD 13, Iflib, 2 CPU problem To: freebsd-bugs@freebsd.org X-Rspamd-Queue-Id: 4Fc6PD2NB9z4SC8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tira-com-ua.20150623.gappssmtp.com header.s=20150623 header.b=V8ZU794t; dmarc=none; spf=none (mx1.freebsd.org: domain of orca@tira.com.ua has no SPF policy when checking 2a00:1450:4864:20::636) smtp.mailfrom=orca@tira.com.ua X-Spamd-Result: default: False [-3.20 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[tira-com-ua.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-bugs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::636:from:127.0.2.255]; DMARC_NA(0.00)[tira.com.ua]; DKIM_TRACE(0.00)[tira-com-ua.20150623.gappssmtp.com:+]; MIME_BASE64_TEXT(0.10)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::636:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::636:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-bugs] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 10:38:41 -0000 Recently I have updated to FreeBSD 13, but it had begun since iflib was added to FreeBSD12. Behavior is the same on different servers. CPU 2 x CPU E5-2670 NIC driver is ix. It is standard. As I understand, when the system is starting it creates threads for every core of the processor. procstat -at | grep if_io 0 100019 kernel if_io_tqg_0 -1 24 sleep - 0 100020 kernel if_io_tqg_1 -1 24 sleep - 0 100021 kernel if_io_tqg_2 -1 24 sleep - 0 100022 kernel if_io_tqg_3 -1 24 sleep - 0 100023 kernel if_io_tqg_4 -1 24 sleep - 0 100024 kernel if_io_tqg_5 -1 24 sleep - 0 100025 kernel if_io_tqg_6 -1 24 sleep - 0 100026 kernel if_io_tqg_7 -1 24 sleep - 0 100027 kernel if_io_tqg_8 -1 24 sleep - 0 100028 kernel if_io_tqg_9 -1 24 sleep - 0 100029 kernel if_io_tqg_10 -1 24 sleep - 0 100030 kernel if_io_tqg_11 -1 24 sleep - 0 100031 kernel if_io_tqg_12 -1 24 sleep - 0 100032 kernel if_io_tqg_13 -1 24 sleep - 0 100033 kernel if_io_tqg_14 -1 24 sleep - 0 100034 kernel if_io_tqg_15 -1 24 sleep - When the network is loaded as I see in "top" only the first 8 threads are working which are binded to the first processor. So the second processor is not used at all. I've been trying a different combination of sysctl tuning and couldn't load the second processor. The only thing that helped was installing ix driver from ports collection which does not use iflib. And only after this i see 8 threads per NIC port that can be binded to different CPU cores. But when a server has more than 16 cores then it does not use excess ones, and if server has less than 16 cores, then few threads go to each core. I thought that it would be changed with FreeBSD13 but it wasn't