From owner-freebsd-drivers@freebsd.org Fri Feb 28 09:03:39 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 63DA025E28A; Fri, 28 Feb 2020 09:03:39 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TNqt2j9hz4B5q; Fri, 28 Feb 2020 09:03:38 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wm1-x32b.google.com with SMTP id m10so8858547wmc.0; Fri, 28 Feb 2020 01:03:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=JWM2CyapOK7wXHewObTe2Y3FnRDjW3i4+6ZxSke8TA4=; b=OtwIJ59v10yKJxMgOiKIcsxIfTDU/Rv8ILpnd6WSWCykOMQTuQY4oRTVieRM7EWsbl wyK4Yt6D3VWj7WYMCyDPzZrkMHw+wzZaRJzr8i73YcVmtg4Ga4quz7LK2caUwHdOzK/Y JYgFgf8yPf7C8CSIz4hfUY2dmyiTwD2pYRn8NUNKtEMV4awoYweVvv540tmxv4FJhI1a s1G1I6ThSPZqZJKJ45l7K0qa/iLfYIqwu2WnU5Pbbs1IiC7eVehIoHLFwk5rlNJoRmf0 qCSWu809KjdDADM89n5ZvnX7/8Vn7nizFhzDa4D8Y0+wnuQDY6PF4+W2FBPINXRDgxex H1Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=JWM2CyapOK7wXHewObTe2Y3FnRDjW3i4+6ZxSke8TA4=; b=Dr7nX62V2jNM8VEXHYQUUWgXECeOHZcHFRKaI3oyGulff6t26M2hN0aYgMC1kj6QY6 LMUzDCXz7+rtzQRV9D9zFwYq2fwXOd2Mk8mfGa9A8PkSfh++462FRf0Rzf9KBbgx8XUz lBztzTpkTVm7c6XrSHT13yhNTls4flqcSSAYX9IXX9KAqmiHhf+ZV5YI1+hXO2jJAV+k OqdO/Fy2ybGuNa32bje87tK1usOHMai/ncckT++e4qSDw/7O2xinYqL+p4U8nvqCRVDZ T2njHju4Ynf+yw0qJsu6jfUoimLhShALNGm7JTb6EY2hRmIBlvthvWJIHC0B0Xd6SoHe wbXw== X-Gm-Message-State: APjAAAV52qdw21Jq4wrq7pGUA0LPJ03wHfW/F4rfATUGQSmGTUnYgNfB br+4yR+oweOXJg7B9QOLSiRJFF420BuHmHBynp5biQ== X-Google-Smtp-Source: APXvYqwPHg4hDda0TTO9t1yolVAElyOSffHxMjveVN/JFvEZhQC2MZ3Ko7s7VX4o9+M+4jir4LozMkuhNlXf9bAOHaM= X-Received: by 2002:a1c:6a15:: with SMTP id f21mr3711316wmc.126.1582880616386; Fri, 28 Feb 2020 01:03:36 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Fri, 28 Feb 2020 14:33:25 +0530 Message-ID: Subject: Network throughput not reaching line rate. Need clarification on iflib. To: freebsd-drivers@freebsd.org, FreeBSD Hackers X-Rspamd-Queue-Id: 48TNqt2j9hz4B5q X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OtwIJ59v; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::32b as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (-9.15), ipnet: 2a00:1450::/32(-2.41), asn: 15169(-1.67), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[b.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 09:03:39 -0000 Hi FreeBSD team, I am writing a network driver using iflib framework and using "iperf3" tool for performance testing. 1) 10G link with 9000 MTU. Seeing line-rate numbers on both Tx and Rx path. 2) 10G link with 1500 MTU. Seeing line-rate numbers on Tx path, but low numbers on Rx path. I suspect the driver design here, but need some clarification on the iflib framework. My driver expects two memory buffers per receive descriptor. So I used "2 fls" per rxq (means isc_nrxq = 1, and isc_nfl = 2). For every descriptor, I use the memory buffers at same index from both the free-lists to refill my descriptors. Also, I return the fragments with appropriate index and flid from rxd_pkt_get. This approach works for me. But I am facing performance issue as stated above. So, Is this approach perfect? or any better way of doing? Also, I did a quick analysis of tcpdump on both iperf3 server (my test side) and client side(support side). iperf3 clients pumps traffic to iperf3 server. After a point, server sends an ACK with SACK option (SLE and SRE) to indicate missing frame to the client. And client is re-transmitting that segment as per protocol and the server receives it. But still I see the ACK sent again with SACK option. So client keeps on re-transmitting the packets and hence the performance drops. Any reason why server is not acknowledging the re-transmitted packet? If the driver Rx isn't proper, then I may not see the packet in tcpdump. But I see it. So any idea, what could be wrong here? Please let me know if any more details are needed. Thanks, Rajesh. From owner-freebsd-drivers@freebsd.org Fri Feb 28 09:34:58 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5BD0A25EB9D; Fri, 28 Feb 2020 09:34:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48TPX073DJz4NRs; Fri, 28 Feb 2020 09:34:56 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2020.home.selasky.org (unknown [62.141.129.235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 3DBD826010C; Fri, 28 Feb 2020 10:34:49 +0100 (CET) Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: Rajesh Kumar , freebsd-drivers@freebsd.org, FreeBSD Hackers References: From: Hans Petter Selasky Message-ID: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> Date: Fri, 28 Feb 2020 10:34:36 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 48TPX073DJz4NRs X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 88.99.82.50 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-5.42 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[selasky.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-3.12)[ip: (-9.32), ipnet: 88.99.0.0/16(-4.71), asn: 24940(-1.55), country: DE(-0.02)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.99.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 09:34:58 -0000 On 2020-02-28 10:03, Rajesh Kumar wrote: > Hi FreeBSD team, > > I am writing a network driver using iflib framework and using "iperf3" tool > for performance testing. > Is there any difference with "iperf" tool and using multiple threads? I think iperf3 is single threaded ??? --HPS From owner-freebsd-drivers@freebsd.org Fri Feb 28 16:35:02 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6761D266BA4 for ; Fri, 28 Feb 2020 16:35:02 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TZrj3B55z3y2R for ; Fri, 28 Feb 2020 16:35:01 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: debdrup) by smtp.freebsd.org (Postfix) with ESMTPSA id 348172C0D for ; Fri, 28 Feb 2020 16:35:01 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: by mail-oi1-f179.google.com with SMTP id v19so3371809oic.12 for ; Fri, 28 Feb 2020 08:35:01 -0800 (PST) X-Gm-Message-State: APjAAAUwUYHJ63fd90t/NanyyB5mQ7Ul7uF+j8HT+RYwYMssftO4Bwpq jHaqFu4KPP4m/7OR9JYKXhC3LP+UfLBAFqGGLqxMPg== X-Google-Smtp-Source: APXvYqyPvngcCRND7FjzNQVIRR3XEnsF0Z5IMzhNkiGpaQRvD3+xel50IeSdVP0xWgHA8w6zSZ7Tso0G6azuz8durWI= X-Received: by 2002:aca:c3d1:: with SMTP id t200mr1574470oif.41.1582907700441; Fri, 28 Feb 2020 08:35:00 -0800 (PST) MIME-Version: 1.0 References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> In-Reply-To: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> From: Daniel Ebdrup Jensen Date: Fri, 28 Feb 2020 17:34:48 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: freebsd-drivers@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 16:35:02 -0000 Yes, iperf3 will default to single-threaded packet generation, et al. which favours fast cores with frequency boosting facilities. You might want to use iperf2 as that's properly multi-threaded, or you can use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky wrote: > On 2020-02-28 10:03, Rajesh Kumar wrote: > > Hi FreeBSD team, > > > > I am writing a network driver using iflib framework and using "iperf3" > tool > > for performance testing. > > > > Is there any difference with "iperf" tool and using multiple threads? I > think iperf3 is single threaded ??? > > --HPS > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-drivers@freebsd.org Fri Feb 28 18:36:25 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DA76A241A0F for ; Fri, 28 Feb 2020 18:36:25 +0000 (UTC) (envelope-from bmah@kitchenlab.org) Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TdXm41dvz4HQx for ; Fri, 28 Feb 2020 18:36:24 +0000 (UTC) (envelope-from bmah@kitchenlab.org) Received: by mail-pj1-x1044.google.com with SMTP id q39so1648000pjc.0 for ; Fri, 28 Feb 2020 10:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kitchenlab.org; s=google; h=from:subject:to:references:autocrypt:organization:message-id:date :user-agent:mime-version:in-reply-to; bh=2Mt2r9pDXCZhGKHvjXzGEVnjqVgAu99DP4bO+aDhxC0=; b=WAHUizXaCs7YeZwoOdYauanJ8B4yrGwFBx9V+RyQpblge44YP6/8YxDeV9fJhkRLpS xMs9CaFQ3QnlTj0U7pzV9z0VkXZda5ert0v/k7ZFERgaQMKCorcBqqHRv4Jv+LXCL3/q J0kKzT3FCjGp8/tFBu9rFsSd/DLW0cAui/+L9BG6belaAe0Kdywt9ZRd6jCXVSyRt4xi mQCPYACPgpnPUezta/c/BCk95PRQAsuVX2jG1PlgGTnD0WnNMBf1+3tUANqZ+CgTe3oK NcS9cNQb5y7aJ19vm7qGFEo9e+Fp6/IPzwyRLbtkxttHut1flApuJJlCEYEYGh7+B+35 8spg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:autocrypt :organization:message-id:date:user-agent:mime-version:in-reply-to; bh=2Mt2r9pDXCZhGKHvjXzGEVnjqVgAu99DP4bO+aDhxC0=; b=dkt3cONusw5s7/C3yzeAMTFHMUv2JoyJQKfJ332ZLGx4fAmRmLzxvJEpz7msW2w4ht XzNlizhUOQdXrVAzLsHuistZFT2suIAZIbTbEv1W+JPF6E3/6zaPFLx3kGEhPb20WTg1 juqopcF/JvdXGbir17aCcqgQ7kxa1YV4omgJroCPnfiA/ykexp0o+c7JQjK3sZcRVqDK Fb8B0y3La1vsJhATVNDdoRPkx0KMlEKIQjZbG09Y4xlrw+oGHiAK7Coz3vuO/2/jjgpy 9FoFHLcohsoVH78t8FSB1wq/9hRwzLmJ4GvqWlTVBTmRZ4VWQdaCsyKeCbyzPXXKfcj9 H5vg== X-Gm-Message-State: APjAAAX52OsXw8zso0Fw+3u2zKY+uzH+cCPm18RIrXzl567nvlAX5aKj gkX4BIT72UbBJHH3MxwWrmu5kQ== X-Google-Smtp-Source: APXvYqzrdiXP7RfYZqKG97pb6ThwG5EMxZdXCENYIIteqsQer027FOXQiR3OMbjQpa5qj7ro9aRZPA== X-Received: by 2002:a17:90a:c20d:: with SMTP id e13mr6090624pjt.95.1582914982488; Fri, 28 Feb 2020 10:36:22 -0800 (PST) Received: from bmah-macbook-pro.int.kitchenlab.org ([2600:1700:db70:3e2f:11f8:e469:756a:9561]) by smtp.gmail.com with ESMTPSA id h3sm12443183pfr.15.2020.02.28.10.36.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 28 Feb 2020 10:36:21 -0800 (PST) From: "Bruce A. Mah" X-Google-Original-From: "Bruce A. Mah" Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: Daniel Ebdrup Jensen , freebsd-drivers@freebsd.org, FreeBSD Hackers References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> Autocrypt: addr=bmah@freebsd.org; prefer-encrypt=mutual; keydata= mQENBFLYd6cBCADHPcs4srbu+OyftI0SoPefUGJ421Ibtfh1iK/NkvUOp/ueMdKK3QI2ha1D t5egMG+pMjiQRio1EMEhkobWL15nX7E5WH6xKgZz6Oml2psrYHKo3AIuTXFPkj+jG18RlfZE Ln9EwTHFrvOf15hTQdCnilfbPsYCCz76rzzo/1jM5AvrBjoXR/zAWzEqCcTFTVxtgNw/QrSq vfGntng/cLQFY7oB1OZOTXYT/8zs4q5aRDZnZPsAcgMse/3SWf5CwCdPX9Y9jnBj8loTHHOr hGarROLeeTlFoNs0LAd0KRJMDrc9+fIJ1Q462T2YBAIgbN4P7SJhHVc9rr1iat9vWnp3ABEB AAG0H0JydWNlIEEuIE1haCA8Ym1haEBmcmVlYnNkLm9yZz6JAVQEEwEKAD4CGwMFCwkIBwMF FQoJCAsFFgIDAQACHgECF4AWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2CAUJENrAGQAK CRBJhJEKjKruivuHB/sH5ALYcG60UlC8Ooy9CGVYxFyQ/sSZdgJlAazl9QJhbCT/z1xyqhCN eFuTJWb4dObkIxhdLwgrzXTltVv2QXSmuS0qvHxDB3L56pfbXlLodgX6Bd3CaJhcg2H3m9U6 VWDMv4hKxRiQI3/NvDTQGaGly1a4orwqAynZv3dVP+ikYsEFL2ZZiMGrjaIUBoUQAWl0AXK4 qdaazQb4X5rjvE4GU5ZWVmLrz2d4slQwd6/4PObtx51ZeZGseDhaJk/gH3rxXnAU8uANazPV OpfIGT7Y6AhbiB2Synglq97e2WsSSZXUmHDWnAJ9CobwU6JEgG3BPetxBZ6tYJOTqZRuUwTb uQENBFLYd6cBCACqVFQGWpmLMtfweSMsQE3JD2XDy4dyux7wcI3MNE9Qx1HcpbVRAm7YB+bD bi/rwyV+92x9SvLlviOj7iRTi297xInn7dB9Q9agYcNTQbKX8QKia3Ie6bgbKpSgeeZ9RgQL h7smDJukqFWofL0PfsB4u+Z7ISWIio7ZS+jRaReyz9Ol2rCyxUmqYzztA3b5yeviY8uAiUrE JTx515TMNHf3Wx+Oa/iJROXWTaSZgRhA8z+l1+v29VVPoHU592kIE1fkxrWYFDmkvnnpdQ3g KkKe/OzjMWb8Ildz/YZB8PheLrA8cA60LeeQr/INzD02pbFyA3YuBa7n9FLNLJ4mfAnhABEB AAGJATwEGAEKACYCGwwWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2PQUJENrAGQAKCRBJ hJEKjKruivm1B/0VfEuucLbqNfWd8YSXb48EXMuyTITilG+oEXJPIObpS792Ri0l/Ubglqoo WFwWeQWVwpa1T7gd0nMYtq8CjK5bRc5qyHjClugTBTFTjRrkv9LhVjR/q6KAzwDodv+CWGB8 Z6PWZOSf6k9zwIxZVHFKO5fLy4r5GIGsGW/Um0IKgPlR0Yqgt5GJS+KWD65EjDj3qQQgDmQJ xee9Nm61Lrzmd9WuJGJEXmyhkVgDjm06fRzoxrkWk/wg+9/Z0FjdXwhMrhC5diOqThIc090O 1Lom08xSSzv05fnwGUDy4FS7IgWTY1Qi9IOhhvtyvS2/1tureX3i0P9dFQ2q1OLgh0XFuQEN BFLYd6cBCACqVFQGWpmLMtfweSMsQE3JD2XDy4dyux7wcI3MNE9Qx1HcpbVRAm7YB+bDbi/r wyV+92x9SvLlviOj7iRTi297xInn7dB9Q9agYcNTQbKX8QKia3Ie6bgbKpSgeeZ9RgQLh7sm DJukqFWofL0PfsB4u+Z7ISWIio7ZS+jRaReyz9Ol2rCyxUmqYzztA3b5yeviY8uAiUrEJTx5 15TMNHf3Wx+Oa/iJROXWTaSZgRhA8z+l1+v29VVPoHU592kIE1fkxrWYFDmkvnnpdQ3gKkKe /OzjMWb8Ildz/YZB8PheLrA8cA60LeeQr/INzD02pbFyA3YuBa7n9FLNLJ4mfAnhABEBAAGJ ATwEGAEKACYCGwwWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2PQUJENrAGQAKCRBJhJEK jKruivm1B/0VfEuucLbqNfWd8YSXb48EXMuyTITilG+oEXJPIObpS792Ri0l/UbglqooWFwW eQWVwpa1T7gd0nMYtq8CjK5bRc5qyHjClugTBTFTjRrkv9LhVjR/q6KAzwDodv+CWGB8Z6PW ZOSf6k9zwIxZVHFKO5fLy4r5GIGsGW/Um0IKgPlR0Yqgt5GJS+KWD65EjDj3qQQgDmQJxee9 Nm61Lrzmd9WuJGJEXmyhkVgDjm06fRzoxrkWk/wg+9/Z0FjdXwhMrhC5diOqThIc090O1Lom 08xSSzv05fnwGUDy4FS7IgWTY1Qi9IOhhvtyvS2/1tureX3i0P9dFQ2q1OLgh0XF Organization: The FreeBSD Project Message-ID: <16c58e55-9b0f-54d0-9aaa-f1b574048671@freebsd.org> Date: Fri, 28 Feb 2020 10:36:17 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="yp9z0dSYmR4pBWMgl78MLE03sI0ggup6Y" X-Rspamd-Queue-Id: 48TdXm41dvz4HQx X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=kitchenlab.org header.s=google header.b=WAHUizXa; dmarc=none; spf=pass (mx1.freebsd.org: domain of bmah@kitchenlab.org designates 2607:f8b0:4864:20::1044 as permitted sender) smtp.mailfrom=bmah@kitchenlab.org X-Spamd-Result: default: False [-4.42 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[kitchenlab.org:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-drivers@freebsd.org]; DMARC_NA(0.00)[kitchenlab.org]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_TO(1.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[kitchenlab.org:+]; RCVD_COUNT_THREE(0.00)[3]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; IP_SCORE(-0.82)[ip: (-0.49), ipnet: 2607:f8b0::/32(-1.88), asn: 15169(-1.67), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 18:36:26 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --yp9z0dSYmR4pBWMgl78MLE03sI0ggup6Y Content-Type: multipart/mixed; boundary="YoOZ58E6dQVYHmzSFzy7ncj7qzHR290ow" --YoOZ58E6dQVYHmzSFzy7ncj7qzHR290ow Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable If memory serves me right, Daniel Ebdrup Jensen wrote: > Yes, iperf3 will default to single-threaded packet generation, et al. w= hich > favours fast cores with frequency boosting facilities. > You might want to use iperf2 as that's properly multi-threaded, or you = can > use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. While it's true that iperf3 is single-threaded, it should be capable of saturating a 10GE link with a single TCP connection, given proper command-line arguments (in particular, specifying a sufficiently large socket-buffer size with the -w option). But based on the symptom of packet loss, I'd say the single-threaded vs. multi-threaded argument might not be relevant to the problem that the OP has. Bruce. > On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky > wrote: >=20 >> On 2020-02-28 10:03, Rajesh Kumar wrote: >>> Hi FreeBSD team, >>> >>> I am writing a network driver using iflib framework and using "iperf3= " >> tool >>> for performance testing. >>> >> >> Is there any difference with "iperf" tool and using multiple threads? = I >> think iperf3 is single threaded ??? >> >> --HPS >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.= org" >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 --YoOZ58E6dQVYHmzSFzy7ncj7qzHR290ow-- --yp9z0dSYmR4pBWMgl78MLE03sI0ggup6Y Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCgAdFiEE+Fo4IENp9xo01E6DSYSRCoyq7ooFAl5ZXaQACgkQSYSRCoyq 7ooCJQgAmtjKq8Al1R2qljwSIGMOCEmScsBDkLyaML7957xdaVTPPzjWizrWCiIi KapqGUrcRv2lBYBSu72dClNiWraBoFZaHJlW3P6ox61enZzXxz1bQK0RkOu7CJ3X yBz4A+IjEgKOysgZJ34SzhB8dFYVhtwHHUprOMc5SUZQ3waRU8JU2KM8jett9mg0 7BO21tJ07ss42DXEVLNLVfg5sHEBe4Da0WTBeZT/ViaHsRaHvi5ENGMbnzjlpCma +YQhCo0Tt1ZTImdc/rZ7PaE4yVfb9fJhU0UqKS1MzZNm9QFn1k0CQzHRmy1IlXtO jEwgBJjooGey1czpCE65IsliCBnQ3g== =wZrW -----END PGP SIGNATURE----- --yp9z0dSYmR4pBWMgl78MLE03sI0ggup6Y-- From owner-freebsd-drivers@freebsd.org Fri Feb 28 18:39:31 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 028BE241AFC for ; Fri, 28 Feb 2020 18:39:31 +0000 (UTC) (envelope-from bmah@kitchenlab.org) Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TdcK24Vbz4Q8l for ; Fri, 28 Feb 2020 18:39:29 +0000 (UTC) (envelope-from bmah@kitchenlab.org) Received: by mail-pg1-x541.google.com with SMTP id 6so1957622pgk.0 for ; Fri, 28 Feb 2020 10:39:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kitchenlab.org; s=google; h=from:subject:to:references:autocrypt:organization:message-id:date :user-agent:mime-version:in-reply-to; bh=rQz6wOpfropt450wAFp062oy5/AsKX/XTRE3W/fap5I=; b=PsIHTMaYDHWje1GdaWMmwxEZ6qy73z3u3uNVT1WFb05yTFvKDUe+qZ5vNZyD4/RAch 2f/FGK2+QqrHoNjyMq7toHlcBpwt0idmF7M2VU6GYFTc02GCH4BRqF1G88WrFLEJ+Ydm BSLrkqxswag+ECPyW0FmKAZd6WMEFGjMxekZmE4FBQtl8BNLiww21UFLOKmsr2KKqidP hc/yoaQngFtcYzX5AR/ingjvc1bHy0UjY5U/Mh0LPvjGgIhqpnRHM/fk0EXQd5fxF32p FJlDjdJrgFRV3vH+0NkpDhhywdLeu26xjCFxU1g/aIHKHVqikreJUt2WtBGpUmHwR+kj rrFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:autocrypt :organization:message-id:date:user-agent:mime-version:in-reply-to; bh=rQz6wOpfropt450wAFp062oy5/AsKX/XTRE3W/fap5I=; b=JiXiYHBvRIu9rCK4VftY2xsvMqlH6Z+s5jglLJlp+jvwr/n5SedHxfwiI040LunYf7 16VUjQV9eDNXQOnHab+um8VUsaZhNqZ1+HA3KMkrX16/IxW0pYVub6Tu3aWclyHcCRk3 2bmzZActbhVd1m84/5EJZuQWCRukSqA5/xgaCa+ISNq+rF5v5v5B6i4lNSHZ+Mb/Bbn+ VxPSJXg+Ri2SPr1rQPJnkPCa1h7HxS/VpqeKdbmRzum/B8WNPSRzpTG5tJsc0+YzYONx O1bxFWxCAVxXlimKHK9ZCfXpggo/c2kY3mqsx9sKBX4jzMBQJbJsCinQXpxWkWI1U7LB B4yA== X-Gm-Message-State: APjAAAVklZ+yDUeI5NAv3s+kMYrWoviMyoTit7s9Ombn2YrUuKkkPL2e CpadxiE+WJEhKuFXRVB5LDODkzz3UFk= X-Google-Smtp-Source: APXvYqwo+WL+NraF6IPsjW0BsUY5qcYUXIuxw9U7m2UMFVjB9VHjOYOV4PSqAVZJlNcmcTopB/WH1Q== X-Received: by 2002:a63:aa07:: with SMTP id e7mr5917525pgf.90.1582915167487; Fri, 28 Feb 2020 10:39:27 -0800 (PST) Received: from bmah-macbook-pro.int.kitchenlab.org ([2600:1700:db70:3e2f:11f8:e469:756a:9561]) by smtp.gmail.com with ESMTPSA id 18sm12345172pfj.20.2020.02.28.10.39.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 28 Feb 2020 10:39:26 -0800 (PST) From: "Bruce A. Mah" X-Google-Original-From: "Bruce A. Mah" Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: Daniel Ebdrup Jensen , freebsd-drivers@freebsd.org, FreeBSD Hackers References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> Autocrypt: addr=bmah@freebsd.org; prefer-encrypt=mutual; keydata= mQENBFLYd6cBCADHPcs4srbu+OyftI0SoPefUGJ421Ibtfh1iK/NkvUOp/ueMdKK3QI2ha1D t5egMG+pMjiQRio1EMEhkobWL15nX7E5WH6xKgZz6Oml2psrYHKo3AIuTXFPkj+jG18RlfZE Ln9EwTHFrvOf15hTQdCnilfbPsYCCz76rzzo/1jM5AvrBjoXR/zAWzEqCcTFTVxtgNw/QrSq vfGntng/cLQFY7oB1OZOTXYT/8zs4q5aRDZnZPsAcgMse/3SWf5CwCdPX9Y9jnBj8loTHHOr hGarROLeeTlFoNs0LAd0KRJMDrc9+fIJ1Q462T2YBAIgbN4P7SJhHVc9rr1iat9vWnp3ABEB AAG0H0JydWNlIEEuIE1haCA8Ym1haEBmcmVlYnNkLm9yZz6JAVQEEwEKAD4CGwMFCwkIBwMF FQoJCAsFFgIDAQACHgECF4AWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2CAUJENrAGQAK CRBJhJEKjKruivuHB/sH5ALYcG60UlC8Ooy9CGVYxFyQ/sSZdgJlAazl9QJhbCT/z1xyqhCN eFuTJWb4dObkIxhdLwgrzXTltVv2QXSmuS0qvHxDB3L56pfbXlLodgX6Bd3CaJhcg2H3m9U6 VWDMv4hKxRiQI3/NvDTQGaGly1a4orwqAynZv3dVP+ikYsEFL2ZZiMGrjaIUBoUQAWl0AXK4 qdaazQb4X5rjvE4GU5ZWVmLrz2d4slQwd6/4PObtx51ZeZGseDhaJk/gH3rxXnAU8uANazPV OpfIGT7Y6AhbiB2Synglq97e2WsSSZXUmHDWnAJ9CobwU6JEgG3BPetxBZ6tYJOTqZRuUwTb uQENBFLYd6cBCACqVFQGWpmLMtfweSMsQE3JD2XDy4dyux7wcI3MNE9Qx1HcpbVRAm7YB+bD bi/rwyV+92x9SvLlviOj7iRTi297xInn7dB9Q9agYcNTQbKX8QKia3Ie6bgbKpSgeeZ9RgQL h7smDJukqFWofL0PfsB4u+Z7ISWIio7ZS+jRaReyz9Ol2rCyxUmqYzztA3b5yeviY8uAiUrE JTx515TMNHf3Wx+Oa/iJROXWTaSZgRhA8z+l1+v29VVPoHU592kIE1fkxrWYFDmkvnnpdQ3g KkKe/OzjMWb8Ildz/YZB8PheLrA8cA60LeeQr/INzD02pbFyA3YuBa7n9FLNLJ4mfAnhABEB AAGJATwEGAEKACYCGwwWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2PQUJENrAGQAKCRBJ hJEKjKruivm1B/0VfEuucLbqNfWd8YSXb48EXMuyTITilG+oEXJPIObpS792Ri0l/Ubglqoo WFwWeQWVwpa1T7gd0nMYtq8CjK5bRc5qyHjClugTBTFTjRrkv9LhVjR/q6KAzwDodv+CWGB8 Z6PWZOSf6k9zwIxZVHFKO5fLy4r5GIGsGW/Um0IKgPlR0Yqgt5GJS+KWD65EjDj3qQQgDmQJ xee9Nm61Lrzmd9WuJGJEXmyhkVgDjm06fRzoxrkWk/wg+9/Z0FjdXwhMrhC5diOqThIc090O 1Lom08xSSzv05fnwGUDy4FS7IgWTY1Qi9IOhhvtyvS2/1tureX3i0P9dFQ2q1OLgh0XFuQEN BFLYd6cBCACqVFQGWpmLMtfweSMsQE3JD2XDy4dyux7wcI3MNE9Qx1HcpbVRAm7YB+bDbi/r wyV+92x9SvLlviOj7iRTi297xInn7dB9Q9agYcNTQbKX8QKia3Ie6bgbKpSgeeZ9RgQLh7sm DJukqFWofL0PfsB4u+Z7ISWIio7ZS+jRaReyz9Ol2rCyxUmqYzztA3b5yeviY8uAiUrEJTx5 15TMNHf3Wx+Oa/iJROXWTaSZgRhA8z+l1+v29VVPoHU592kIE1fkxrWYFDmkvnnpdQ3gKkKe /OzjMWb8Ildz/YZB8PheLrA8cA60LeeQr/INzD02pbFyA3YuBa7n9FLNLJ4mfAnhABEBAAGJ ATwEGAEKACYCGwwWIQT4WjggQ2n3GjTUToNJhJEKjKruigUCXeL2PQUJENrAGQAKCRBJhJEK jKruivm1B/0VfEuucLbqNfWd8YSXb48EXMuyTITilG+oEXJPIObpS792Ri0l/UbglqooWFwW eQWVwpa1T7gd0nMYtq8CjK5bRc5qyHjClugTBTFTjRrkv9LhVjR/q6KAzwDodv+CWGB8Z6PW ZOSf6k9zwIxZVHFKO5fLy4r5GIGsGW/Um0IKgPlR0Yqgt5GJS+KWD65EjDj3qQQgDmQJxee9 Nm61Lrzmd9WuJGJEXmyhkVgDjm06fRzoxrkWk/wg+9/Z0FjdXwhMrhC5diOqThIc090O1Lom 08xSSzv05fnwGUDy4FS7IgWTY1Qi9IOhhvtyvS2/1tureX3i0P9dFQ2q1OLgh0XF Organization: The FreeBSD Project Message-ID: Date: Fri, 28 Feb 2020 10:39:26 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qrNmNCjXwFmmeToMAo61h8GL3XdXiU5U5" X-Rspamd-Queue-Id: 48TdcK24Vbz4Q8l X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=kitchenlab.org header.s=google header.b=PsIHTMaY; dmarc=none; spf=pass (mx1.freebsd.org: domain of bmah@kitchenlab.org designates 2607:f8b0:4864:20::541 as permitted sender) smtp.mailfrom=bmah@kitchenlab.org X-Spamd-Result: default: False [-4.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[kitchenlab.org:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-drivers@freebsd.org]; DMARC_NA(0.00)[kitchenlab.org]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_TO(1.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[kitchenlab.org:+]; RCVD_IN_DNSWL_NONE(0.00)[1.4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_THREE(0.00)[3]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; IP_SCORE(-0.84)[ip: (-0.63), ipnet: 2607:f8b0::/32(-1.88), asn: 15169(-1.67), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 18:39:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qrNmNCjXwFmmeToMAo61h8GL3XdXiU5U5 Content-Type: multipart/mixed; boundary="pKTErZ1aK2VLvBLO35ATmgxOt2uY6UAa4" --pKTErZ1aK2VLvBLO35ATmgxOt2uY6UAa4 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable [Resending with a From: address that hopefully works better.] If memory serves me right, Daniel Ebdrup Jensen wrote: > Yes, iperf3 will default to single-threaded packet generation, et al. w= hich > favours fast cores with frequency boosting facilities. > You might want to use iperf2 as that's properly multi-threaded, or you = can > use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. While it's true that iperf3 is single-threaded, it should be capable of saturating a 10GE link with a single TCP connection, given proper command-line arguments (in particular, specifying a sufficiently large socket-buffer size with the -w option). But based on the symptom of packet loss, I'd say the single-threaded vs. multi-threaded argument might not be relevant to the problem that the OP has. Bruce. > On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky > wrote: >=20 >> On 2020-02-28 10:03, Rajesh Kumar wrote: >>> Hi FreeBSD team, >>> >>> I am writing a network driver using iflib framework and using "iperf3= " >> tool >>> for performance testing. >>> >> >> Is there any difference with "iperf" tool and using multiple threads? = I >> think iperf3 is single threaded ??? >> >> --HPS >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.= org" >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 --pKTErZ1aK2VLvBLO35ATmgxOt2uY6UAa4-- --qrNmNCjXwFmmeToMAo61h8GL3XdXiU5U5 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCgAdFiEE+Fo4IENp9xo01E6DSYSRCoyq7ooFAl5ZXl4ACgkQSYSRCoyq 7oqhsggAq+89fH/jmgMNHr8Um2yKdWKrojVtsuvbTBZJfeQA61JXJ1g+Rf3SO5Fy uIqoH0nN0UL4UB2pc5JPV/CxuO8RagZwzDXD6fcJxjdM7FD3CE/EkbEdFe+T+wKH PTkSC9dLFzPNQypV8Lzfd9lWOhJYIoOgoCDTYxrYa1Mv4NnxU5P8BvAeo2peKD4G mae8dGfQICnjTEI0rIYCy0pn1zugICpTZ/6Xe3mMN4+jKCGI4Rm1wiULfeGYC+TZ rwfMGlmnSse8ZukCvd3/T3r7IQ5BTEfcd/nhDp7mcJ4YySxHyqKw9pu4V7n01vy9 +TOq6qBOY+r9l7HPaU08priiayQOZA== =Wg/V -----END PGP SIGNATURE----- --qrNmNCjXwFmmeToMAo61h8GL3XdXiU5U5-- From owner-freebsd-drivers@freebsd.org Fri Feb 28 22:17:18 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6989C247719 for ; Fri, 28 Feb 2020 22:17:18 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TkRf0bvcz4H8P for ; Fri, 28 Feb 2020 22:17:18 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: debdrup) by smtp.freebsd.org (Postfix) with ESMTPSA id B58B5567D for ; Fri, 28 Feb 2020 22:17:17 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: by mail-oi1-f177.google.com with SMTP id l12so4375923oil.9 for ; Fri, 28 Feb 2020 14:17:17 -0800 (PST) X-Gm-Message-State: APjAAAUknlHV/B7OEHzRkhIs6u9miFW5PIp83FKyvis6tS2ZCyc7LdI3 WvHMSWOH8NO/zNqj2RHCE6X6wJXjxIgbJpDLbmN14A== X-Google-Smtp-Source: APXvYqy0abzTF+dPdXim9UuaxsOpYqLukL/Fq3LKhgjx9WBuQpCtBVfC0l8lj9eJ/ZWkHa2Sb+bNT46ezjZ6SPmqpeU= X-Received: by 2002:a05:6808:aba:: with SMTP id r26mr4520520oij.4.1582928236971; Fri, 28 Feb 2020 14:17:16 -0800 (PST) MIME-Version: 1.0 References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> In-Reply-To: From: Daniel Ebdrup Jensen Date: Fri, 28 Feb 2020 23:17:03 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: freebsd-drivers@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 22:17:18 -0000 On Fri, Feb 28, 2020 at 7:39 PM Bruce A. Mah wrote: > [Resending with a From: address that hopefully works better.] > > If memory serves me right, Daniel Ebdrup Jensen wrote: > > Yes, iperf3 will default to single-threaded packet generation, et al. > which > > favours fast cores with frequency boosting facilities. > > You might want to use iperf2 as that's properly multi-threaded, or you > can > > use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. > > While it's true that iperf3 is single-threaded, it should be capable of > saturating a 10GE link with a single TCP connection, given proper > command-line arguments (in particular, specifying a sufficiently large > socket-buffer size with the -w option). > > But based on the symptom of packet loss, I'd say the single-threaded vs. > multi-threaded argument might not be relevant to the problem that the OP > has. > > Bruce. > > > On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky > > wrote: > > > >> On 2020-02-28 10:03, Rajesh Kumar wrote: > >>> Hi FreeBSD team, > >>> > >>> I am writing a network driver using iflib framework and using "iperf3" > >> tool > >>> for performance testing. > >>> > >> > >> Is there any difference with "iperf" tool and using multiple threads? I > >> think iperf3 is single threaded ??? > >> > >> --HPS > >> > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > Oh, I didn't mean to imply that that wasn't part of the issue - I'm sorry if I made it sound like that. I was just confirming what Hans was asking, and possibly using the excuse to mention some things in base/ports that I think are also pretty neat. :) Also no longer top-posting, which was rather ghastly of me. I apologise.