From owner-freebsd-drivers@freebsd.org Sun Mar 1 23:08:06 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 38C3C2516F1; Sun, 1 Mar 2020 23:08:06 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48VzTH5nHWz4CBn; Sun, 1 Mar 2020 23:08:03 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wm1-x332.google.com with SMTP id 6so709247wmi.5; Sun, 01 Mar 2020 15:08:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TfanTrN2Blo9Kozjc1qqPPtADXhQbomL9tGQ1PXhYNY=; b=Few4htTciWeYGC5FU34iyc/ZxqBOEGn/KhXeeuiNj1NGHl++zqmD7gTHPBmRG34IwM PyQ70ZZxloyBRbYtl6L6FvzRuT+lCpPmd+vw4wQ84PMVrfnN7gYCtF2IQLfy78YFs1Ig 0WFL6JdooZINdMALI6CBG2+l2ZveLsHWvHlgQqEXI3JmmNhTkL27EheUpkep/xARb8Ij 2PqGTZSDSiwLheOyG5wlyi+4v6Hl3oH0+aZh2NPVIf3kun02m6RuBzjW1knpdBUK4jva 6aAuj7K51tl8lZxNglMJk+Bmw2NuppLGil+GWd9wRkrr8I1M9ypZg9ywgS4ajkAkoCHx c8qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TfanTrN2Blo9Kozjc1qqPPtADXhQbomL9tGQ1PXhYNY=; b=AxFH28x1kf6cazOfLKrquc5zty5/GpL/yTkcPZdLNBzUf6OdJMtfLfsjipilWL4IhS WKQT2J5CjGtmHqh10HHjKM9QG9xLbZNbmskRbOXIEyYdufyvGX5ZziYIpE3x/3ZAen14 xLv78ixoHTOVWDHJkV/qPT+75sL9fluj4xbiMrurSuXXoixbdvWZHrePr+ZoHVPPwPmc 3HDBpHIMyjYhWWziyANyyi9xU6du49qUOPdZBs/dtIQ9QRRAWsXSCS7EltxyQjgUX0ax ly+lFcUPrqRTarF0OmJ/kghMBSnJ4bLZFiKIQYYGPcHz1NhT6JQi5cb4Qn7VBV+tAqf1 xZJw== X-Gm-Message-State: APjAAAWmVf7vn828aL9dTuf85+x3dq+YW+zDgEL5mscHeA5akU4t4xft z7Q+GukqeNmjw05T3MHrCXAyrrsg4avM/3JjUWIZWA== X-Google-Smtp-Source: APXvYqyHpm46QhtP6+AlsSJEN/Mx8X57UcPTO5tjMeVMghFxdOAZ5KPUsOQHuWld29UjleTNpMh1dIzxXKodQYKhaPw= X-Received: by 2002:a1c:6a15:: with SMTP id f21mr15676114wmc.126.1583104079815; Sun, 01 Mar 2020 15:07:59 -0800 (PST) MIME-Version: 1.0 References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> In-Reply-To: From: Rajesh Kumar Date: Mon, 2 Mar 2020 04:37:48 +0530 Message-ID: Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: Daniel Ebdrup Jensen Cc: freebsd-drivers@freebsd.org, FreeBSD Hackers X-Rspamd-Queue-Id: 48VzTH5nHWz4CBn X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Few4htTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::332 as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[9]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (-9.06), ipnet: 2a00:1450::/32(-2.41), asn: 15169(-1.67), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2020 23:08:06 -0000 Hi Guys, Thanks for your time in responding. As you people stated, the problem here doesn't seem to be single threaded vs multi threaded. I am experience similar behavior with "iperf". It's mostly to do with iflib and my driver(mostly my driver). It looks like the receiver is slow in reading the packets than the sender sending the packets. In this case, iflib drives the packet read through the exposed interfaces (rxd_available, rxd_pkt_get etc.,). So, how to make the receiver side read the packet faster with iflib? Is there anything that I should take care in my driver in this regard? Thanks, Rajesh. On Sat, Feb 29, 2020 at 3:47 AM Daniel Ebdrup Jensen wrote: > On Fri, Feb 28, 2020 at 7:39 PM Bruce A. Mah wrote: > > > [Resending with a From: address that hopefully works better.] > > > > If memory serves me right, Daniel Ebdrup Jensen wrote: > > > Yes, iperf3 will default to single-threaded packet generation, et al. > > which > > > favours fast cores with frequency boosting facilities. > > > You might want to use iperf2 as that's properly multi-threaded, or you > > can > > > use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. > > > > While it's true that iperf3 is single-threaded, it should be capable of > > saturating a 10GE link with a single TCP connection, given proper > > command-line arguments (in particular, specifying a sufficiently large > > socket-buffer size with the -w option). > > > > But based on the symptom of packet loss, I'd say the single-threaded vs. > > multi-threaded argument might not be relevant to the problem that the OP > > has. > > > > Bruce. > > > > > On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky > > > wrote: > > > > > >> On 2020-02-28 10:03, Rajesh Kumar wrote: > > >>> Hi FreeBSD team, > > >>> > > >>> I am writing a network driver using iflib framework and using > "iperf3" > > >> tool > > >>> for performance testing. > > >>> > > >> > > >> Is there any difference with "iperf" tool and using multiple threads? > I > > >> think iperf3 is single threaded ??? > > >> > > >> --HPS > > >> > > >> _______________________________________________ > > >> freebsd-hackers@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > >> To unsubscribe, send any mail to " > > freebsd-hackers-unsubscribe@freebsd.org" > > >> > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to " > > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > Oh, I didn't mean to imply that that wasn't part of the issue - I'm sorry > if I made it sound like that. > I was just confirming what Hans was asking, and possibly using the excuse > to mention some things in base/ports that I think are also pretty neat. :) > > Also no longer top-posting, which was rather ghastly of me. I apologise. > _______________________________________________ > freebsd-drivers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-drivers > To unsubscribe, send any mail to "freebsd-drivers-unsubscribe@freebsd.org" >