From owner-freebsd-current@freebsd.org Tue Apr 21 17:07:10 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6C39F2B040D for ; Tue, 21 Apr 2020 17:07:10 +0000 (UTC) (envelope-from moremore2@outlook.com) Received: from APC01-HK2-obe.outbound.protection.outlook.com (mail-oln040092255045.outbound.protection.outlook.com [40.92.255.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49693H3998z4fxg for ; Tue, 21 Apr 2020 17:07:07 +0000 (UTC) (envelope-from moremore2@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E2QxYiIAgmIr+aLiiPNyw6+6Qq/imnQtjCoR4iQEk/oKRd/hi9zX6+rZnZUBtmf+vyTYyNNbE8varGULhIqDNxpRrnQEzwNTCUKRllTF7yOveSY0dQ6rGGl2S0JSYi0dVKVzYsMvTp+GAxhP/dUeClGnyZ3mARiq6aao4xAeUYzJEgcjQJGlJNUvkn8GakoxQTVFfpw6LCpU5ebHDjZLM78wPbyVkMsGMzxdq32D+6MNSFR1JhngGNGU8Fd+nHDJDIlOnMVGWEfHO3kzIe3paZ8E6hiU2PQbIckLQMPd+O+rJbizYFcLIHenAH5WyUoIPxRAW6HpaWeFKSU2HZfs1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/Ru+ZS8YL1M/PFpojqX+g48NOGtp6jOMqTYpZBqyC+k=; b=MAXPkZUe4OGDsitged2D+OXKx7YavGsqvtYKH3EH2nGlQDFd80FUNl6GbDy7PI+wWqCJ6N7Kp5aq1JBPw3W0jtM9DExMWzEybwt1RsQAPq7Nw1Kg6Uqo8h/XzY6mNORw1e5xhRINtD3v22p7NUuitdDLRx5ng+mRQuhnEhcwjLR3EhnO1HrIRaMtsb2lzUTbHuLGjbncVMOenfQgk/Y2uvN7h7Of5PXMPR2YEJUA90XnUiE08b+bLBAf7qqsrKgGWBct1R6AH8LvZ1rgwPb36dyMbC+/NfMStvOhYbv/dBOskx5QlWgD7hL924KYbqxglsJjL5A77ijB8Cyt5ax/bw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=outlook.com; dmarc=pass action=none header.from=outlook.com; dkim=pass header.d=outlook.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/Ru+ZS8YL1M/PFpojqX+g48NOGtp6jOMqTYpZBqyC+k=; b=Nh9RKnjQvHXVacb/sN2CpAaGO+9zSaCM+OfAYpYeLazMwStjIrUGnkwjc2QZZ/Xcs6EXbsAXCmzV7jcjahMMenufRj1blJyv0gDgM1c/Jj22YTvTvj33vYa9mfUqPsqDdjYcPWPJ6bVnSB6o3joU1WSOvDqcn5IV+2o2PzCFTlYVryd5/u12tJJ5PYzTMsowRiGn2rzBJCppRKynCgPko70JqhQjK2JSsGiH63dci4ppSjCVOKvB6mCGINf85k4pu7HjA7gYSzqEZYwa5e6ytvSUZUwJJ9+6+NqA1jk9zIFmVsvmaDJIZuU5x57a5fQsJ2KWAYo91aJrRErew6yLNQ== Received: from PU1APC01FT004.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebe::49) by PU1APC01HT034.eop-APC01.prod.protection.outlook.com (2a01:111:e400:7ebe::349) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.25; Tue, 21 Apr 2020 17:07:04 +0000 Received: from HK0PR03MB3026.apcprd03.prod.outlook.com (2a01:111:e400:7ebe::44) by PU1APC01FT004.mail.protection.outlook.com (2a01:111:e400:7ebe::98) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.25 via Frontend Transport; Tue, 21 Apr 2020 17:07:04 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:C03E965FAA0BB0294DABF4468B0137FCFE2556A5CE25CB87B22D50D9C5F33231; UpperCasedChecksum:8A821059A8AD9396F452F79DF638CD6033E97251C5A3EF0371B7117C15DE970E; SizeAsReceived:8862; Count:49 Received: from HK0PR03MB3026.apcprd03.prod.outlook.com ([fe80::8d52:228e:e18e:e7a7]) by HK0PR03MB3026.apcprd03.prod.outlook.com ([fe80::8d52:228e:e18e:e7a7%6]) with mapi id 15.20.2937.012; Tue, 21 Apr 2020 17:07:03 +0000 Subject: Re: CFT: if_bridge performance improvements To: freebsd-current@freebsd.org References: <5377E42E-4C01-4BCC-B934-011AC3448B54@FreeBSD.org> From: k simon Message-ID: Date: Wed, 22 Apr 2020 01:06:48 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: HK2PR0302CA0014.apcprd03.prod.outlook.com (2603:1096:202::24) To HK0PR03MB3026.apcprd03.prod.outlook.com (2603:1096:203:43::14) X-Microsoft-Original-Message-ID: <1a27b779-3fc5-63f6-0663-389f49369546@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kmatoMacBook-Pro.local (27.17.158.91) by HK2PR0302CA0014.apcprd03.prod.outlook.com (2603:1096:202::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.9 via Frontend Transport; Tue, 21 Apr 2020 17:07:02 +0000 X-Microsoft-Original-Message-ID: <1a27b779-3fc5-63f6-0663-389f49369546@outlook.com> X-TMN: [OqP1MT/GXm9gGJhRwIHRKUOI2K/TakK/] X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 49 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 75b8fa15-0719-4290-814b-08d7e6166944 X-MS-TrafficTypeDiagnostic: PU1APC01HT034: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zCee5JXn1Sz+cziNaQCv71wWc1ziK5ciwjXOWYDFFrNHgRDmRfPkp452Q1KKtwx0oJufoXOpa4oxc9+HcQ0vthxi823heC6llDv1gID39jCorRR97gzzgTWxOmR2L6hg7vuRu4L053R+xQoEnqfTcsMCcCbuxEnAZCVzIRfj1bDyYa5sfc2T3gxJrEX4Q+XPzOgTjPXmMafcnw/cXhTb8kLo4qtzVwdabzTUtfcqHoXlahWJWCwmqkto8B8z2UWZ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:0; SRV:; IPV:NLI; SFV:NSPM; H:HK0PR03MB3026.apcprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:; DIR:OUT; SFP:1901; X-MS-Exchange-AntiSpam-MessageData: QRDRx15vvY8dwl0Rp746BgaoV/tBZQ9VP9/sgL8YK36EblL7W88yne5Kcx3Ls+0po8H/6iUZVWYTH6/fmfX9w6ups+o+KH49fO6TEvy98jGz9Lpi3fTKAZva2prBLJ0d3iwNgHLjXpl7dhoKmRRouw== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 75b8fa15-0719-4290-814b-08d7e6166944 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2020 17:07:03.8203 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT034 X-Rspamd-Queue-Id: 49693H3998z4fxg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=outlook.com header.s=selector1 header.b=Nh9RKnjQ; dmarc=pass (policy=none) header.from=outlook.com; spf=pass (mx1.freebsd.org: domain of moremore2@outlook.com designates 40.92.255.45 as permitted sender) smtp.mailfrom=moremore2@outlook.com X-Spamd-Result: default: False [-1.50 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:40.92.0.0/15]; FREEMAIL_FROM(0.00)[outlook.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[outlook.com:+]; DMARC_POLICY_ALLOW(-0.50)[outlook.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[91.158.17.27.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FREEMAIL_ENVFROM(0.00)[outlook.com]; DWL_DNSWL_NONE(0.00)[outlook.com.dwl.dnswl.org : 127.0.3.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[outlook.com:s=selector1]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; ARC_ALLOW(-1.00)[i=1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ipnet: 40.64.0.0/10(-3.72), asn: 8075(-3.18), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[45.255.92.40.list.dnswl.org : 127.0.3.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[45.255.92.40.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Apr 2020 17:07:11 -0000 Hi, Interesting, maybe ng_eiface + if_bridge is a good idea . Simon 20200422 On 4/21/20 8:51 PM, Jan Bramkamp wrote: > On 16.04.20 08:34, Pavel Timofeev wrote: > >> вт, 14 апр. 2020 г., 12:51 Kristof Provost : >> >>> Hi, >>> >>> Thanks to support from The FreeBSD Foundation I’ve been able to work >>> on improving the throughput of if_bridge. >>> It changes the (data path) locking to use the NET_EPOCH infrastructure. >>> Benchmarking shows substantial improvements (x5 in test setups). >>> >>> This work is ready for wider testing now. >>> >>> It’s under review here: https://reviews.freebsd.org/D24250 >>> >>> Patch for CURRENT: https://reviews.freebsd.org/D24250?download=true >>> Patches for stable/12: >>> https://people.freebsd.org/~kp/if_bridge/stable_12/ >>> >>> I’m not currently aware of any panics or issues resulting from these >>> patches. >>> >>> Do note that if you run a Bhyve + tap on bridges setup the tap code >>> suffers from a similar bottleneck and you will likely not see major >>> improvements in single VM to host throughput. I would expect, but have >>> not tested, improvements in overall throughput (i.e. when multiple VMs >>> send traffic at the same time). >>> >>> Best regards, >>> Kristof >>> >> Hi! >> Thank you for your work! >> Do you know if epair suffers from the same issue as tap? > > As Kirstof Provost said if_epair locks has per CPU locks, but a problem > exists a layer about the epair driver. At leas on FreeBSD 12.0 and 12.1 > all the packet processing happens in a single netisr thread that becomes > CPU bound and limits how fast useful traffic can move through epair > interfaces. Afaik TCP doesn't benifit from multiple netisr threads, but > unorderer protocols (e.g. UDP) could profit from multiple threads. > > > I have only tested with iperf (using multiple connections) between the > FreeBSD 12.x host and a vnet enabled jail connected via an epair > interface and maxed out at about 1-2Gb/s depending on the CPUs single > threaded throughput. > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"