From owner-freebsd-net@freebsd.org Tue May 11 12:43:28 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E47CC6368B5 for ; Tue, 11 May 2021 12:43:28 +0000 (UTC) (envelope-from ftk@nanoteq.com) Received: from mailguard.liquidtelecom.co.za (delivery.mailguard.neotel.co.za [41.168.2.25]) by mx1.freebsd.org (Postfix) with ESMTP id 4FfczK3yTBz3rXm for ; Tue, 11 May 2021 12:43:25 +0000 (UTC) (envelope-from ftk@nanoteq.com) Received: from SEC-NGP-AG07 ([192.168.202.37]) by mailguard.liquidtelecom.co.za with Microsoft SMTPSVC(7.5.7601.17514); Tue, 11 May 2021 14:42:54 +0200 Received: from sec-ngp-spt05.e-purifier.com ([192.168.201.1]) by SEC-NGP-AG07.neotel.e-purifier.co.za with Microsoft SMTPSVC(7.5.7601.17514); Tue, 11 May 2021 14:42:50 +0200 Received: from localhost (localhost [127.0.0.1]) by sec-ngp-spt05.e-purifier.com (Postfix) with ESMTP id CB39FE3CCD1; Tue, 11 May 2021 14:43:17 +0200 (SAST) X-Virus-Scanned: by SpamTitan at e-purifier.com Received: from sec-ngp-spt05.e-purifier.com (localhost [127.0.0.1]) by sec-ngp-spt05.e-purifier.com (Postfix) with ESMTP id 0E644E3CCC7; Tue, 11 May 2021 14:43:12 +0200 (SAST) Received: from NTQ-EXC.nanoteq.co.za (unknown [41.170.5.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sec-ngp-spt05.e-purifier.com (Postfix) with ESMTPS id 0433CE3CCB8; Tue, 11 May 2021 14:43:12 +0200 (SAST) Received: from NTQ-EXC.nanoteq.co.za ([fe80::a5b3:4700:5af3:78b2]) by NTQ-EXC.nanoteq.co.za ([fe80::a5b3:4700:5af3:78b2%12]) with mapi id 14.03.0513.000; Tue, 11 May 2021 14:43:11 +0200 From: Francois ten Krooden To: Konstantin Belousov CC: "freebsd-net@freebsd.org" Subject: RE: Vector Packet Processing (VPP) portability on FreeBSD Thread-Topic: Vector Packet Processing (VPP) portability on FreeBSD Thread-Index: AQHXRaZHkx80/avbnEmSwhKdxlugfareNk8w Date: Tue, 11 May 2021 12:43:10 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US, en-ZA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 11 May 2021 12:42:50.0267 (UTC) FILETIME=[271D7EB0:01D74663] x-archived: yes x-dbused: RGF0YSBTb3VyY2U9MTkyLjE2OC4yMDEuMjc= X-Rspamd-Queue-Id: 4FfczK3yTBz3rXm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of ftk@nanoteq.com has no SPF policy when checking 41.168.2.25) smtp.mailfrom=ftk@nanoteq.com X-Spamd-Result: default: False [-2.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[41.168.2.25:from]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[Nanoteq.com]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[41.168.2.25:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:36937, ipnet:41.168.0.0/17, country:ZA]; RCVD_COUNT_SEVEN(0.00)[7]; MAILMAN_DEST(0.00)[freebsd-net]; RCVD_IN_DNSWL_LOW(-0.10)[41.168.2.25:from] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 May 2021 12:43:28 -0000 On Monday, 10 May 2021 16:10 Konstantin Belousov wrote: > On Mon, May 10, 2021 at 11:08:18AM +0000, Francois ten Krooden wrote: > > Greetings > > > > We have a vested interest in high-speed IPsec VPN on FreeBSD. We have > started with the porting of VPP (https://fd.io/) to FreeBSD. > > > > Currently we have VPP compiled and running with netmap. The speeds we > measure are nowhere near the performance of a 10Gbps link, at around > 350kpps for 1500 byte IPv4 packets. We suspect the biggest issue is relat= ed > to how VPP implements huge pages (Linux) and our modifications to support > super pages on FreeBSD. > > > > Apart from the above, there are remaining issues we need to sort out an= d > "Linuxisms" that need porting to FreeBSD, but this is going reasonably we= ll. > We are working in a public Github repository and have started listing our > issues there alongside the code. Our main working branch is "freebsd" > (https://github.com/ftk-ntq/vpp/tree/freebsd). > > > > Our aim with this mail is to get the discussion started on porting VPP = to > FreeBSD and to invite interested parties to help with the effort. We inte= nd > to upstream the work hoping that the original authors will adopt our port= ed > code and continue maintaining future compatibility with FreeBSD. > > > > Some of our questions or comments to start the conversation: > > 1. netmap vs. DPDK (VPP relies on DPDK by default with the netmap > integration deprecated). Which will be the best to choose? > > 2. How to correctly implement using super pages / huge pages in FreeBSD > in order to allow VPP to allocate contiguous memory blocks for packet > buffers to process packets from the packet handling framework > (netmap/DPDK)? > Superpage CPU mappings, as opposed to small pages mappings, typically > give you several units of percents improvement in best setup. Having larg= e > page support in hardware for things like memory registration/rings mappin= g > could give you some more if hardware DMA engine is limited in capacity, > e.g. have fixed number of descriptors. > > To allocate physically-contiguous superpage-mapped regions in userspace, > create special posix shmfd with shm_create_largepage(), then map it with > mmap(2). The backing pages are allocated on creation, so you can e.g. pa= ss > them to hardware without mmaping into userspace at all, if > needed/preferred. Thank you I will have a look at this as well. I knew about memfd_create wh= ich was also added in FreeBSD 13.0 which I did use. > > > 3. What are suitable alternatives for reading information from procfs a= nd > sysfs on FreeBSD? > Understand what information is obtained, then what for is it actually use= d, > then match it against equivalent FreeBSD approach, then gather the > required information. Thank you. This was basically what we suspected. One of the ones we are unsure about is what the equivalent of /proc/self/pa= gemap on Linux would be. The one idea we had is using procstat_getvmmap from libprocstat, but haven'= t finished investigating yet. Regards Francois > > > 4. Functionality relying on Linux epoll is currently supported using ep= oll- > shim. Is this the correct approach? > > > > Any help and input to aid in the effort will be greatly appreciated. > > > > Regards > > > > Francois ten Krooden > > > > > > Important Notice: > > > > This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mai= l > legal notice available at: > > http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Important Notice: This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail le= gal notice available at: http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx