From owner-freebsd-net@freebsd.org Mon May 10 14:10:24 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A41F063934D for ; Mon, 10 May 2021 14:10:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ff2y73SpRz3Qpw for ; Mon, 10 May 2021 14:10:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14AEAFWP025101 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 10 May 2021 17:10:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14AEAFWP025101 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14AEAF2R025100; Mon, 10 May 2021 17:10:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 10 May 2021 17:10:15 +0300 From: Konstantin Belousov To: Francois ten Krooden Cc: "freebsd-net@freebsd.org" Subject: Re: Vector Packet Processing (VPP) portability on FreeBSD Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Ff2y73SpRz3Qpw X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.09 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-0.999]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; NEURAL_SPAM_MEDIUM(0.91)[0.907]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-net]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 May 2021 14:10:24 -0000 On Mon, May 10, 2021 at 11:08:18AM +0000, Francois ten Krooden wrote: > Greetings > > We have a vested interest in high-speed IPsec VPN on FreeBSD. We have started with the porting of VPP (https://fd.io/) to FreeBSD. > > Currently we have VPP compiled and running with netmap. The speeds we measure are nowhere near the performance of a 10Gbps link, at around 350kpps for 1500 byte IPv4 packets. We suspect the biggest issue is related to how VPP implements huge pages (Linux) and our modifications to support super pages on FreeBSD. > > Apart from the above, there are remaining issues we need to sort out and "Linuxisms" that need porting to FreeBSD, but this is going reasonably well. We are working in a public Github repository and have started listing our issues there alongside the code. Our main working branch is "freebsd" (https://github.com/ftk-ntq/vpp/tree/freebsd). > > Our aim with this mail is to get the discussion started on porting VPP to FreeBSD and to invite interested parties to help with the effort. We intend to upstream the work hoping that the original authors will adopt our ported code and continue maintaining future compatibility with FreeBSD. > > Some of our questions or comments to start the conversation: > 1. netmap vs. DPDK (VPP relies on DPDK by default with the netmap integration deprecated). Which will be the best to choose? > 2. How to correctly implement using super pages / huge pages in FreeBSD in order to allow VPP to allocate contiguous memory blocks for packet buffers to process packets from the packet handling framework (netmap/DPDK)? Superpage CPU mappings, as opposed to small pages mappings, typically give you several units of percents improvement in best setup. Having large page support in hardware for things like memory registration/rings mapping could give you some more if hardware DMA engine is limited in capacity, e.g. have fixed number of descriptors. To allocate physically-contiguous superpage-mapped regions in userspace, create special posix shmfd with shm_create_largepage(), then map it with mmap(2). The backing pages are allocated on creation, so you can e.g. pass them to hardware without mmaping into userspace at all, if needed/preferred. > 3. What are suitable alternatives for reading information from procfs and sysfs on FreeBSD? Understand what information is obtained, then what for is it actually used, then match it against equivalent FreeBSD approach, then gather the required information. > 4. Functionality relying on Linux epoll is currently supported using epoll-shim. Is this the correct approach? > > Any help and input to aid in the effort will be greatly appreciated. > > Regards > > Francois ten Krooden > > > Important Notice: > > This e-mail and its contents are subject to the Nanoteq (Pty) Ltd e-mail legal notice available at: > http://www.nanoteq.com/AboutUs/EmailDisclaimer.aspx > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"