From nobody Thu Dec 30 19:55:17 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4C1D0192B3EE for ; Thu, 30 Dec 2021 19:55:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JPzXF1QgMz3jDJ for ; Thu, 30 Dec 2021 19:55:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1BUJtH5F073881 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 30 Dec 2021 21:55:20 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1BUJtH5F073881 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1BUJtHD3073879 for stable@freebsd.org; Thu, 30 Dec 2021 21:55:17 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 30 Dec 2021 21:55:17 +0200 From: Konstantin Belousov To: stable@freebsd.org Subject: Re: What is FreeBSD 12 equivalent to Linux' "sysctl -w vm.nr_hugepages=1280"? Message-ID: References: <431b35b0-7745-d2d7-ab04-bbf76e58608a@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4JPzXF1QgMz3jDJ X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [0.98 / 15.00]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.76)[0.757]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.996]; NEURAL_SPAM_LONG(0.22)[0.219]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Thu, Dec 30, 2021 at 08:15:18PM +0100, Christoph Moench-Tegeder wrote: > Hi, > > ## Konstantin Belousov (kostikbel@gmail.com): > > > > There is no equivalent setting on FreeBSD (as of yet), and there's > > > no way to explicitely request huge pages (super pages, large pages, > > > whatever you call them) on FreeBSD in the mmap()/shmget() interfaces > > > (as there is in Linux) (there is MAP_ALIGNED_SUPER in FreeBSD to > > > "maximize the potential use of large (“super”) pages", but from the > > > desription alone that looks like slightly different semantics - and > > > based on how I read those sementics, that randomX code in question > > > should either have a fallback or avoid large pages on any BSD). > > > > I have no idea what the referenced setting does on Linux. Reply > > below is about 'explicitly request large pages'. This is not true. > > I was talking about "via mmap()/shmget() as in Linux", where you can > request large ("huge") pages by setting MAP_HUGETLB/SHM_HUGETLB in > the flags argument. We do not have that flag value. > > > There is shm_create_largepages() function which returns fd referencing > > shm segment. This segment mappings are guaranteed to be backed by large > > pages of the requested size. More, you cannot split them into mappings > > backed by smaller pages, so e.g. mprotect() or munmap() would only work > > on the chosen superpage size boundary. > > Hey, that's a great thing to have. Is that supposed to be part of > our official interface? I can't find any manpage on that. > I only knew about MAP_ALIGNED_SUPER and memfd_create() MFD_HUGETLB > "This flag is currently unsupported". It already is. The only known issue is that the largepages stuff is not documented. MFD_HUGETLB is emulated using shm largepages stuff. > > On Linux, the sysctl vm.nr_hugepages sets the minimun size of the > huge page pool, that is the number of pre-reserved pages of the > default huge page size (2MB on x86-like), and those pages will > be used for {MAP,SHM}_HUGETLB allocations - they are not used for > "normal" usage, that is, if you set nr_hugepages too high, you will > be out of memory very quickly. That thing is not to be confused We do not reserve anything. Largepage shm needs to be truncated to desired size before mapping, at which time the contigous physical memory chunks of right alignment are allocated. There are policy flags SHM_LARGEPAGE_ALLOC_XXX which specify how hard kernel should try to defrag if there is no suitable contigous memory. > with Linux Transparent Huge Pages mechanism (background compaction/ > migration etc.). This should be similar to our transparent superpages, which has no relations to largepage stuff. Same as in Linux, I suppose. > As you're working in that area - did you see "A Comprehensive Analysis > of Superpage Management Mechanisms and Policies" > https://www.usenix.org/conference/atc20/presentation/zhu-weixi ? I remember seeing this paper some time ago.