From nobody Mon Apr 22 16:25:25 2024 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNVwQ1J2Yz5HQ7n for ; Mon, 22 Apr 2024 16:25:26 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNVwQ0pDVz49dT; Mon, 22 Apr 2024 16:25:26 +0000 (UTC) (envelope-from brooks@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713803126; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=2QmdfZm7nnXqC12aZiT5509QcPkuPu2Hzzmjmmc0Iis=; b=FH5j/XpYffuzbWXb5fLb+8AXF4MOvExOAbL+sABME2e0byGGYfQWjwP+lnsORpqRGbFRzp KkL2dFNyGVIMFkSwebiOYaNcziRhuMuwhKw0m+jT4CGJS9vZ9EtlOlBQkllixEgMd1c1tv wFvI9uBEisqYBNzFsACuWgAFKobnTZV4DQyIboDu6QmD9eqPyz9QaNyhhAokzwqm3TqE5h YFpkkyO5YKOLXuY7i7vquF4T7kiuiKscU31NWg2MWyEeLK0yTLu/bid2nEt0mKKXsuSSkY QVTzaTc0Am4qK+t+ogqmTHiyEUxP6DI3RB7Cx6lEPhLeIh3+hpa08F7IHmk26g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713803126; a=rsa-sha256; cv=none; b=Qr2xhqZNcbPIOW6TasLx0KyRRuwm1Nt7Ds/alcZwKo01ckDjjiJikzyvlTlCGAZ5KrpKNX pqYaAsILx7NB87Ht4SIbGjLhqol2ISj5hYi0EA3J0eEPS2sd5S5x0MymCLSec01oXx90RS Vp6mRxkHNSZFXK+fHcP7/wE5a2x3vlnu4mTuvKbpRRow3G3z79wK6iGFs6bP1+IX7qTWsX oRRVjWOKw/K12d8yXoK1e0vY4C+rfx7Pod6wQI0uRaaup6dpxyfD4kHNNOo7doM20eej+l GVq19eyxnkXtMWvCNyScVc138uwupFFVOeoj1S/pkuc9V60zthcpmV4hO6KJlw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713803126; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=2QmdfZm7nnXqC12aZiT5509QcPkuPu2Hzzmjmmc0Iis=; b=CS6pdveB7x1cFyVfe2AfiqSJddlVRNiYFNlyZl59bNR0pTl9EW1L5CEdsE0GbqE3sKcsDl jfsX5wSVzKQx8e2LvWSq2ulTU7J8mt81vO45NSPEXzXfi7G61e6SoiuUO7KZfRuH60PPax 7vMZlyZKFu2MiWv9RsDFqXPnCcbf0RU4wXYVStuJGz6q16NvRRht2IMYuGmOHf5+lpIg6o Y8F3EB9vXCkLR8ptt/c29aRpX254odwQ5Q9C7Dwrz5MVAdAM63o9bNN7WEFBoUY+9fxBAL EWcbBU7nNjizAd3DKCmY3XyN2Opa0oRPq03KgKuiPp3RPu7qJ3VW267X1A4QlQ== Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: brooks/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNVwQ09R0z1PKx; Mon, 22 Apr 2024 16:25:25 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 4582D3C019B; Mon, 22 Apr 2024 16:25:25 +0000 (UTC) Date: Mon, 22 Apr 2024 16:25:25 +0000 From: Brooks Davis To: Lexi Winter Cc: hackers@freebsd.org Subject: Re: building memstick without root Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Sun, Apr 21, 2024 at 07:45:22PM +0100, Lexi Winter wrote: > hello, > > is it expected that running 'make -C release memstick' requires root? > the build fails for me with: > > install: /src/obj/src/freebsd/src/main/riscv.riscv64/release/dist/kernel/boot/kernel/kernel: chown/chgrp: Operation not permitted > > as i understand it, makefs should be able to build images as a non-root > user using mtree, but i'm not sure if this is hooked up to the build > system, or if i'm doing something else wrong - maybe i need a > make.conf/src.conf option set? You'll need to define NO_ROOT. I think that's sufficent, but I'm not 100% certain. We build CheriBSD releases without root on Linux hosts. -- Brooks From nobody Mon Apr 22 16:28:24 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNW044khTz5HQV1 for ; Mon, 22 Apr 2024 16:28:36 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNW0357C8z4CwS for ; Mon, 22 Apr 2024 16:28:35 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jeffpc@josefsipek.net designates 71.174.62.3 as permitted sender) smtp.mailfrom=jeffpc@josefsipek.net Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 90CFB2C25A; Mon, 22 Apr 2024 16:28:29 +0000 (UTC) Date: Mon, 22 Apr 2024 12:28:24 -0400 From: Josef 'Jeff' Sipek To: Warner Losh Cc: FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.25 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.65)[-0.646]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_TWO(0.00)[2]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; TO_DN_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[josefsipek.net]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VNW0357C8z4CwS On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: ... > ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required by /lib/libc++.so.1 not found So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by 'make installworld' because the newly built library ends up in /usr/lib [1]. Therefore, the broken system has: /lib/libcxxrt.so.1 from 14.0-RELEASE /usr/lib/libcxxrt.so.1 from -CURRENT ld-elf.so finds the one in /lib fails to find the required version (CXXABI_1.3.11) and terminates. Workaround: # cp /usr/lib/libcxx.so.1 /lib Even on a working system (i.e., RELEASE -> STABLE -> CURRENT), there are two libs of different vintages: root@odin:~ # ls -lh /lib/libcxxrt* /usr/lib/libcxxrt* -r--r--r-- 1 root wheel 105K Mar 23 08:48 /lib/libcxxrt.so.1 -r--r--r-- 1 root wheel 350K Apr 18 20:31 /usr/lib/libcxxrt.a lrwxr-xr-x 1 root wheel 13B Apr 18 20:31 /usr/lib/libcxxrt.so -> libcxxrt.so.1 -r--r--r-- 1 root wheel 104K Apr 18 20:31 /usr/lib/libcxxrt.so.1 I'm guessing that moving to 14-STABLE first updates the /lib copy to a version that includes CXXABI_1.3.11 which makes the subsequent -CURRENT happy enough. For completeness, a 14.0-RELEASE-p6 has: jeffpc@satis$ ls -lh /lib/libcxxrt.* /usr/lib/libcxxrt.* -r--r--r-- 1 root wheel 105K Jan 18 21:25 /lib/libcxxrt.so.1 -r--r--r-- 1 root wheel 354K Jan 18 21:27 /usr/lib/libcxxrt.a lrwxr-xr-x 1 root wheel 23B Apr 9 2021 /usr/lib/libcxxrt.so -> ../../lib/libcxxrt.so.1 I haven't looked at the makefiles to see where things go wrong - I'll try to take a look today after work (if nobody beats me to it). Jeff. [1] installworld invokes 'install -s -o root -g wheel -m 444 -C -S libcxxrt.so.1 /usr/lib' From nobody Mon Apr 22 16:46:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNWNR70w5z5HRsf for ; Mon, 22 Apr 2024 16:46:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f47.google.com (mail-ot1-f47.google.com [209.85.210.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNWNR0X3mz4Fv4 for ; Mon, 22 Apr 2024 16:46:15 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.47 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-ot1-f47.google.com with SMTP id 46e09a7af769-6eb848b5a2eso2641316a34.3 for ; Mon, 22 Apr 2024 09:46:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713804373; x=1714409173; h=content-transfer-encoding:cc:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mjSnNUo8zv8KKS/OfKqVcwzz/4nV5GVqMFFlin2VP2U=; b=qxXJecY9AhGGHm6/54DCaADBdgaSXwN5zY76ixUA0iXPgcLarkXnmgy/7i6e/lF9Cu xpXTrq4VFiN0CO1FDkFKg1AlqzeRYS6T374o0mYTO8TdlJ4dI4TAtyWxGC8iwMuRjvAA z8mELiQ2QFOrIe2YDgWpJELBKmkH8Ywvyb5aVYULP+RPnbw1gOsWn4HggKkdjEtsOz3J hZGLKiDkvC6Z5BNblMkPRrPJBvUbIbYcZN3NYa3gsz27qHHDeK8B79UsCub1Tn8f1hiF xIM1zeWlzuFI+lyl/m6qg8kFVhtsHrpgp1f7ewCQd6cPolI8LqeVFLYQ9DjiTGfdvDHD ZyeA== X-Forwarded-Encrypted: i=1; AJvYcCV6mbOaEfgXggZZ916AMHATypW6ane7dShZOsTmSfAdA6uUt2EYGRreFlYHJR+NL+p/CCht+93i7MuJzqtIvSStZnI2gSk1HKk3Kh8= X-Gm-Message-State: AOJu0YxsuOdX3Tsup3BzdPSbgU1To0JVMl9nl5kFaBzQuxwG4E6Pp8nh tMIC7q1wKybNy1Ww/KKoWvUAqtnE9fhONZ7y1DauuAUhiN5EGw4MkyH7QNarLG4q5bB+mUZqjab 8Sp/orp/vVP6mYwBnQ0tYMGs+aobZjVct X-Received: by 2002:a05:6358:ed11:b0:183:86c4:7603 with SMTP id hy17-20020a056358ed1100b0018386c47603mt14005103rwb.15.1713804373501; Mon, 22 Apr 2024 09:46:13 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Mon, 22 Apr 2024 10:46:01 -0600 Message-ID: Subject: Re: Stressing malloc(9) Cc: Mark Johnston , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: + X-Spamd-Result: default: False [1.75 / 15.00]; MISSING_TO(2.00)[]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_SPAM_MEDIUM(0.40)[0.396]; NEURAL_SPAM_SHORT(0.35)[0.346]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RWL_MAILSPIKE_GOOD(-0.10)[209.85.210.47:from]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.47:from]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; R_DKIM_NA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VNWNR0X3mz4Fv4 On Sun, Apr 21, 2024 at 5:47=E2=80=AFPM Alan Somers w= rote: > > On Sun, Apr 21, 2024 at 10:09=E2=80=AFAM Mark Johnston wrote: > > > > On Sat, Apr 20, 2024 at 11:23:41AM -0600, Alan Somers wrote: > > > On Sat, Apr 20, 2024 at 9:07=E2=80=AFAM Mark Johnston wrote: > > > > > > > > On Fri, Apr 19, 2024 at 04:23:51PM -0600, Alan Somers wrote: > > > > > TLDR; > > > > > How can I create a workload that causes malloc(9)'s performance t= o plummet? > > > > > > > > > > Background: > > > > > I recently witnessed a performance problem on a production server= . > > > > > Overall throughput dropped by over 30x. dtrace showed that 60% o= f the > > > > > CPU time was dominated by lock_delay as called by three functions= : > > > > > printf (via ctl_worker_thread), g_eli_alloc_data, and > > > > > g_eli_write_done. One thing those three have in common is that t= hey > > > > > all use malloc(9). Fixing the problem was as simple as telling C= TL to > > > > > stop printing so many warnings, by tuning > > > > > kern.cam.ctl.time_io_secs=3D100000. > > > > > > > > > > But even with CTL quieted, dtrace still reports ~6% of the CPU cy= cles > > > > > in lock_delay via g_eli_alloc_data. So I believe that malloc is > > > > > limiting geli's performance. I would like to try replacing it wi= th > > > > > uma(9). > > > > > > > > What is the size of the allocations that g_eli_alloc_data() is doin= g? > > > > malloc() is a pretty thin layer over UMA for allocations <=3D 64KB. > > > > Larger allocations are handled by a different path (malloc_large()) > > > > which goes directly to the kmem_* allocator functions. Those funct= ions > > > > are very expensive: they're serialized by global locks and need to > > > > update the pmap (and perform TLB shootdowns when memory is freed). > > > > They're not meant to be used at a high rate. > > > > > > In my benchmarks so far, 512B. In the real application the size is > > > mostly between 4k and 16k, and it's always a multiple of 4k. But it's > > > sometimes great enough to use malloc_large, and it's those > > > malloc_large calls that account for the majority of the time spent in > > > g_eli_alloc_data. lockstat shows that malloc_large, as called by > > > g_elI_alloc_data, sometimes blocks for multiple ms. > > > > > > But oddly, if I change the parameters so that g_eli_alloc_data > > > allocates 128kB, I still don't see malloc_large getting called. And > > > both dtrace and vmstat show that malloc is mostly operating on 512B > > > allocations. But dtrace does confirm that g_eli_alloc_data is being > > > called with 128kB arguments. Maybe something is getting inlined? > > > > malloc_large() is annotated __noinline, for what it's worth. > > > > > I > > > don't understand how this is happening. I could probably figure out > > > if I recompile with some extra SDT probes, though. > > > > What is g_eli_alloc_sz on your system? > > 33kiB. That's larger than I expected. When I use a larger blocksize > in my benchmark, then I do indeed see malloc_large activity, and 11% > of the CPU is spend in g_eli_alloc_data. > > I guess I'll add some UMA zones for this purpose. I'll try 256k and > 512k zones, rounding up allocations as necessary. Thanks for the tip. When I said "33kiB" I meant "33 pages", or 132 kB. And the solution turns out to be very easy. Since I'm using ZFS on top of geli, with the default recsize of 128kB, I'll just set vfs.zfs.vdev.aggregation_limit to 128 kB. That way geli will never need to allocate more than 128kB contiguously. ZFS doesn't even need those big allocations to be contiguous; it's just aggregating smaller operations to reduce disk IOPs. But aggregating up to 1MB (the default) is overkill; any rotating HDD should easily be able to max out its consecutive write IOPs with 128kB operation size. I'll add a read-only sysctl for g_eli_alloc_sz too. Thanks Mark. -Alan From nobody Mon Apr 22 17:06:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNWsT4jNCz5HTgX for ; Mon, 22 Apr 2024 17:07:57 +0000 (UTC) (envelope-from kpn@neutralgood.org) Received: from gunsight1.NeutralGood.ORG (gunsight1.neutralgood.org [206.196.19.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gunsight1.neutralgood.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNWsS74ftz4JjW for ; Mon, 22 Apr 2024 17:07:56 +0000 (UTC) (envelope-from kpn@neutralgood.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of kpn@neutralgood.org designates 206.196.19.100 as permitted sender) smtp.mailfrom=kpn@neutralgood.org Received: from gunsight1.neutralgood.org (localhost [127.0.0.1]) by gunsight1.NeutralGood.ORG (8.17.1/8.17.1) with ESMTPS id 43MH6fMB001635 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 22 Apr 2024 13:06:41 -0400 (EDT) (envelope-from kpn@gunsight1.neutralgood.org) Received: (from kpn@localhost) by gunsight1.neutralgood.org (8.17.1/8.17.1/Submit) id 43MH6fRB001632 for freebsd-hackers@freebsd.org; Mon, 22 Apr 2024 13:06:41 -0400 (EDT) (envelope-from kpn) Date: Mon, 22 Apr 2024 13:06:41 -0400 From: "Kevin P. Neal" To: freebsd-hackers@freebsd.org Subject: clang fails libc++experimental test case... Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-No-archive: Yes X-Spamd-Bar: - X-Spamd-Result: default: False [-1.81 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.49)[0.491]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:13649, ipnet:206.196.0.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MISSING_XM_UA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[neutralgood.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4VNWsS74ftz4JjW I'm looking at failing test cases with clang+llvm and I'm developing on FreeBSD. There are multiple failing test cases, but at the moment I'm looking at: clang/test/Driver/experimental-library-flag.cpp This case tests to see if clang properly attempts to link against either libc++ or libstdc++ depending on the flags given to clang. Except FreeBSD's implementation in clang/lib/Driver/ToolChains/FreeBSD.cpp doesn't support libstdc++ at all and just links against libc++. The result is a failing test. Is this intentional? If it is we can mark the test as unsupported on FreeBSD. If not intentional then we need to fix the clang driver. I'd really, really like to be able to do a 'make check' on a build of clang+llvm on FreeBSD and have all tests pass. This is part of that effort. -- Kevin P. Neal http://www.pobox.com/~kpn/ "Not even the dumbest terrorist would choose an encryption program that allowed the U.S. government to hold the key." -- (Fortune magazine is smarter than the US government, Oct 29 2001, page 196.) From nobody Mon Apr 22 17:11:46 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNWxw1XtQz5HV4R for ; Mon, 22 Apr 2024 17:11:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNWxw128Jz4L4v; Mon, 22 Apr 2024 17:11:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713805908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MdxqHkrPlKnJOJOrNL/sXRfx1NzT/zOuBRzEnwF/3R0=; b=ZUIrFn/g/820daKID9wwski53Ztf/KnkZeE1PXC/UDP26aEqR8npZ7K6xWLKFoXuzl7Xfo zhZWnWRR7xfQnIIhWsGK6O+Si2Yc3N0rYLtqY8mnlUHFUS5VeL7li58uvIgDyvrD8iwXbJ /BHa2wg0cNL5fZQhJf99O2WK6+7iHRImG3KGWBnpDma73gSzRBs3ojBgY+uUqe4OO7YnXR YAVMWtCuI4oi4Z90p1ocn9Uyz+jZk43KCh8eJlV/QkJThWf5Z3Fnwk65Ld7i/4DFKQQ5hg YNs4JBouQCuZ+xI4EFAKDp3gMD2mDWTcQEbZfp2m1JsTNYuMkqcqO2Ida36oEQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713805908; a=rsa-sha256; cv=none; b=E5NDgnxUKQ+hIwrDgdepMjmh+huwp5f04mCmkXQIGoQuxOYIrd54dm33xhF8x+q1IYsifg MNm+YgxfE9lduxyYaYZCEjlzZA0Uiu8Hxd8kLrXf9xVU6KaafcW1MlXZTQZy8vy+qeqm+b FMd1/XTOlsgiKOW7Oph0eQwreGgVGCMKKJFODJMmBf5LB4TnKk0prlOfa7PcHlcf6gYukn 9B0OE07HM6cO3VbJOIrytYH42JzGWJWMSZN2c8Sl7GhxtrK7E8TYozWaWG5/mrtBwdlHSt CWjUePGpjzdV9s7GvkJbLOJkB0pZIdtwkplaCS4IonoG5xq6mNLdoYtUTIoLvA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713805908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MdxqHkrPlKnJOJOrNL/sXRfx1NzT/zOuBRzEnwF/3R0=; b=pJQ4nEttoUsZK2j/NRJLEXSJXpmrnxKc0PoJ349eJuinxnAven5Fsu2akKc0HZLUzpirhg zuZlf7nbE1LiasnPhgdmCBT6M1hi/fj/IWpUowKO3kWOBdZNbEdwNjY5Pd7NZ/7GD58WXV YJdJD/TjhAHfEFNNUtzBPcsiCQp3rAT/DX8Ze0VkchmYGJiCyW3neS3L7HWmHlg0J6BKYk SeXLyG+/DxkR8fznb1pM60072+QxMKwjDyikDFsOiBEqst7OhnNOE1GDbl76jVq+ROKa3S TguOWYno1kfHnee7/pUY3MQJr7fP0aN7sDMBe85X0iLJdnLYH0+ChPnfIoR4OA== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNWxw02nPz1Ptw; Mon, 22 Apr 2024 17:11:47 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id B7FB910E08; Mon, 22 Apr 2024 19:11:46 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Upgrading -RELEASE to -CURRENT From: Dimitry Andric In-Reply-To: Date: Mon, 22 Apr 2024 19:11:46 +0200 Cc: Warner Losh , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Josef 'Jeff' Sipek X-Mailer: Apple Mail (2.3731.700.6.1.1) On 22 Apr 2024, at 18:28, Josef 'Jeff' Sipek = wrote: >=20 > On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: > .. >> ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required by = /lib/libc++.so.1 not found >=20 > So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by = 'make > installworld' because the newly built library ends up in /usr/lib [1]. = Therefore, > the broken system has: >=20 > /lib/libcxxrt.so.1 from 14.0-RELEASE > /usr/lib/libcxxrt.so.1 from -CURRENT >=20 > ld-elf.so finds the one in /lib fails to find the required version > (CXXABI_1.3.11) and terminates. This is very strange, and should not happen. The Makefile for libcxxrt specifies SHLIBDIR?=3D/lib, so have you somehow overridden SHLIBDIR somewhere in your environment? -Dimitry From nobody Mon Apr 22 17:16:18 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNX376nwzz5HVYH for ; Mon, 22 Apr 2024 17:16:19 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNX376ClPz4MWP; Mon, 22 Apr 2024 17:16:19 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713806179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZC6DPwchbvuVAARXFDuH4S0YvKtmfvc3AGarhToQYwg=; b=oD/iSBbgdRyB0eLMR94IiUT0ld7aV0DQQbJOCW6WVZiZ6ph+DlBFd/v1XPSgPOidHRg+BA YEUg40vAGupAFQ6Cc/ZFnr+pEgOu14CURzRh336u/JxfMUGQ+8z8OtXZVYbw3CaM2E3LTc MmElwFkc7F1IWb32C9Ul9QWCmFhNQucVnxi343hBfEbw4XSy1T8H9RkV/CyrZ3BKaF3cmn odKbsSFPXIT/pdtnF0gcW064c73Vu6++NSMjoD5WNmdS+8MV608F29HvsigGtovXAHThlu GBvucHvbNVKnhMYdKmeFH86yfzUdJUBJO5yev1+YH4WTy1GAihgkzPN6TaLS1w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713806179; a=rsa-sha256; cv=none; b=UVBTwDRTh+z4sfZmLvJih2vkOpnAHYiTVkPKegwQBOtpUv84toR08/43W956605mEl8cq3 xe6kvS9wxqwE9HXaEI3c8L4G2tPqMEiLFMHuKLVtbpKc9xzghUT96CxZtyxugn8jpArqg2 hjxk8VjjHEpgJbhMXc7pXf7BmPxSY8i9QMoB2roDI7moIbzyD6TG1ol1Z2j44XWySfKHz/ LHQwOwAhVmpvXRslMlAcUQLZ65zR8qtbRCbHVPr3s8PK/TaPK7mulY11WYU/gO7uGnkV35 5uI3tzORIMNJCzWtVtgIxx+DftJ045d+Q8OSNsVrSlzMl5A7YpKeMjzKQYMXrA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713806179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZC6DPwchbvuVAARXFDuH4S0YvKtmfvc3AGarhToQYwg=; b=d1dS9aTDdpn88jl17RDUPKl0Jlycdga55dZ38zwi4uIrCWbj48X0X/3Bqc1M9e7G+2Ie5F RjdOfNhePKV53m3KbCigOjBTvKnib2fvSkBPd7JqaYwZ5AT6o/tl5oNQenBwLCwlldBPPm btlhvxvF+4jk2JkyeowojASYAxfi+u3WbPINqu7brorpvyzAAxGgqOlkWntI1bZBuEka4V qmW3z4lCqkN3ROq1Kgzd2LQOZm4xHF2I0+q3JBZPTVu23SBxi6BL2sDvPrOTkVHBtdtvdB ciZJ1fjpQM/h6XEGV5gCu9bBF9YrfEgVv5QVQjFopslycpRh8CrKLOykSFhnwQ== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNX3755Q2z1Q31; Mon, 22 Apr 2024 17:16:19 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id D0E0310BCD; Mon, 22 Apr 2024 19:16:18 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: clang fails libc++experimental test case... From: Dimitry Andric In-Reply-To: Date: Mon, 22 Apr 2024 19:16:18 +0200 Cc: FreeBSD Hackers Content-Transfer-Encoding: 7bit Message-Id: References: To: "Kevin P. Neal" X-Mailer: Apple Mail (2.3731.700.6.1.1) On 22 Apr 2024, at 19:06, Kevin P. Neal wrote: > > I'm looking at failing test cases with clang+llvm and I'm developing > on FreeBSD. There are multiple failing test cases, but at the moment > I'm looking at: clang/test/Driver/experimental-library-flag.cpp > > This case tests to see if clang properly attempts to link against > either libc++ or libstdc++ depending on the flags given to clang. > Except FreeBSD's implementation in clang/lib/Driver/ToolChains/FreeBSD.cpp > doesn't support libstdc++ at all and just links against libc++. The > result is a failing test. > > Is this intentional? If it is we can mark the test as unsupported on > FreeBSD. If not intentional then we need to fix the clang driver. I think we ripped out libstdc++ support many years ago now. If somebody is really interested in that use case, they could spend the time to make it work again, although it would be tricky to do, since there is no standard location for those headers in FreeBSD anymore. :) So short-term it is probably best to avoid running any of those tests with -stdlib=libstdc++. > I'd really, really like to be able to do a 'make check' on a build of > clang+llvm on FreeBSD and have all tests pass. This is part of that effort. How exactly are you running "make check"? Use the CMake-generated Makefile from the upstream build system? -Dimitry From nobody Mon Apr 22 17:17:50 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNX4x2yHjz5HVmT for ; Mon, 22 Apr 2024 17:17:53 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNX4x15F9z4NBN; Mon, 22 Apr 2024 17:17:53 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; none Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 799EB2C415; Mon, 22 Apr 2024 17:17:51 +0000 (UTC) Date: Mon, 22 Apr 2024 13:17:50 -0400 From: Josef 'Jeff' Sipek To: Dimitry Andric Cc: Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US] X-Rspamd-Queue-Id: 4VNX4x15F9z4NBN On Mon, Apr 22, 2024 at 19:11:46 +0200, Dimitry Andric wrote: > On 22 Apr 2024, at 18:28, Josef 'Jeff' Sipek wrote: > > > > On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: > > .. > >> ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required by /lib/libc++.so.1 not found > > > > So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by 'make > > installworld' because the newly built library ends up in /usr/lib [1]. Therefore, > > the broken system has: > > > > /lib/libcxxrt.so.1 from 14.0-RELEASE > > /usr/lib/libcxxrt.so.1 from -CURRENT > > > > ld-elf.so finds the one in /lib fails to find the required version > > (CXXABI_1.3.11) and terminates. > > This is very strange, and should not happen. Agreed :) > The Makefile for libcxxrt > specifies SHLIBDIR?=/lib, so have you somehow overridden SHLIBDIR > somewhere in your environment? Nope! This is a fresh 14.0-RELEASE install (updated to -p6 via freebsd-update), then I log in as root, install git, get the source, and build. Jeff. From nobody Mon Apr 22 17:19:27 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNX6m6sRJz5HVpd for ; Mon, 22 Apr 2024 17:19:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNX6m6KZRz4PBr; Mon, 22 Apr 2024 17:19:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713806368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9NJAisuPob2jEDnNFX9iPojxxQSH3fMChDYR1jD478Q=; b=tXV0vSiJap5emVSktFl5yCGjR2rAcYBn9WqKrsGT3APBJM8HlPrdWRzEKZhngncFmd/3il UAL4PDE0SU9xDre1FXQ2NxCuwfjJH625fZCymutxowXpLqBLG8jQEUSzzTLbfXZLEgNd5a 2EtvCQPpzmGD6SZKIJ9ogdLOelrh7JCxTrCjt6o4eV6pPxVxfgqq/1tgYIh08T2tLtoWdD vu1iv46NcCzjzNP0I42DhIuyvDiNdRQAKc52Ksu1w3Di++T4sUEnR1uBpi9eMjq0TmmicQ 45ejnM4nZWDcp1HY9XIvUsMc8axNjk8KD8p1QJDlUkgeJaDnxltBpos+7alUog== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713806368; a=rsa-sha256; cv=none; b=L2gqfgVicZLNXRDT9AHbIdKLLVdu52ngq8fUSjpuTwcM49y5gEBbaeUp4suS+lB9ReKz1o DqApZ7SqoPADQkwMiLCIprglMi428o7NLGzUDgO2tlBkMLYOVjjsUkD5MtgmpYKUX310gJ 7fRJMOH0qDssoyYxo98VatvBl1sRs6o3cE40ntj+4BH4rTDsd+jIGU/5OYr9gn09/gDpo7 4xyTyGa/+E+3gRkg4ulPKPM8sfwFW5RPLhvt7SEdpk5YfvjQLXFBY+jFG1230u6lPR+BSU dff09gyXwOiNQ8QUb8fwBJzdbF2wEukI7A/7N9BbP5tfh0/4KQeKbP+rDnZL8Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713806368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9NJAisuPob2jEDnNFX9iPojxxQSH3fMChDYR1jD478Q=; b=rXKb40WUXE5+yi0uSds+j9LW35h0Vnm+aKj9AsV1b59lb58GKurPjqkXHfpjaAmG2sSBZz dqqhNYjNFw2HEZ77KCqGupshDszV6JZWyohTiGzdKnAuEMYSLWMW+oBINKq7zSqBKH4JsY OeaQNF0KEWXQGRqUGuDgHDpiSCCfYfBG6648Cff/XDoG9kijI6+mi8FjJSQYyqvdH65GGD 0xVmriPTOyeaig8/HnlqY6WhzLLsZZnAmTo8mOmfE61gE5OpNrBx6nSKyLFAMNxRAlndCW /2Nb/LMtUWedZqymdHIcUF6tOH647jjH1JWXgWrCKdkhsyg/Dqtz2UwO2vlo3Q== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNX6m5MRVz1Pty; Mon, 22 Apr 2024 17:19:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id D24CD10BD1; Mon, 22 Apr 2024 19:19:27 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Upgrading -RELEASE to -CURRENT From: Dimitry Andric In-Reply-To: Date: Mon, 22 Apr 2024 19:19:27 +0200 Cc: Warner Losh , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> References: To: Josef 'Jeff' Sipek X-Mailer: Apple Mail (2.3731.700.6.1.1) On 22 Apr 2024, at 19:17, Josef 'Jeff' Sipek = wrote: >=20 > On Mon, Apr 22, 2024 at 19:11:46 +0200, Dimitry Andric wrote: >> On 22 Apr 2024, at 18:28, Josef 'Jeff' Sipek = wrote: >>>=20 >>> On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: >>> .. >>>> ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required = by /lib/libc++.so.1 not found >>>=20 >>> So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by = 'make >>> installworld' because the newly built library ends up in /usr/lib = [1]. Therefore, >>> the broken system has: >>>=20 >>> /lib/libcxxrt.so.1 from 14.0-RELEASE >>> /usr/lib/libcxxrt.so.1 from -CURRENT >>>=20 >>> ld-elf.so finds the one in /lib fails to find the required version >>> (CXXABI_1.3.11) and terminates. >>=20 >> This is very strange, and should not happen. >=20 > Agreed :) >=20 >> The Makefile for libcxxrt >> specifies SHLIBDIR?=3D/lib, so have you somehow overridden SHLIBDIR >> somewhere in your environment? >=20 > Nope! This is a fresh 14.0-RELEASE install (updated to -p6 via > freebsd-update), then I log in as root, install git, get the source, = and > build. How, exactly, are you building? -Dimitry From nobody Mon Apr 22 17:21:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNX8x6j57z5HVtQ for ; Mon, 22 Apr 2024 17:21:21 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNX8x6JC3z4Q7P; Mon, 22 Apr 2024 17:21:21 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; none Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 780E02C2DF; Mon, 22 Apr 2024 17:21:21 +0000 (UTC) Date: Mon, 22 Apr 2024 13:21:20 -0400 From: Josef 'Jeff' Sipek To: Dimitry Andric Cc: Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US] X-Rspamd-Queue-Id: 4VNX8x6JC3z4Q7P On Mon, Apr 22, 2024 at 19:19:27 +0200, Dimitry Andric wrote: > On 22 Apr 2024, at 19:17, Josef 'Jeff' Sipek wrote: > > > > On Mon, Apr 22, 2024 at 19:11:46 +0200, Dimitry Andric wrote: > >> On 22 Apr 2024, at 18:28, Josef 'Jeff' Sipek wrote: > >>> > >>> On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: > >>> .. > >>>> ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required by /lib/libc++.so.1 not found > >>> > >>> So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by 'make > >>> installworld' because the newly built library ends up in /usr/lib [1]. Therefore, > >>> the broken system has: > >>> > >>> /lib/libcxxrt.so.1 from 14.0-RELEASE > >>> /usr/lib/libcxxrt.so.1 from -CURRENT > >>> > >>> ld-elf.so finds the one in /lib fails to find the required version > >>> (CXXABI_1.3.11) and terminates. > >> > >> This is very strange, and should not happen. > > > > Agreed :) > > > >> The Makefile for libcxxrt > >> specifies SHLIBDIR?=/lib, so have you somehow overridden SHLIBDIR > >> somewhere in your environment? > > > > Nope! This is a fresh 14.0-RELEASE install (updated to -p6 via > > freebsd-update), then I log in as root, install git, get the source, and > > build. > > How, exactly, are you building? See my earlier email from 19 Apr 2024 22:05:17 -0400 for details. But in short, 'make buildworld' 'make buildkernel' 'make installkernel' 'make installworld'. Jeff. From nobody Mon Apr 22 18:47:04 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNZ5L1VYLz5HfYn for ; Mon, 22 Apr 2024 18:48:22 +0000 (UTC) (envelope-from kpn@neutralgood.org) Received: from gunsight1.NeutralGood.ORG (gunsight1.neutralgood.org [206.196.19.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gunsight1.neutralgood.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNZ5L11Ntz4ZK2; Mon, 22 Apr 2024 18:48:22 +0000 (UTC) (envelope-from kpn@neutralgood.org) Authentication-Results: mx1.freebsd.org; none Received: from gunsight1.NeutralGood.ORG (localhost [127.0.0.1]) by gunsight1.NeutralGood.ORG (8.17.1/8.17.1) with ESMTPS id 43MIl4Yp061624 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 22 Apr 2024 14:47:05 -0400 (EDT) (envelope-from kpn@gunsight1.NeutralGood.ORG) Received: (from kpn@localhost) by gunsight1.NeutralGood.ORG (8.17.1/8.17.1/Submit) id 43MIl4fD061621; Mon, 22 Apr 2024 14:47:04 -0400 (EDT) (envelope-from kpn) Date: Mon, 22 Apr 2024 14:47:04 -0400 From: "Kevin P. Neal" To: Dimitry Andric Cc: FreeBSD Hackers Subject: Re: clang fails libc++experimental test case... Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-No-archive: Yes X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:13649, ipnet:206.196.0.0/19, country:US] X-Rspamd-Queue-Id: 4VNZ5L11Ntz4ZK2 On Mon, Apr 22, 2024 at 07:16:18PM +0200, Dimitry Andric wrote: > On 22 Apr 2024, at 19:06, Kevin P. Neal wrote: > > > > I'm looking at failing test cases with clang+llvm and I'm developing > > on FreeBSD. There are multiple failing test cases, but at the moment > > I'm looking at: clang/test/Driver/experimental-library-flag.cpp > > > > This case tests to see if clang properly attempts to link against > > either libc++ or libstdc++ depending on the flags given to clang. > > Except FreeBSD's implementation in clang/lib/Driver/ToolChains/FreeBSD.cpp > > doesn't support libstdc++ at all and just links against libc++. The > > result is a failing test. > > > > Is this intentional? If it is we can mark the test as unsupported on > > FreeBSD. If not intentional then we need to fix the clang driver. > > I think we ripped out libstdc++ support many years ago now. If somebody > is really interested in that use case, they could spend the time to make > it work again, although it would be tricky to do, since there is no > standard location for those headers in FreeBSD anymore. :) > > So short-term it is probably best to avoid running any of those tests > with -stdlib=libstdc++. That sounds like an argument for disabling this particular test. > > I'd really, really like to be able to do a 'make check' on a build of > > clang+llvm on FreeBSD and have all tests pass. This is part of that effort. > > How exactly are you running "make check"? Use the CMake-generated > Makefile from the upstream build system? I work full time developing LLVM and only build it from a tree checked out from the official LLVM repo at GitHub. Yes, I run cmake from my build directory and have it create all the files needed by make or by ninja. I actually run 'ninja check' but 'make check' is easier to talk about since everyone here already knows about make. I only build clang. LLVM's cmake configuration allows one to build only the parts of the project that you need. I only need clang+llvm, and it still takes nearly two hours to run the tests on a 12-way login server. >From the shell script to I use to run cmake: exec cmake -G Ninja \ -DLLVM_PARALLEL_LINK_JOBS=10 \ -DCMAKE_BUILD_TYPE=Debug \ -DLLVM_ENABLE_ASSERTIONS=On \ -DLLVM_ENABLE_EXPENSIVE_CHECKS=On \ -DLLVM_ENABLE_PROJECTS="clang" \ -- Kevin P. Neal http://www.pobox.com/~kpn/ "Good grief, I've just noticed I've typed in a rant. Sorry chaps!" Keir Finlow Bates, circa 1998 From nobody Mon Apr 22 18:58:34 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNZK86MW5z5HgHD for ; Mon, 22 Apr 2024 18:58:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNZK85JW2z4bwl; Mon, 22 Apr 2024 18:58:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713812316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oJO0U/OqsMDe0SGhD/zz06pK7qNTonJ3cRpF2zjtXTg=; b=yPi44ZZ15sMXBOeDtKPc7V3Fg/tz/4yvWXCaiv4jQ7+4uX9yLEFgnd0Qu3XiyPF0bP3FVD onQeInwzURTmHxL4gvog35ZQeU0XQeXKCS+UwE9BadLzTCI+Hd8bdT2+qPOCAIu9HCN0ma rMbDqr91mNVUmf3n6/Ne2SbykzOaBY688xqIAYUPsoeYXYKL1u8y015ZfnWX4O7YBeENGM txPbXafKPCFkiWBwqh4Ba6MJHkLNKv6AIhr8GGLubCaVl856Y3prJkmK6oXuZd43oAvzR2 tTtL44GQjCGis0floTr1FHK1g+m2PANIRG0/dHE6kYT3uv4z1WLq5NpbqWOcQQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713812316; a=rsa-sha256; cv=none; b=So/NWjOI+e6pU+v2RYikEFsC0NqVI7DlKCpJxJVs2bg1IQbsAuxlLoLI80lDKdOFimkfvp jeHVxCn5diLSngo4ruNvMgvBMyJDKHZpQMp6m+ZqmVUAE99JNAkWT8C8Z0fy3RF1GLmHwW 9Sc0NInFOt+cSTklMmakmd8ZV4Be2lyLBhntfSF7fWz8k1uy8hiCA/RAhaP6tQB9GedzNT kO764apCF+IoiQk12QAtSz/dFedyo3Nai2jvPGIQB6b+oI52oisqe33Ks+5CEDgZ6xlJtC GucRnd2bNUQEAefI/CkHxEgBdkYwWbbunYubW+urg9FpFxnYJJJISkM0Hm4ObQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713812316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oJO0U/OqsMDe0SGhD/zz06pK7qNTonJ3cRpF2zjtXTg=; b=OEwwKzV/pZR5LhW6XvBxyX/CmWzlqNzFdCXbYn8sZiRphdeSfcgVeHIqDqt47dQh2Lcd+7 nbLT0fY1DgAC3sWw3qSjtz1F3VOPMFZzcZeCX1N5OBsg6UuWTuuEbSumhKKSDfKiEOqEPf 9MGkz2RYXnz83IaFwzI/RA3juAGz3+acvXoCfqmQ1oVQmoRlXshG7SjkkkPa8XF1KKeUWw oz3hTLsuDT23ABcJTG+pR+KEkDPp+AE5QZb1AOUHvc8bkHBM7shSngR0uSZmXQgm3ZAz8F 5xhr7j9zc7usBK058gcSvvGxpri0uOl12fAsEroWty5mmKB6eC0eYNKx7WEwrQ== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNZK84BN7z1RFn; Mon, 22 Apr 2024 18:58:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 1802A10E77; Mon, 22 Apr 2024 20:58:35 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Upgrading -RELEASE to -CURRENT From: Dimitry Andric In-Reply-To: Date: Mon, 22 Apr 2024 20:58:34 +0200 Cc: Warner Losh , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> To: Josef 'Jeff' Sipek X-Mailer: Apple Mail (2.3731.700.6.1.1) On 22 Apr 2024, at 19:21, Josef 'Jeff' Sipek = wrote: >=20 > On Mon, Apr 22, 2024 at 19:19:27 +0200, Dimitry Andric wrote: >> On 22 Apr 2024, at 19:17, Josef 'Jeff' Sipek = wrote: >>>=20 >>> On Mon, Apr 22, 2024 at 19:11:46 +0200, Dimitry Andric wrote: >>>> On 22 Apr 2024, at 18:28, Josef 'Jeff' Sipek = wrote: >>>>>=20 >>>>> On Fri, Apr 19, 2024 at 22:05:17 -0400, Josef 'Jeff' Sipek wrote: >>>>> .. >>>>>> ld-elf.so.1: /lib/libcxxrt.so.1: version CXXABI_1.3.11 required = by /lib/libc++.so.1 not found >>>>>=20 >>>>> So, the problem is /lib/libcxxrt.so.1 is *not* getting updated by = 'make >>>>> installworld' because the newly built library ends up in /usr/lib = [1]. Therefore, >>>>> the broken system has: >>>>>=20 >>>>> /lib/libcxxrt.so.1 from 14.0-RELEASE >>>>> /usr/lib/libcxxrt.so.1 from -CURRENT >>>>>=20 >>>>> ld-elf.so finds the one in /lib fails to find the required version >>>>> (CXXABI_1.3.11) and terminates. >>>>=20 >>>> This is very strange, and should not happen. >>>=20 >>> Agreed :) >>>=20 >>>> The Makefile for libcxxrt >>>> specifies SHLIBDIR?=3D/lib, so have you somehow overridden SHLIBDIR >>>> somewhere in your environment? >>>=20 >>> Nope! This is a fresh 14.0-RELEASE install (updated to -p6 via >>> freebsd-update), then I log in as root, install git, get the source, = and >>> build. >>=20 >> How, exactly, are you building? >=20 > See my earlier email from 19 Apr 2024 22:05:17 -0400 for details. But = in > short, 'make buildworld' 'make buildkernel' 'make installkernel' 'make = installworld'. To properly finish up this thread, Jeff was right, and https://cgit.freebsd.org/src/commit/?id=3Dda77a1b4f0dff was the cause. That commit added a .include at the top of libcxxrt's Makefile, which is normally fine, but not if you use SHLIBDIR?=3D/lib. That sort of assignment should always be done before including any of the bsd.*.mk files. I have committed https://cgit.freebsd.org/src/commit/?id=3D911a6479e18bc for now, which should fix the problem. It also adds an ObsoleteFiles.inc entry for /usr/lib/libcxxrt.so.1, so the file should be removed when you run "make delete-old-libs". -Dimitry From nobody Mon Apr 22 19:26:05 2024 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNZx76m7rz5HkTb for ; Mon, 22 Apr 2024 19:26:19 +0000 (UTC) (envelope-from lexi@le-fay.org) Received: from fuchsia.eden.le-Fay.ORG (fuchsia.eden.le-fay.org [81.187.47.195]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNZx74hN8z4jKT; Mon, 22 Apr 2024 19:26:19 +0000 (UTC) (envelope-from lexi@le-fay.org) Authentication-Results: mx1.freebsd.org; none Received: from iris.eden.le-Fay.ORG (iris.eden.le-fay.org [IPv6:2001:8b0:aab5:106:3::6]) by fuchsia.eden.le-Fay.ORG (Postfix) with ESMTP id DE9EA95DA; Mon, 22 Apr 2024 19:26:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=le-fay.org; s=fuchsia; t=1713813976; bh=ohuqsFi76CCbsjDul7AMFgo0Zkhgy1CZ6uKrr+Iojn4=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=Hz4xSqoK5dqwbXXxoaBYrDPbo/xKvnc3xz07SNV2XwN/OnAA4LQ/wuqlaivQIldg8 GFtZTHNKaa/uCzPnLdCbcPlBZLfxurLispe/Lxi00hz6oanuKAN0cKn5PZoukMqIcl /RRiv/G94wjxZ3kdSnehxDJIPLZcuem5AgNtEQqk= Received: from ilythia.eden.le-fay.org (ilythia.eden.le-fay.org [IPv6:2001:8b0:aab5:106:3::10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by iris.eden.le-Fay.ORG (Postfix) with ESMTPSA id A691B2C0421; Mon, 22 Apr 2024 20:26:16 +0100 (BST) Date: Mon, 22 Apr 2024 20:26:05 +0100 From: Lexi Winter To: Brooks Davis Cc: hackers@freebsd.org Subject: Re: building memstick without root Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="q/FFPPcr7Gc0cGky" Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20712, ipnet:81.187.0.0/16, country:GB] X-Rspamd-Queue-Id: 4VNZx74hN8z4jKT --q/FFPPcr7Gc0cGky Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Brooks Davis: > On Sun, Apr 21, 2024 at 07:45:22PM +0100, Lexi Winter wrote: > > is it expected that running 'make -C release memstick' requires root? =20 > You'll need to define NO_ROOT. I think that's sufficent, but I'm not > 100% certain. We build CheriBSD releases without root on Linux hosts. thanks Brooks, that has sorted it. --q/FFPPcr7Gc0cGky Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCAAdFiEEuwt6MaPcv/+Mo+ftDHqbqZ41x5kFAmYmucoACgkQDHqbqZ41 x5nM8Av+KHoru/nCbFQN3IRQzq5l/16pqMDE0ONsOgRV/i26sxQ68YKg5oUbjeDZ V/MbBnOH9mVfWuvc5eS/ogkTfFMCLQBEDBotFH6B4vhF/0BTOibiDyT//mzA8U+j 245cVszt/wAyW6eL0gpeCTyd9LpzPKfNRhtFx/mFAZg+FYNSBI9T1CLSUUi+40xQ x0mlaAh5/N5tSc8Y0bTvnoXtfF+QsSAJ77zg4YbiUjT6szJXLdYxkIYqrBUZygbu M8EESc4aKk5u7SuZY0CGc+e+1V7me29wVP0D0jS0PqRO3MhAZCkVSUzar78YMiyT y5KB1G5mMcMztYfBVo9KX01N+HaCBld962C8i7Ja1KTZ4/pbGovua4SiLp3Qjk7i zuCxPmriEMnoOzDPVd1NUljokMuzGdZCtTdP2/RQ/NaEr7QjiAXIqjPwh9fN5q+n aIyZfwSoz0RkgZlCx9B1Hd8YmqiVcbWHsZz4nXxMUU9baMiZpMPUNI13w4hWyCaP cdPIU8KX =2/wv -----END PGP SIGNATURE----- --q/FFPPcr7Gc0cGky-- From nobody Mon Apr 22 19:28:42 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNZzx0WFSz5Hkdb for ; Mon, 22 Apr 2024 19:28:45 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNZzw5qLxz4jrT; Mon, 22 Apr 2024 19:28:44 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; none Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id ED4D52C0F8; Mon, 22 Apr 2024 19:28:43 +0000 (UTC) Date: Mon, 22 Apr 2024 15:28:42 -0400 From: Josef 'Jeff' Sipek To: Dimitry Andric Cc: Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US] X-Rspamd-Queue-Id: 4VNZzw5qLxz4jrT On Mon, Apr 22, 2024 at 20:58:34 +0200, Dimitry Andric wrote: > To properly finish up this thread, Jeff was right, and > https://cgit.freebsd.org/src/commit/?id=da77a1b4f0dff was the cause. > That commit added a .include at the top of libcxxrt's > Makefile, which is normally fine, but not if you use SHLIBDIR?=/lib. > That sort of assignment should always be done before including any of > the bsd.*.mk files. > > I have committed https://cgit.freebsd.org/src/commit/?id=911a6479e18bc > for now, which should fix the problem. It also adds an ObsoleteFiles.inc > entry for /usr/lib/libcxxrt.so.1, so the file should be removed when you > run "make delete-old-libs". FWIW, with this change, I just did a successful upgrade of a 14.0-RELEASE directly to -CURRENT. Thanks! Jeff. From nobody Mon Apr 22 19:48:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNbQm3ZJ7z5HmRv for ; Mon, 22 Apr 2024 19:48:32 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNbQm32ZZz4rNg; Mon, 22 Apr 2024 19:48:32 +0000 (UTC) (envelope-from brooks@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713815312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sy0oqh3Y6XmHmaFFmk89eXWaG7hwWDUi7l5Md3YBrMY=; b=Z5Zb7AzmAf77/AtDep80VPF0VtFFaPpNr1hxiJblKazfWXko7F5oNKsrIldXAfTl79ujI1 144DikH3cQccGPZL4IDPJqi/mdULBdfiPKskEglJC/b2fgvc8rsi9bbYwm5LHOUvpbpxBZ 8BrwYhdemoXxNrD1cJEi1Y93SajdkhGE2Cy/EEXjK4XHC3FGdWHz3jreNiFEV6w9mcHWM5 y6Kz3EC0mFT8vdKuVSQMKdoM8Nd6RIBOOmjZ2y/c+7RZty4NW+8xwBeH48DXcZLvFgvjdd pXd0/cnizFm9YCJyzEpBME8Z3RkGmo3J1pTevu0jnsCYukGBjQ0+HhWSzzWD0w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713815312; a=rsa-sha256; cv=none; b=OXdq81vzNUfaw/aNqRZPDHMEvceC2taWKAwQuVfn/I0YW1r8jMCFSqZLOYI2lVcTCr593n faJaE/SYTqy/qxWvGZ4BrYeIabhpb0m/w65Dm+0Xu/zLyQ2kbd370yt0ia/NFIfzchb+GF 3ccC5Roxy5UysL3EwZKYZnMF4LH+PL9wmp0emvchXdChIuG6U42SU8hRDh616+Y7aUGpuq vrKN6C7XbcwRhhHHSgXOLMEWLSaLRkhmBcUailLuKERBNQYXY+JslbhUP68fdjBbaOcFix NI74caJHaavaxqozvRW/vCYyoUZJU0djBzfyl8Mttd9DuAWbil3u5yKpBNU+jQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713815312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sy0oqh3Y6XmHmaFFmk89eXWaG7hwWDUi7l5Md3YBrMY=; b=tVZDOIWNSgfxajqHHc2/yCW0bk6QzR2GTt9Wf7WfsyrO6SazDnxGsgOmMYzu0SeQE1amFV zFu+P+SGuKFjdeu0iYtZIr7YPCj/xDKu0P6V7k8YyR3DCQ0+cLB3dDDZ0XTwB4y8vMMECj 1pIHjoOIBJNd1bOC88qYYMWOY9uhnQ46NRanVykeGUjaOju/iUO+lC31PGq/72uPADUfeI JSpTXH1VGHs4HfLBgCck1onwRG+fB5ehbIXnjGPQlGfTUXgxQCo2tGbCwhaRdS9BZonrQM 1uttA5Koi89U97fFH9KbLLZGpRnHJul4y2zlfltz4zYLSQXNYVjTX1yNepLCzA== Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: brooks/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VNbQm2P9fz1RpW; Mon, 22 Apr 2024 19:48:32 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id A99CC3C019B; Mon, 22 Apr 2024 19:48:31 +0000 (UTC) Date: Mon, 22 Apr 2024 19:48:31 +0000 From: Brooks Davis To: Josef 'Jeff' Sipek Cc: Dimitry Andric , Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Mon, Apr 22, 2024 at 03:28:42PM -0400, Josef 'Jeff' Sipek wrote: > On Mon, Apr 22, 2024 at 20:58:34 +0200, Dimitry Andric wrote: > > To properly finish up this thread, Jeff was right, and > > https://cgit.freebsd.org/src/commit/?id=da77a1b4f0dff was the cause. > > That commit added a .include at the top of libcxxrt's > > Makefile, which is normally fine, but not if you use SHLIBDIR?=/lib. > > That sort of assignment should always be done before including any of > > the bsd.*.mk files. > > > > I have committed https://cgit.freebsd.org/src/commit/?id=911a6479e18bc > > for now, which should fix the problem. It also adds an ObsoleteFiles.inc > > entry for /usr/lib/libcxxrt.so.1, so the file should be removed when you > > run "make delete-old-libs". > > FWIW, with this change, I just did a successful upgrade of a 14.0-RELEASE > directly to -CURRENT. Sorry about the bug. I need to think how to efficently check for that sort of bug without causing undue pain. -- Brooks From nobody Mon Apr 22 20:07:00 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNbrr0SNSz5HnWx for ; Mon, 22 Apr 2024 20:07:40 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNbrp1v0kz4tJp for ; Mon, 22 Apr 2024 20:07:38 +0000 (UTC) (envelope-from karl@denninger.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net Received: from denninger.net (unknown [96.33.195.197]) by colo1.denninger.net (Postfix) with ESMTP id AD3412111E9 for ; Mon, 22 Apr 2024 16:07:30 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id A3C50377C48 for ; Mon, 22 Apr 2024 16:07:01 -0400 (EDT) Message-ID: Date: Mon, 22 Apr 2024 16:07:00 -0400 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Stressing malloc(9) To: freebsd-hackers@freebsd.org References: Content-Language: en-US From: Karl Denninger In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms080707000903030302050808" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.61 / 15.00]; SIGNED_SMIME(-2.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.997]; NEURAL_SPAM_MEDIUM(0.74)[0.737]; NEURAL_HAM_LONG(-0.56)[-0.559]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; MIME_BASE64_TEXT(0.10)[]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; RCVD_TLS_LAST(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEFALL_USER(0.00)[karl]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VNbrp1v0kz4tJp This is a cryptographically signed message in MIME format. --------------ms080707000903030302050808 Content-Type: multipart/alternative; boundary="------------H2q7nl5KNV3fGY01y1MXZ54v" --------------H2q7nl5KNV3fGY01y1MXZ54v Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gNC8yMi8yMDI0IDEyOjQ2LCBBbGFuIFNvbWVycyB3cm90ZToNCj4gV2hlbiBJIHNhaWQg IjMza2lCIiBJIG1lYW50ICIzMyBwYWdlcyIsIG9yIDEzMiBrQi4gIEFuZCB0aGUgc29sdXRp b24NCj4gdHVybnMgb3V0IHRvIGJlIHZlcnkgZWFzeS4gIFNpbmNlIEknbSB1c2luZyBaRlMg b24gdG9wIG9mIGdlbGksIHdpdGgNCj4gdGhlIGRlZmF1bHQgcmVjc2l6ZSBvZiAxMjhrQiwg SSdsbCBqdXN0IHNldA0KPiB2ZnMuemZzLnZkZXYuYWdncmVnYXRpb25fbGltaXQgdG8gMTI4 IGtCLiAgVGhhdCB3YXkgZ2VsaSB3aWxsIG5ldmVyDQo+IG5lZWQgdG8gYWxsb2NhdGUgbW9y ZSB0aGFuIDEyOGtCIGNvbnRpZ3VvdXNseS4gIFpGUyBkb2Vzbid0IGV2ZW4gbmVlZA0KPiB0 aG9zZSBiaWcgYWxsb2NhdGlvbnMgdG8gYmUgY29udGlndW91czsgaXQncyBqdXN0IGFnZ3Jl Z2F0aW5nIHNtYWxsZXINCj4gb3BlcmF0aW9ucyB0byByZWR1Y2UgZGlzayBJT1BzLiAgQnV0 IGFnZ3JlZ2F0aW5nIHVwIHRvIDFNQiAodGhlDQo+IGRlZmF1bHQpIGlzIG92ZXJraWxsOyBh bnkgcm90YXRpbmcgSEREIHNob3VsZCBlYXNpbHkgYmUgYWJsZSB0byBtYXgNCj4gb3V0IGl0 cyBjb25zZWN1dGl2ZSB3cml0ZSBJT1BzIHdpdGggMTI4a0Igb3BlcmF0aW9uIHNpemUuICBJ J2xsIGFkZCBhDQo+IHJlYWQtb25seSBzeXNjdGwgZm9yIGdfZWxpX2FsbG9jX3N6IHRvby4g IFRoYW5rcyBNYXJrLg0KPg0KPiAtQWxhbg0KDQpTZXR0aW5nIHRoaXMgb24gb25lIG9mIG15 IHByb2R1Y3Rpb24gbWFjaGluZXMgdGhhdCB1c2VzIHpmcyBiZWhpbmQgZ2VsaSANCmRyb3Bz IHRoZSBsb2FkIGF2ZXJhZ2UgcXVpdGUgbWF0ZXJpYWxseSB3aXRoIHplcm8gaW1wYWN0IG9u IHRocm91Z2hwdXQgDQp0aGF0IEkgY2FuIHNlZSAodGh1cyBmYXIuKcKgIEkgd2lsbCBydW4g dGhpcyBmb3IgYSB3aGlsZSBidXQgaXQgY2VydGFpbmx5IA0KZG9lc24ndCBhcHBlYXIgdG8g aGF2ZSBhbnkgbmVnYXRpdmVzIGFzc29jaWF0ZWQgd2l0aCBpdCBhbmQgZG9lcyBhcHBlYXIg DQp0byBpbXByb3ZlIGVmZmljaWVuY3kgcXVpdGUgYSBiaXQuDQoNCi0tIA0KS2FybCBEZW5u aW5nZXINCmthcmxAZGVubmluZ2VyLm5ldA0KL1RoZSBNYXJrZXQgVGlja2VyLw0KL1tTL01J TUUgZW5jcnlwdGVkIGVtYWlsIHByZWZlcnJlZF0vDQo= --------------H2q7nl5KNV3fGY01y1MXZ54v Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On 4/22/2024 12:46, Alan Somers wrote:= 
When I said "33kiB" I meant =
"33 pages", or 132 kB.  And the solution
turns out to be very easy.  Since I'm using ZFS on top of geli, with
the default recsize of 128kB, I'll just set
vfs.zfs.vdev.aggregation_limit to 128 kB.  That way geli will never
need to allocate more than 128kB contiguously.  ZFS doesn't even need
those big allocations to be contiguous; it's just aggregating smaller
operations to reduce disk IOPs.  But aggregating up to 1MB (the
default) is overkill; any rotating HDD should easily be able to max
out its consecutive write IOPs with 128kB operation size.  I'll add a
read-only sysctl for g_eli_alloc_sz too.  Thanks Mark.

-Alan

Setting this on one of my production machines that uses zfs behind geli drops the load average quite materially with zero impact on throughput that I can see (thus far.)=C2=A0 I will run th= is for a while but it certainly doesn't appear to have any negatives associated with it and does appear to improve efficiency quite a bit.

--
Karl Denninger
karl@denninger.net
The Market Ticker
[S/MIME encrypted email preferred]<= /div> --------------H2q7nl5KNV3fGY01y1MXZ54v-- --------------ms080707000903030302050808 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DbowggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBxIwggT6oAMCAQICEgLG8yH4PQFdbd9x Ugmpzl1jXzANBgkqhkiG9w0BAQsFADB7MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlk YTEZMBcGA1UECgwQQ3VkYSBTeXN0ZW1zIExMQzEYMBYGA1UECwwPQ3VkYSBTeXN0ZW1zIENB MSUwIwYDVQQDDBxDdWRhIFN5c3RlbXMgTExDIDIwMTcgSW50IENBMB4XDTIyMDYyOTE2MTYz NloXDTI3MDYyODE2MTYzNlowOjELMAkGA1UEBhMCVVMxEjAQBgNVBAgMCVRlbm5lc3NlZTEX MBUGA1UEAwwOS2FybCBEZW5uaW5nZXIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC AQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvW ZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B 3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTgy+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYI XgVVPgfZZrbJJb5HWOQpvvhILpPCD3xsYJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMi WapsatKm8mxuOOGOEBhAoTVTwUHlMNTg6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMb NQm1mWREQhw3axgGLSntjjnznJr5vsvXSYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZM qa20JLAF1YagutDiMRURU23iWS7bA9tMcXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5 CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1l y+5ZOZbxBAZZMod4y4b4FiRUhRI97r9lCxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY 2BlA7ExM8XShMd9bRPZrNTokPQPUCWCgCdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEE MDAuMCwGCCsGAQUFBzABhiBodHRwOi8vb2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNV HRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIFoDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYI KwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCGSAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBD bGllbnQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNV HSMEgcIwgb+AFF3AXsKnjdPND5+bxVECGKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQ MA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5 c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lz dGVtcyBMTEMgMjAxNyBDQYITAORIioIQzl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJs QGRlbm5pbmdlci5uZXQwDQYJKoZIhvcNAQELBQADggIBAKquc7cu0xc8FNtAQauZvocDzWQj 7HG9YvMbWnMi+ckhiA3rdW5NwWg0HBhBho1YlnqV+ntCVE2L8ezohHWm+KAdfXgpraL86Vsn 3ywNlZu/3COMpo2ALuHln8YQtH3Y8ebvzKMdlf2b5WB+7mOFIxXIr4AnNOLKCkq5ZhzC6JW6 Jvw3P0csiGa3UrfatYID5NvPgkaQvEgimEjG3psZqwQTL2Wxohvw783PrDt3wS0XeNhvQ61g 3QJFZKuv+bmGH3YBSPo1t6NUGAr+JozX5lDihB8JGkBt/NwdYec49a08uL0BbPaAJ7NjuIPG 7Y0Ak7PXZT37yx/Zla9PzLMJFgbelOkaatdzbblMZPDEVZ27l4lGMmV83Lm3YP17sdAyS/Wp mav7WmJUkQ9iuIKzSpdc82i9Mfujl1vbBtwtkHNPPtKuulIFM4ZwrPKjlVdLqTSqD8m9yHEi Y0PuAooq63OpJWF6hvMaiIPBWEAVIaDW9uG0MshLl9DnHnMyrJTfuC33Z9mOGMz7dRBjJd5Y W02xAzYnUuEBOpj+LQv5R8XIFMHFXktqEKvQrXeM2RU+PcZqKOBkTktxBLn3NI5VfA15Jk0c 5V5XcOqo3p2hvrwvXrinrb2pEREnoqmfrkXT3zOq5Y6ryRH8u734lGEF0dILXzoV4PM7XFit oTePoEjmMYIFBDCCBQACAQEwgZEwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGEx GTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTEl MCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQQISAsbzIfg9AV1t33FSCanO XWNfMA0GCWCGSAFlAwQCAwUAoIICQzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqG SIb3DQEJBTEPFw0yNDA0MjIyMDA3MDBaME8GCSqGSIb3DQEJBDFCBECOHt1wXHx5CvSPENR3 5jx0y3K/3IwD92Z+srA3GqAnVSLlnNEd5mBHmvHgpNZB5d7lChofKrpMccmYd5HqeIV6MGwG CSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAO BggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgw gaIGCSsGAQQBgjcQBDGBlDCBkTB7MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTEZ MBcGA1UECgwQQ3VkYSBTeXN0ZW1zIExMQzEYMBYGA1UECwwPQ3VkYSBTeXN0ZW1zIENBMSUw IwYDVQQDDBxDdWRhIFN5c3RlbXMgTExDIDIwMTcgSW50IENBAhICxvMh+D0BXW3fcVIJqc5d Y18wgaQGCyqGSIb3DQEJEAILMYGUoIGRMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9y aWRhMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMg Q0ExJTAjBgNVBAMMHEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0ECEgLG8yH4PQFdbd9x Ugmpzl1jXzANBgkqhkiG9w0BAQEFAASCAgAAOOCpIE2WCS8QRNvUJGAm+kJHs/cidPacOwqd 9Wz/fQGZxOqGplh1cjNp1J82yR8yquGFiMM/ivncWU0k/Lo6AoTlAZt2YcYUPB/SQ5/+4+q/ 3qHvy5OsrpwZHdZFZvgI5YrDsB+G0GAxQULI8AA2BAytNSAZ6CZOExf4TgLCdpKcsjj0YyNQ +HBMvMRhrQy2HS3o+Xdru2jwGiWqvcfVng2kLySdBeHiqN7jR/ncbqd82wtDdtFgSxTgWmtj 6GTqw/CqWykJaP3mlY9/KvMa3Fu5OMqxEzbAyKylTXMVPt5qPqRXtXNBkgolwYkVbWy9sapg PSImC4pILbgurxUEKpSvHPQdJig9hGs+eouUJAMV4c+nhmFcwgs0afnRd5ibS7H9Qpqft+oP K/D6EaRVMOSYseIpoa3mBW4Az8Q3pQHZJPpNiDVDQHphFS1lPwfmMt1mDFaAQ71DhmhlRaLm SSLz6wGxRjnkMkMxnVRNNSB7oD/2jn4jDxQNEHXMwW5Swg6HgXKyj8idJjFX3wBMZW1YDlV1 zFM9cCu0M5lGh/TC7z3SFJXzJZFu+B6IlrccrqGKdfSLvL5eRqFhlyjD5Cu7Fg/0n01ukkAC FbpiZBwdyLWYN8yd0BiE4FcgCf5XvDVN7IPiAypEIXfuz0oUzHrtou3lP1KpZ5+ps8onPwAA AAAAAA== --------------ms080707000903030302050808-- From nobody Mon Apr 22 22:05:15 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNfSn39bPz5Hlbp for ; Mon, 22 Apr 2024 22:05:29 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNfSn1MxTz58P5 for ; Mon, 22 Apr 2024 22:05:29 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-f45.google.com with SMTP id ca18e2360f4ac-7d9480d96bdso258157939f.1 for ; Mon, 22 Apr 2024 15:05:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713823527; x=1714428327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aAEqVlnUvuujTVODFsf3vstUM8uB9THIEIklkd8qHqg=; b=LkkkE3b7FFIoigFexddGpLV4iE+Hd0zJf9uNTyXpHipTqfGfx5PW2bcS0PYrT2mEx3 p/mxH5L8ofQhHmesDMr0maee2PW81vNPHkWE1zFvm4NkHy5dl2/zdj2hvrr1oWNN+eLU 6c2ySG/6H/wS1qgmPw3kbTp9ELZ/qZ+dbXnpINmegnxIodynnhZDea82wjrbWg1FNV0/ wFi9ORhUtuQTSgBKvbEZA1VLcvU825D2NWx7S8PiINMVCYHT3U5xIxmHon/kO131Ipyv 5X1jjti74ixN+0dt4A1vuA1mYxF6DOG25VyBTRdHwoA/2Ozz06HA41uNdE8QuUBeoi+w 95xw== X-Gm-Message-State: AOJu0YxsYQbNs7I4dDGBA32iJtwsf5S7hdrZecYH7Ub5WtrzgxcSgxMa 1UomsheTl3g0H7PyaWfXB50f+h9dWmDvIYlaYj30HmoqfZamu4cl3GaHD9GFBT1isdbFfcJSK0x sUcl1ofWhLFfp5IuvA2xoIIAEb6H1mg== X-Google-Smtp-Source: AGHT+IHRXD6OKzjbRkutsxblQMJnPBWM4DHRsDjQgtJ0H4O0pDyLUhHCydPoGXcZV6Kq16BJhRuqh2wVWXv45NlE2QQ= X-Received: by 2002:a92:4a02:0:b0:36b:ffc1:d1bf with SMTP id m2-20020a924a02000000b0036bffc1d1bfmr11792798ilf.18.1713823527516; Mon, 22 Apr 2024 15:05:27 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Mon, 22 Apr 2024 16:05:15 -0600 Message-ID: Subject: Re: Stressing malloc(9) To: Karl Denninger Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VNfSn1MxTz58P5 On Mon, Apr 22, 2024 at 2:07=E2=80=AFPM Karl Denninger = wrote: > > On 4/22/2024 12:46, Alan Somers wrote: > > When I said "33kiB" I meant "33 pages", or 132 kB. And the solution > turns out to be very easy. Since I'm using ZFS on top of geli, with > the default recsize of 128kB, I'll just set > vfs.zfs.vdev.aggregation_limit to 128 kB. That way geli will never > need to allocate more than 128kB contiguously. ZFS doesn't even need > those big allocations to be contiguous; it's just aggregating smaller > operations to reduce disk IOPs. But aggregating up to 1MB (the > default) is overkill; any rotating HDD should easily be able to max > out its consecutive write IOPs with 128kB operation size. I'll add a > read-only sysctl for g_eli_alloc_sz too. Thanks Mark. > > -Alan > > Setting this on one of my production machines that uses zfs behind geli d= rops the load average quite materially with zero impact on throughput that = I can see (thus far.) I will run this for a while but it certainly doesn't= appear to have any negatives associated with it and does appear to improve= efficiency quite a bit. Great news! Also, FTR I should add that this advice only applies to people who use HDDs. For SSDs zfs uses a different aggregation limit, and the default value is already low enough. -Alan From nobody Tue Apr 23 00:22:28 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNjVv4l06z5J1BV for ; Tue, 23 Apr 2024 00:22:31 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VNjVt6NnNz48vK; Tue, 23 Apr 2024 00:22:30 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jeffpc@josefsipek.net designates 71.174.62.3 as permitted sender) smtp.mailfrom=jeffpc@josefsipek.net Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 7D9332C489; Tue, 23 Apr 2024 00:22:29 +0000 (UTC) Date: Mon, 22 Apr 2024 20:22:28 -0400 From: Josef 'Jeff' Sipek To: Dimitry Andric Cc: Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.57 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; NEURAL_HAM_MEDIUM(-0.98)[-0.977]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; ONCE_RECEIVED(0.10)[]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_NA(0.00)[josefsipek.net]; TO_DN_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[3] X-Rspamd-Queue-Id: 4VNjVt6NnNz48vK On Mon, Apr 22, 2024 at 15:28:42 -0400, Josef 'Jeff' Sipek wrote: > On Mon, Apr 22, 2024 at 20:58:34 +0200, Dimitry Andric wrote: > > To properly finish up this thread, Jeff was right, and > > https://cgit.freebsd.org/src/commit/?id=da77a1b4f0dff was the cause. > > That commit added a .include at the top of libcxxrt's > > Makefile, which is normally fine, but not if you use SHLIBDIR?=/lib. > > That sort of assignment should always be done before including any of > > the bsd.*.mk files. > > > > I have committed https://cgit.freebsd.org/src/commit/?id=911a6479e18bc > > for now, which should fix the problem. It also adds an ObsoleteFiles.inc > > entry for /usr/lib/libcxxrt.so.1, so the file should be removed when you > > run "make delete-old-libs". > > FWIW, with this change, I just did a successful upgrade of a 14.0-RELEASE > directly to -CURRENT. I think a little bit more is needed to completely fix the issue. It looks like delete-old-libs gets rid of /usr/lib32/libcxxrt.so.1: root@odin# make delete-old-libs ... remove /usr/lib/debug/usr/lib/libcxxrt.so.1.debug? y remove /usr/lib32/libcxxrt.so.1? y remove /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug? y ... >>> Old libraries removed before installworld: root@odin# find / -name 'libcxxrt.*' -xdev -ls 35733 113 -r--r--r-- 1 root wheel 106712 Apr 18 20:31 /usr/lib/libcxxrt.so.1 35633 377 -r--r--r-- 1 root wheel 358086 Apr 18 20:31 /usr/lib/libcxxrt.a 103067 225 -r--r--r-- 1 root wheel 205880 Mar 23 08:48 /usr/lib/debug/lib/libcxxrt.so.1.debug 50292 217 -r--r--r-- 1 root wheel 157116 Apr 18 20:37 /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug 35002 241 -r--r--r-- 1 root wheel 208792 Apr 18 20:31 /usr/lib/debug/usr/lib/libcxxrt.so.1.debug 35735 1 lrwxr-xr-x 1 root wheel 13 Apr 18 20:31 /usr/lib/libcxxrt.so -> libcxxrt.so.1 50386 105 -r--r--r-- 1 root wheel 88116 Apr 18 20:37 /usr/lib32/libcxxrt.so.1 50291 321 -r--r--r-- 1 root wheel 220338 Apr 18 20:37 /usr/lib32/libcxxrt.a 50951 1 lrwxr-xr-x 1 root wheel 13 Apr 18 20:37 /usr/lib32/libcxxrt.so -> libcxxrt.so.1 102692 113 -r--r--r-- 1 root wheel 107512 Mar 23 08:48 /lib/libcxxrt.so.1 after installworld: root@odin# find / -name 'libcxxrt.*' -xdev -ls 35633 377 -r--r--r-- 1 root wheel 358086 Apr 18 20:31 /usr/lib/libcxxrt.a 101307 241 -r--r--r-- 1 root wheel 208792 Apr 22 19:59 /usr/lib/debug/lib/libcxxrt.so.1.debug 50292 217 -r--r--r-- 1 root wheel 157116 Apr 18 20:37 /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug 35002 241 -r--r--r-- 1 root wheel 208792 Apr 18 20:31 /usr/lib/debug/usr/lib/libcxxrt.so.1.debug 101309 1 lrwxr-xr-x 1 root wheel 23 Apr 22 19:59 /usr/lib/libcxxrt.so -> ../../lib/libcxxrt.so.1 50386 105 -r--r--r-- 1 root wheel 88116 Apr 18 20:37 /usr/lib32/libcxxrt.so.1 50291 321 -r--r--r-- 1 root wheel 220338 Apr 18 20:37 /usr/lib32/libcxxrt.a 117893 1 lrwxr-xr-x 1 root wheel 13 Apr 22 20:00 /usr/lib32/libcxxrt.so -> libcxxrt.so.1 101970 113 -r--r--r-- 1 root wheel 106712 Apr 22 19:59 /lib/libcxxrt.so.1 after delete-old-libs: root@odin# find / -name 'libcxxrt.*' -xdev -ls 35633 377 -r--r--r-- 1 root wheel 358086 Apr 18 20:31 /usr/lib/libcxxrt.a 101307 241 -r--r--r-- 1 root wheel 208792 Apr 22 19:59 /usr/lib/debug/lib/libcxxrt.so.1.debug 101309 1 lrwxr-xr-x 1 root wheel 23 Apr 22 19:59 /usr/lib/libcxxrt.so -> ../../lib/libcxxrt.so.1 50291 321 -r--r--r-- 1 root wheel 220338 Apr 18 20:37 /usr/lib32/libcxxrt.a 117893 1 lrwxr-xr-x 1 root wheel 13 Apr 22 20:00 /usr/lib32/libcxxrt.so -> libcxxrt.so.1 101970 113 -r--r--r-- 1 root wheel 106712 Apr 22 19:59 /lib/libcxxrt.so.1 Compared to what I see on 14.0-RELEASE, note that: 1. /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug is missing 2. /usr/lib32/libcxxrt.so.1 is missing 3. /usr/lib32/libcxxrt.so is a broken symlink (because of #2) At least I'm assuming that lib32 is still supposed to exist. (A repeated 'make installworld' recreates the missing files.) Jeff. From nobody Tue Apr 23 08:37:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VNwV254Pfz5Jktl for ; Tue, 23 Apr 2024 08:37:30 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VNwV20rRqz45mH; Tue, 23 Apr 2024 08:37:30 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; none List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1713861437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/UE9oHsref+oVEZ3QeL92GrSsMyUhYDMkeQPH7hAw/k=; b=HhzgvbqQbed84WBRlRHjYp2yi1ENv+AwEf8BFECQDmkr4EHr/TkHPvX5fahPkn9ktBomYR a6C2qqxg+v88d3zDOFFXsmghBnQczT2f0nr4WrUtoaxpLug4310vKO/oMOQl3w9FAFdszY r3s4tNHNgHh1JzfoDV1DN7l4N/sX0AmWNUNyGkMfN65fS2ySlC/26rmxyfH/72Ue+SjOYT TywYOQ2P4O/v/zWoby8MDRX3bV4+IMYRCn+6o3Fn2sXkyctglGDSJfrhOUSy3RInzOUykq 6Yz/vZnKF5oBCm49pyr/B1oEbYA8sdUXEw1U/MGTPsRl/NdEpLpuJwLXlgjV4A== Date: Tue, 23 Apr 2024 10:37:01 +0200 From: Alexander Leidinger To: Alan Somers Cc: Karl Denninger , freebsd-hackers@freebsd.org Subject: Re: Stressing malloc(9) In-Reply-To: References: Message-ID: <2b72c4f749e93dfec08a164d5a664ee3@Leidinger.net> Organization: No organization, this is a private message. Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_354318f0c1c8e56ac65f3fe01cb4c8cd"; micalg=pgp-sha256 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE] X-Rspamd-Queue-Id: 4VNwV20rRqz45mH This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_354318f0c1c8e56ac65f3fe01cb4c8cd Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Am 2024-04-23 00:05, schrieb Alan Somers: > On Mon, Apr 22, 2024 at 2:07 PM Karl Denninger > wrote: >> >> On 4/22/2024 12:46, Alan Somers wrote: >> >> When I said "33kiB" I meant "33 pages", or 132 kB. And the solution >> turns out to be very easy. Since I'm using ZFS on top of geli, with >> the default recsize of 128kB, I'll just set >> vfs.zfs.vdev.aggregation_limit to 128 kB. That way geli will never >> need to allocate more than 128kB contiguously. ZFS doesn't even need >> those big allocations to be contiguous; it's just aggregating smaller >> operations to reduce disk IOPs. But aggregating up to 1MB (the >> default) is overkill; any rotating HDD should easily be able to max >> out its consecutive write IOPs with 128kB operation size. I'll add a >> read-only sysctl for g_eli_alloc_sz too. Thanks Mark. >> >> -Alan >> >> Setting this on one of my production machines that uses zfs behind >> geli drops the load average quite materially with zero impact on >> throughput that I can see (thus far.) I will run this for a while but >> it certainly doesn't appear to have any negatives associated with it >> and does appear to improve efficiency quite a bit. > > Great news! Also, FTR I should add that this advice only applies to > people who use HDDs. For SSDs zfs uses a different aggregation limit, > and the default value is already low enough. You basically say, that it is not uncommon to have such large allocations with kernels we ship (even in releases). Wouldn't it make sense to optimize the kernel to handle larger uma allocations? Or do you expect it to be specific to ZFS and it may be more sane to discuss with the OpenZFS developers to reduce this default setting? Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_354318f0c1c8e56ac65f3fe01cb4c8cd Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=833 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmYnczkACgkQEg2wmwP4 2IZnIhAAk5V4Y5SVbxQf3ackPNLDIvb3/rxDo0+UgYi+3mQKcqAbxv469Jis3JTr fr4b/lO8tBtI19M9uJLRlAkXPjcI3emf78H4wh2YpTTnveJAKuv2wWiNWe/r4J5e MEwIDOfzfY3SCY0wrhwaTMUOWuS4F4paWy1A1lVdYGEhLy3j1TptuP0ddL3D2SJT 5QrpGoLoERZ0EnddeBIEpILPBvNVSUkHT5JWvGfp2bk+Zgc3STR1TjPSTPsknLwv qI6PkCXEN+3Cub1XM2O2uFkLSRr50M1T/pLZX1k76En3PBGKc+LcglFoQ7CM901s y6uN70j3vy6k6lLGfVPcZ96cQ+3aRppUbCh7HV9FcNwkQH5sRUNrQHyloXTFaoXM ghPQVUnAk/JEMTRlbHwSgPr6/0U42OZ9DVVpdWWmzazN/pbU5cYZoBk1jG1R8CDK wKz1kutQbb/M4oenR0MVeky7GyJJdv4/GDcLr7o0rhy1UIIzwdCQAi3le7oKEqSL uApeTR9VSfm2WDem4FXXXZlbm9pOqGwT1waskf+4YTUI92VoOF1I9Z2REtbOdAJe QnSZqwFE00hEVI2/bOr4c8sUouqQLXArIeHi4S7/23O2+gg4BAY1vKtX+f+d+MRN O2X/NTQZINRXcHKN4Ei9LyciV0x02g5ixWYzkuVZBeDlrsAm8k0= =zKch -----END PGP SIGNATURE----- --=_354318f0c1c8e56ac65f3fe01cb4c8cd-- From nobody Tue Apr 23 12:47:03 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VP22G436bz5HB05 for ; Tue, 23 Apr 2024 12:47:18 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vs1-f49.google.com (mail-vs1-f49.google.com [209.85.217.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VP22G0M5Rz4dxS for ; Tue, 23 Apr 2024 12:47:18 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vs1-f49.google.com with SMTP id ada2fe7eead31-47a21267aa8so1842632137.3 for ; Tue, 23 Apr 2024 05:47:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713876435; x=1714481235; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P+DbBTyPji2ua9XACFXO+uUyhQjQuv4z7zR49zUFxxY=; b=e+eNcyI1qzImO56r5LXHSoneO8DyTjp0RSkb2jOJvUKytAlzHF1v2YBzDRUKCjpWdR Y+jCSMx1CuuW4bSpoUk4YsONMJ/zYrTRJHawzGhFbPiv0UKjYX5LlLR2B4D0xEGZdDdN PUHACCEKOHb1wSUfr7MoA1OYAae6k7kW1RNBcoodYzGIgCMP1JcNdo9fAuCvdyH1yLxl SCH+SfWcs7r9mKNFlpBrYrgLKxBjnVuSmB5au9pDWeXDXmA5j0QszXiexviwy6BswkKE L9gQnfmDvboE9ILTytPFiMQFZhJ+iUWfLEqcUuCJdSM5cbSfmTcRhw9owvDnXQ8NQTmM VPfA== X-Forwarded-Encrypted: i=1; AJvYcCWZQS6wZdN1iCOeie95cq9MKneRnIg832l2S1yX5PkCXCYMaZXBwCax90S8TUlYzGJFH6t62U8egV1jpOYyp5Il3abVSuTorxPlpy0= X-Gm-Message-State: AOJu0Yw0A3JU1K8WtvPmYXX6JP4TTwEJ1N2MIFGG68krkXI17bgV4Qd9 wbPUTpNeUJw2hn0cU02rU1L4u6HI+xEGFLX5uPwb2xiqjmbB4VRfvKXfLnA0+DLhbLAGKJkXjkU pjWgg08pZz2f4VHDrzfWsyzG2LYwltQ== X-Google-Smtp-Source: AGHT+IHQWpwJZ+SRBPnsGgLATUjh8YNrYNIPWnPmc2VRYN2w1NevfT545i/ZsezSN65u+AGbIB9X4NUHh3sGCW6XsXk= X-Received: by 2002:a67:ce03:0:b0:47b:a44a:bad6 with SMTP id s3-20020a67ce03000000b0047ba44abad6mr13742435vsl.25.1713876434957; Tue, 23 Apr 2024 05:47:14 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <2b72c4f749e93dfec08a164d5a664ee3@Leidinger.net> In-Reply-To: <2b72c4f749e93dfec08a164d5a664ee3@Leidinger.net> From: Alan Somers Date: Tue, 23 Apr 2024 06:47:03 -0600 Message-ID: Subject: Re: Stressing malloc(9) To: Alexander Leidinger Cc: Karl Denninger , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VP22G0M5Rz4dxS On Tue, Apr 23, 2024 at 2:37=E2=80=AFAM Alexander Leidinger wrote: > > Am 2024-04-23 00:05, schrieb Alan Somers: > > On Mon, Apr 22, 2024 at 2:07=E2=80=AFPM Karl Denninger > > wrote: > >> > >> On 4/22/2024 12:46, Alan Somers wrote: > >> > >> When I said "33kiB" I meant "33 pages", or 132 kB. And the solution > >> turns out to be very easy. Since I'm using ZFS on top of geli, with > >> the default recsize of 128kB, I'll just set > >> vfs.zfs.vdev.aggregation_limit to 128 kB. That way geli will never > >> need to allocate more than 128kB contiguously. ZFS doesn't even need > >> those big allocations to be contiguous; it's just aggregating smaller > >> operations to reduce disk IOPs. But aggregating up to 1MB (the > >> default) is overkill; any rotating HDD should easily be able to max > >> out its consecutive write IOPs with 128kB operation size. I'll add a > >> read-only sysctl for g_eli_alloc_sz too. Thanks Mark. > >> > >> -Alan > >> > >> Setting this on one of my production machines that uses zfs behind > >> geli drops the load average quite materially with zero impact on > >> throughput that I can see (thus far.) I will run this for a while but > >> it certainly doesn't appear to have any negatives associated with it > >> and does appear to improve efficiency quite a bit. > > > > Great news! Also, FTR I should add that this advice only applies to > > people who use HDDs. For SSDs zfs uses a different aggregation limit, > > and the default value is already low enough. > > You basically say, that it is not uncommon to have such large > allocations with kernels we ship (even in releases). > Wouldn't it make sense to optimize the kernel to handle larger uma > allocations? > > Or do you expect it to be specific to ZFS and it may be more sane to > discuss with the OpenZFS developers to reduce this default setting? Yes, both of those things are true. It might make sense to reduce the setting's default value. OTOH, the current value is probably fine for people who don't use geli (and possibly other transforms that require allocating data). And it would also be good to optimize the kernel to perform these allocations more efficiently. My best idea is to teach g_eli_alloc_data how to allocate scatter/gather lists of 64k buffers instead of contiguous memory. The memory doesn't need to be contiguous, after all. But that's a bigger change, and I don't know that I have the time for it right now. -Alan From nobody Tue Apr 23 13:29:10 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VP2zb5mjgz5HFGS for ; Tue, 23 Apr 2024 13:30:03 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VP2zZ6NCTz4k9d; Tue, 23 Apr 2024 13:30:02 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; none List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1713878996; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X7qzkttSS5x3nuPyvAUEN1Wtd3tndBBtL79mJZxeHFM=; b=EuJeybRisZFlPH0samj27I3UgSImt7z/cch2bRy3qcfnZt49A/YwCcJUIfcCQR33FN4JZ9 q7iwOSiDenl3MLjYUk9fCK8ytaDPoNYb+zCZUyoI3/QSbStXlHUiH/5JiEWt7Ww5GdYFIF 0BsDtiox+8Ine/pVdXl6Smscc8e13SKtcctcCIR4/BdCvYdBpc4sX9/xXKL2vbzAl8fuQj gBiMF5EQCuOGz8y7jDWq21ihFePUBJBc8PhNZnt9dg3za0qqwIUUexTYaONGmHXfnM8dZ9 V6W6L4fmzDtbzc/I4RYwrQdLr9bfXomsRAEUlTo3rkS87c0zw/vw1adPkCAtbQ== Date: Tue, 23 Apr 2024 15:29:10 +0200 From: Alexander Leidinger To: Alan Somers Cc: Karl Denninger , freebsd-hackers@freebsd.org Subject: Re: Stressing malloc(9) In-Reply-To: References: <2b72c4f749e93dfec08a164d5a664ee3@Leidinger.net> Message-ID: <5abe7eb5b80ab164b91c858bfd8121d7@Leidinger.net> Organization: No organization, this is a private message. Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_ce155aa0aa94a110feb1fb8ce418aa06"; micalg=pgp-sha256 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34240, ipnet:89.238.64.0/18, country:DE] X-Rspamd-Queue-Id: 4VP2zZ6NCTz4k9d This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_ce155aa0aa94a110feb1fb8ce418aa06 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Am 2024-04-23 14:47, schrieb Alan Somers: > On Tue, Apr 23, 2024 at 2:37 AM Alexander Leidinger > wrote: >> You basically say, that it is not uncommon to have such large >> allocations with kernels we ship (even in releases). >> Wouldn't it make sense to optimize the kernel to handle larger uma >> allocations? >> >> Or do you expect it to be specific to ZFS and it may be more sane to >> discuss with the OpenZFS developers to reduce this default setting? > > Yes, both of those things are true. It might make sense to reduce the > setting's default value. OTOH, the current value is probably fine for > people who don't use geli (and possibly other transforms that require > allocating data). And it would also be good to optimize the kernel to > perform these allocations more efficiently. My best idea is to teach > g_eli_alloc_data how to allocate scatter/gather lists of 64k buffers > instead of contiguous memory. The memory doesn't need to be > contiguous, after all. But that's a bigger change, and I don't know > that I have the time for it right now. > -Alan Do you have time do make a nice description of what would have to be done in the wiki? https://wiki.freebsd.org/IdeasPage Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_ce155aa0aa94a110feb1fb8ce418aa06 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=833 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmYnt7MACgkQEg2wmwP4 2Ibk7BAAgj+l/Q2jXVsJdzNRWFmF9jacdO1htchyUU3nNjbi1q56PMjNdpLpRztw IRy6ewJUeTBg61C1xAAToaPKnYWgJFdbGanTa5T3+PTGXhu9OatFZyY6enVCOAsE DM06RHQa+odUeuJGKmT4BBpGaAbYAtBuQVjAWi1zM0c7XxH/WkZ31l6NfcTXGzuA sLpBXpFe7T18zoPTpC8/UTsQjKNmxILd5gzVQDV6E5+M/53S7r5Rt0a8oJ0P8kSb FQzSDMPV/umzeumexvVC6n3XZ6rJPTU1J8W3YnF6d9lgeu3sx7UJmHnCkTqEGlkN MaCBy+98R1uH57F44OJ/nm5j3+OfV7/ubditUcE1XWQ9ZGwLjMasNGhbpFGx8an/ p7y/ZYHEBpgQkLJ3RtFuswDnlqfy8P1b9yDA7wKj1V/+fbaFxMO7x3/+5fgNgASK IRINGAzRuAr+UtPqw5pBN6oVuboRPkBzNd4fNICDVb5w+TzOp6sNeI+g8wInRn7r gvuvbrijxxrdsPBvODP4mQH++2uczF+TVQ9avVrKYpuKVTUQG7G7jhEQlv+Ra+Lg E2uMYG7phnd8mXOAOGF2hkViasf+5IIJICeArGeXKzdXWEesLJDnw8WfATWTW90a OfcrqN+JYBV4nKS5grloA+gybR5HvMNv0ReazTMBvKGmkmmWbrE= =0jZa -----END PGP SIGNATURE----- --=_ce155aa0aa94a110feb1fb8ce418aa06-- From nobody Tue Apr 23 17:45:21 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VP8fB2wsRz5HjRC for ; Tue, 23 Apr 2024 17:45:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VP8fB2NMWz4LjQ; Tue, 23 Apr 2024 17:45:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713894322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b5AvOpcl/gYFEer0My6H8zJRqq9amILO21zhu2q/VQ4=; b=FQkomWQzRiDrkk6vEAO/RGBVnnzSCw05S+eiDZECtwDE7pDoROTIb1Arh0biEPcEGkfe1L JlPrO+mDnWqsk3QpJakw/1OBC+uN6yTDgiN1BBemxBdUJyAGyLMEy68IcrKRcbov9q3hFq 7ZZV1Db6HQGRJ53hc6aIEhke9QQ2gHKrvuIY6zp29C1KjxYnef2o9jF0tTwckMqgLRTUUM f7ENOsj+auqal4RS2QNMYKdhXCwwRYZTVudzMKo8IRaGH3ynEOzaDGr1Plh6W2XcbKiExA qtcCPf1SsTceW7ok+yMqbypjM8l+oX7ZXfkHO1muGoEvxmt7ZmJ7wuQBj3jI5Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713894322; a=rsa-sha256; cv=none; b=JuAWIpsIUeuYlA24ZYIYTlZeZF0yzpejYfurmsX9iNY8skAdD4vagXVRnPFng+zPvMm0Hg UetTgu7ACJuTr8hbBQPTKgLdGIi+HTpOivowePEFFlt6Z/71AW7kLmg6deiBSreCXu2RbP iON0tPgu32KmJARzP+JLDLwFK+uxdwUH+TouGMHHSZC2HD9qyvVT295mKRAVi2iu9Oo6Ci MT2q1lHRjVwOYPwdsJxRdE4CzIZ6hoGqy9Gw2eOBLTyMgU9idVw85AgVMcaLOZcOpfh/N2 pJfNMRcRSHuNNSuwPt8Ba9bK/0shtL7Zu6yaBANzQxJoeGXjpkknnmfemvis4Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713894322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b5AvOpcl/gYFEer0My6H8zJRqq9amILO21zhu2q/VQ4=; b=dh7THwn/CjwltE580WQUYcZWCI6iUZ1rBennVRYRS6mmEm/x8glRKu933qhBmd3nQjRl7k PuTGKky63TAKMID/TZlNYN3JnGe5OxdhhMmvcGM8+TtZT9r2mNXRFqa+vxZ8rqvmCi2q/+ 6dXWN5nPR4UZgWQ4IOK0U/Cr8Y2A2yrdy2ZzurCpOd+VJupnYOIJkixu2zxzo8g9FhVWZ7 UBWtZOlHZOy9tpSrgIP0E8dqbSzJgfae5/pxl1T6JfCgSjHYAXLP99EG2o/M6KymOs/amo ZaoeAOtWnR+VMMfTolhLauNfEQt6YdmjxIHpU1LFGBScLu+wxAmJtqhs/htRHQ== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VP8fB0znbzh0Y; Tue, 23 Apr 2024 17:45:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 37D421F37B; Tue, 23 Apr 2024 19:45:21 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Upgrading -RELEASE to -CURRENT From: Dimitry Andric In-Reply-To: Date: Tue, 23 Apr 2024 19:45:21 +0200 Cc: Warner Losh , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> To: Josef 'Jeff' Sipek X-Mailer: Apple Mail (2.3731.700.6.1.1) On 23 Apr 2024, at 02:22, Josef 'Jeff' Sipek = wrote: >=20 > On Mon, Apr 22, 2024 at 15:28:42 -0400, Josef 'Jeff' Sipek wrote: >> On Mon, Apr 22, 2024 at 20:58:34 +0200, Dimitry Andric wrote: >>> To properly finish up this thread, Jeff was right, and >>> https://cgit.freebsd.org/src/commit/?id=3Dda77a1b4f0dff was the = cause. >>> That commit added a .include at the top of libcxxrt's >>> Makefile, which is normally fine, but not if you use SHLIBDIR?=3D/lib.= >>> That sort of assignment should always be done before including any = of >>> the bsd.*.mk files. >>>=20 >>> I have committed = https://cgit.freebsd.org/src/commit/?id=3D911a6479e18bc >>> for now, which should fix the problem. It also adds an = ObsoleteFiles.inc >>> entry for /usr/lib/libcxxrt.so.1, so the file should be removed when = you >>> run "make delete-old-libs". >>=20 >> FWIW, with this change, I just did a successful upgrade of a = 14.0-RELEASE >> directly to -CURRENT. >=20 > I think a little bit more is needed to completely fix the issue. It = looks > like delete-old-libs gets rid of /usr/lib32/libcxxrt.so.1: >=20 > root@odin# make delete-old-libs > .. > remove /usr/lib/debug/usr/lib/libcxxrt.so.1.debug? y > remove /usr/lib32/libcxxrt.so.1? y > remove /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug? y Ok, that should hopefully be fixed by: https://cgit.freebsd.org/src/commit/?id=3Df48643d376a4 Thanks to jhb@ for suggesting MOVED_LIBS, which should handle this = 'transition' smoothly. -Dimitry =20= From nobody Tue Apr 23 21:58:53 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VPGH101TQz5J8my for ; Tue, 23 Apr 2024 21:59:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VPGH04qh6z509w for ; Tue, 23 Apr 2024 21:59:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a588fba6c78so15357966b.2 for ; Tue, 23 Apr 2024 14:59:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713909546; x=1714514346; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ZDSzqdx0sV5ZK1nKKfWoGCthT4U7JdRSc4QZldGi05I=; b=OPev8bYAMF39uarT3vZHnokTLzGQLvy8gfBTrhBlomgbb+Zwz3YYfHCn7ZZ/uA9PPO qleAPyANj0c2eDsDa+Fo48INMD9TPdUnOo0u9IPAbAlbIHexq1z7r8h//VFWeF/RdjVL GrqCdgrZuuPuRo1e3dEmzpGSXQw0ZB5PdkUvJOdtCaDOIRnWH07y0OLhfV8EJkknVnvY hlCEKSgOlgqAUKk+00NDQbuNZIm/J3CKPXiEYz/0hP10URZgs7ydA8/V0D/JA+23IHBB 7vXAhe7eFtNsuv33dcrDpQmvmbcE/zeTcc6vRQT26x8EW4PiAZq2SEYU7Um42gkvpIzb UCXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713909546; x=1714514346; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZDSzqdx0sV5ZK1nKKfWoGCthT4U7JdRSc4QZldGi05I=; b=vlKJJ8uZW5mT/t7yvGwRDdawg+8VzfreNq8pnZxxpiNfYDrCjMKLcm1fC39ce/AeuD Yol8hbHpN3iPzTnJRNmAr2qL2n9im8keiEd+SHWT8cmtsuGcWiXlkgK0ORRqTBdvEwxB JwoglsG5neBC4dPDWEsP4bvoebwu5F3YYwOcFeR8QvEDF3Frvi0gDqL1zBlwOFv13kNd q1HRIykOyHaY433RQnNa6tcJHg9FTMS1151+w9ck8JSkXVrCjDMGi0b0JSuUkobNaT63 /qOdfXby2ZeagQdMrxR3A1So1BfmA1KBqriGUDyZSJk4e9CgBnx2J8kZGd3uFxATAAUz mOlw== X-Forwarded-Encrypted: i=1; AJvYcCWbQFg/UXSfssJRiFW5IJSK9h1qPs06LnhnoONkoBIBllcl4JnuSWO/UhDMMgcy2vvnzFYcDvYaVC1eD8N/DwSnTwo9t5ckyhHnhXs= X-Gm-Message-State: AOJu0Yzd4CrT+jiXl3kMAM5cMViBE4IEpb1GhRo5VHW0XYafOmgkc29l 3deVBehKMuxtFUOI1rwMCI7jozo1PpkyxJ26EH9ysoq7PKxgc5I2taU4hZesGBJD47yAt652mFV 8enPK/rD43GgPG+CIv6h9yI5t2m/CDuTBRW1jwDOuBkGRCySK+AL9eA== X-Google-Smtp-Source: AGHT+IHZ3DoCLs6jatUYIJS/LNZb+/2lOPAP/GBhQ9maugqOJJUsHa8YpT+Yj5wg8OUfSygs+B5vQy1IRHc+OdvNv84= X-Received: by 2002:a17:906:f1d1:b0:a52:1432:b790 with SMTP id gx17-20020a170906f1d100b00a521432b790mr352510ejb.31.1713909545853; Tue, 23 Apr 2024 14:59:05 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <2b72c4f749e93dfec08a164d5a664ee3@Leidinger.net> <5abe7eb5b80ab164b91c858bfd8121d7@Leidinger.net> In-Reply-To: <5abe7eb5b80ab164b91c858bfd8121d7@Leidinger.net> From: Warner Losh Date: Tue, 23 Apr 2024 15:58:53 -0600 Message-ID: Subject: Re: Stressing malloc(9) To: Alexander Leidinger Cc: Alan Somers , Karl Denninger , freebsd-hackers@freebsd.org Content-Type: multipart/alternative; boundary="000000000000a882910616caa977" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VPGH04qh6z509w --000000000000a882910616caa977 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Apr 23, 2024 at 7:30=E2=80=AFAM Alexander Leidinger wrote: > Am 2024-04-23 14:47, schrieb Alan Somers: > > On Tue, Apr 23, 2024 at 2:37=E2=80=AFAM Alexander Leidinger > > wrote: > > >> You basically say, that it is not uncommon to have such large > >> allocations with kernels we ship (even in releases). > >> Wouldn't it make sense to optimize the kernel to handle larger uma > >> allocations? > >> > >> Or do you expect it to be specific to ZFS and it may be more sane to > >> discuss with the OpenZFS developers to reduce this default setting? > > > > Yes, both of those things are true. It might make sense to reduce the > > setting's default value. OTOH, the current value is probably fine for > > people who don't use geli (and possibly other transforms that require > > allocating data). And it would also be good to optimize the kernel to > > perform these allocations more efficiently. My best idea is to teach > > g_eli_alloc_data how to allocate scatter/gather lists of 64k buffers > > instead of contiguous memory. The memory doesn't need to be > > contiguous, after all. But that's a bigger change, and I don't know > > that I have the time for it right now. > > -Alan > > Do you have time do make a nice description of what would have to be > done in the wiki? > https://wiki.freebsd.org/IdeasPage I've added the super-brief verrsion to https://wiki.freebsd.org/WarnerLosh which has my crazy ideas list... Warner --000000000000a882910616caa977 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, Apr 23, 2024 at 7:30=E2=80=AF= AM Alexander Leidinger <Alexa= nder@leidinger.net> wrote:
Am 2024-04-23 14:47, schrieb Alan Somers:
> On Tue, Apr 23, 2024 at 2:37=E2=80=AFAM Alexander Leidinger
> <Alexa= nder@leidinger.net> wrote:

>> You basically say, that it is not uncommon to have such large
>> allocations with kernels we ship (even in releases).
>> Wouldn't it make sense to optimize the kernel to handle larger= uma
>> allocations?
>>
>> Or do you expect it to be specific to ZFS and it may be more sane = to
>> discuss with the OpenZFS developers to reduce this default setting= ?
>
> Yes, both of those things are true.=C2=A0 It might make sense to reduc= e the
> setting's default value.=C2=A0 OTOH, the current value is probably= fine for
> people who don't use geli (and possibly other transforms that requ= ire
> allocating data).=C2=A0 And it would also be good to optimize the kern= el to
> perform these allocations more efficiently.=C2=A0 My best idea is to t= each
> g_eli_alloc_data how to allocate scatter/gather lists of 64k buffers > instead of contiguous memory.=C2=A0 The memory doesn't need to be<= br> > contiguous, after all.=C2=A0 But that's a bigger change, and I don= 't know
> that I have the time for it right now.
> -Alan

Do you have time do make a nice description of what would have to be
done in the wiki?
=C2=A0 =C2=A0 =C2=A0https://wiki.freebsd.org/IdeasPage

I've added the super-brief verrsion to=C2=A0https://wiki.freebsd.org/Warn= erLosh which has my crazy ideas list...=C2=A0

= Warner
--000000000000a882910616caa977-- From nobody Wed Apr 24 13:57:33 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VPgXv2tgkz5HPvc for ; Wed, 24 Apr 2024 13:57:35 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Received: from smtp.jeffnet.31bits.net (josefsipek.net [71.174.62.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4VPgXv2PN1z4Tn9; Wed, 24 Apr 2024 13:57:35 +0000 (UTC) (envelope-from jeffpc@josefsipek.net) Authentication-Results: mx1.freebsd.org; none Received: from satis (satis [172.27.0.85]) by smtp.jeffnet.31bits.net (Postfix) with ESMTPSA id 1D3372C761; Wed, 24 Apr 2024 13:57:35 +0000 (UTC) Date: Wed, 24 Apr 2024 09:57:33 -0400 From: Josef 'Jeff' Sipek To: Dimitry Andric Cc: Warner Losh , FreeBSD Hackers Subject: Re: Upgrading -RELEASE to -CURRENT Message-ID: References: <9C0C7B9F-9780-4B7D-9B46-CD83B516BD2A@FreeBSD.org> <849819B4-9EE6-4776-8079-B0E12ED457D0@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:701, ipnet:71.174.0.0/16, country:US] X-Rspamd-Queue-Id: 4VPgXv2PN1z4Tn9 On Tue, Apr 23, 2024 at 19:45:21 +0200, Dimitry Andric wrote: > On 23 Apr 2024, at 02:22, Josef 'Jeff' Sipek wrote: > > > > On Mon, Apr 22, 2024 at 15:28:42 -0400, Josef 'Jeff' Sipek wrote: > >> On Mon, Apr 22, 2024 at 20:58:34 +0200, Dimitry Andric wrote: > >>> To properly finish up this thread, Jeff was right, and > >>> https://cgit.freebsd.org/src/commit/?id=da77a1b4f0dff was the cause. > >>> That commit added a .include at the top of libcxxrt's > >>> Makefile, which is normally fine, but not if you use SHLIBDIR?=/lib. > >>> That sort of assignment should always be done before including any of > >>> the bsd.*.mk files. > >>> > >>> I have committed https://cgit.freebsd.org/src/commit/?id=911a6479e18bc > >>> for now, which should fix the problem. It also adds an ObsoleteFiles.inc > >>> entry for /usr/lib/libcxxrt.so.1, so the file should be removed when you > >>> run "make delete-old-libs". > >> > >> FWIW, with this change, I just did a successful upgrade of a 14.0-RELEASE > >> directly to -CURRENT. > > > > I think a little bit more is needed to completely fix the issue. It looks > > like delete-old-libs gets rid of /usr/lib32/libcxxrt.so.1: > > > > root@odin# make delete-old-libs > > .. > > remove /usr/lib/debug/usr/lib/libcxxrt.so.1.debug? y > > remove /usr/lib32/libcxxrt.so.1? y > > remove /usr/lib/debug/usr/lib32/libcxxrt.so.1.debug? y > > Ok, that should hopefully be fixed by: > https://cgit.freebsd.org/src/commit/?id=f48643d376a4 Yep, that fixes it. Thanks! Jeff. From nobody Sat Apr 27 13:08:40 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VRVKs04WLz5J54c; Sat, 27 Apr 2024 13:09:20 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VRVKr1dtHz4kc4; Sat, 27 Apr 2024 13:09:20 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=bDSFMYl3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2607:f8b0:4864:20::102f as permitted sender) smtp.mailfrom=marietto2008@gmail.com Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2a78c2e253aso2464663a91.3; Sat, 27 Apr 2024 06:09:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714223357; x=1714828157; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=fAMeeuH9xTI68X+n0CL+rtcXC18ozA2zytiDh4k8Sd4=; b=bDSFMYl30pQwRTprpzgIgRqGeIJLO4zXFvuhkUPEQ6AV2yGshM9j+Hw6TRi2GchkLV QCHxwMfdpT1egWeCSjfvF62gNQtv658VAsUMJxrMgRETxo41rqjf6MOBLUdiprYPPV8l urlXbS25T5eGUxHQtiSoX+ybSDvixzcgcLzrCMhDmfd4VX5/bMQPXmomR49VkeXls6xI t9VtzYF2pmUyaDGjSvfclUCAZOC76f0JIeZClTZ5VEuW8kwMpf/Seq5K/TQi1JEs0BZ7 XujjzDisGicQz1gB81y7lSjoP24WZ0MbtqKumc/DPIo1K0DfhTM3YVBDV0bP0orvUP6N vCfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714223357; x=1714828157; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fAMeeuH9xTI68X+n0CL+rtcXC18ozA2zytiDh4k8Sd4=; b=onnR8Z0Um1Y+atr/6vznxdB7m/BS8LvuchpcFq0ULMD5arOaOK9EbQhB1BoOjrYxMC cjPCyY27/P+A9FbWe/KVVe+09eKpnSrWRByPF6GNsnRhq/MmpbFTFxABXl0Lpp18GkJo j2erAIfxHxvDo0bmss7CSYgIYtKM/bDotrlHx7qrtJYZIjdr5veWuFOcLvzuWmwstGDg xzojxdPigd/Rk5VNT3hpTw8c1bL7q8J1T9aOxpo6rgzxoA8abqB85rM5G2a/d2aMYQDy KayosdUMsV8e5K52ktJD612/AhUhnR3Ce5ooxzTXvh/fmh/7cfTXRnhQIUQC4GFMERSw E4LQ== X-Forwarded-Encrypted: i=1; AJvYcCUstcElrxMv2rIN9SzOcjf1iVRoecwj7tT6+gK9JQM1He3BngWyxGTfgm6NRaolQQtfusulv5IhzZCW8wH6bcP5Ea3F3EH01M2wn4qp4u0NrL/Zi61wHL5+ZvsaBu4HIWldEmtE/1pXtdbewezIeiP1BD2RLTCIczaR/cbHzN0HcGKCKxtnh79DbLXulzFnsymBa2xsx+5DyhyH49KCO1fp7L8WJ0RP8w== X-Gm-Message-State: AOJu0Yw7DqLQJp1/8ZAo07m0HJ/7lT+B7CMa6tn+CgQl3+Q2Ya/Z0J/G uW6+I2nvmTEK4sr+bwz2LzHzSeWDjOFDDBAAmHYeNoEG/ssVHaStsuQWccSoPiHHC9buYUWQa9O K6WuZn2fT5wi2sx5Md1DDLJWV3rNKHHoENOeWfA== X-Google-Smtp-Source: AGHT+IHKIAQqg3sGjnguTJtMWacxARchXVteXjnrfBnmyZO9/76O8Ah60Dz5pJvv4j/dK1kwc7Rtb6aSUeVLfqcKXmA= X-Received: by 2002:a17:90a:654c:b0:2ab:b480:5018 with SMTP id f12-20020a17090a654c00b002abb4805018mr5740421pjs.34.1714223357135; Sat, 27 Apr 2024 06:09:17 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: Mario Marietto Date: Sat, 27 Apr 2024 15:08:40 +0200 Message-ID: Subject: How to use Virtio GPU on FreeBSD as guest OS. To: freebsd-drivers@freebsd.org, freebsd-x11@freebsd.org, freebsd-hackers , FreeBSD Mailing List , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000449a38061713baff" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.975]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::102f:from]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-drivers@freebsd.org,freebsd-x11@freebsd.org,freebsd-hackers@freebsd.org,freebsd-questions@freebsd.org,freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_FIVE(0.00)[5] X-Rspamd-Queue-Id: 4VRVKr1dtHz4kc4 --000000000000449a38061713baff Content-Type: text/plain; charset="UTF-8" Hello. I've virtualized FreeBSD 14 on Windows 11 with qemu using the Hyper-V as a hypervisor. The parameters that I've used to launch the vm are the following ones : qemu-system-x86_64w.exe -accel whpx -machine q35 \ -cpu kvm64,hv_relaxed,hv_time,hv_synic -m 8G -vga virtio \ -display gtk,gl=on -audiodev dsound,id=snd0 -device ich9-intel-hda \ -device hda-duplex,audiodev=snd0 \ -hda "I:\Backup\FreeBSD\FreeBSD-140-zfs.img" \ -drive file=\\.\PhysicalDrive8 -rtc base=localtime \ -device usb-ehci,id=usb,bus=pcie.0,addr=0x3 -device usb-tablet \ -device usb-kbd -smbios type=2 -nodefaults \ -netdev tap,id=mynet0,ifname="OpenVPN-TAP-Windows",script=no,downscript=no \ -device e1000,netdev=mynet0,mac=52:55:00:d1:55:01 \ -device ich9-ahci,id=sata \ -bios "I:\OS\qemu\FreeBSD\OSX-KVM-master\OVMF_combined.fd" as you can see as graphic adapter I've added : -vga virtio -display gtk,gl=on That's because I want to use the virtio GPU instead of the VMware SVGA,but I'm not able to make it work. On FreeBSD 14.0 guest os I did : # lspci 00:01.0 : VGA compatible controller : Red Hat Inc. Virtio 1.0 GPU (rev. 01) and then,I've added on /boot/loader.conf the following kernel modules : virtio_load="YES" virtio_pci_load="YES" virtio_blk_load="YES" virtio_balloon_load="YES" I tried to load the virtio kernel modules manually : [root@marietto /home/marietto]==> kldload virtio kldload: can't load virtio: module already loaded or in kernel [root@marietto /home/marietto]==> kldload virtio_pci kldload: can't load virtio_pci: module already loaded or in kernel [root@marietto /home/marietto]==> kldload virtio_blk kldload: can't load virtio_blk: module already loaded or in kernel [root@marietto /home/marietto]==> kldload virtio_balloon kldload: can't load virtio_balloon: module already loaded or in kernel At this point,I've tried to use two different xorg.conf files to see what happened : nano /etc/X11/xorg.conf : Section "Device" Identifier "Card0" Driver "modesetting" BusID "PCI:0:1:0" Xorg.1.log.modesetting : https://pastebin.ubuntu.com/p/JYbks5yNnV/ nano /etc/X11/xorg.conf : Section "Device" Identifier "Card0" Driver "virtio" BusID "PCI:0:1:0" Xorg.1.log.virtio : https://pastebin.ubuntu.com/p/tt9Pnd5Zz4/ None of them worked. Can you give some suggestions ? FULL thread : https://forums.freebsd.org/threads/how-to-virtualize-freebsd-14-release-as-a-vm-on-top-of-windows-11-using-qemu-hyperv.93158/#post-652770 Mario. --000000000000449a38061713baff Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hello.

I've virtualized FreeBSD 14 on Windows 11 with qemu using the Hyper= -V as a hypervisor.

The parameters that I've used to launch the vm are the following on= es :=C2=A0 


qemu-system-x86_64w.exe -accel whpx -machine q35 \
-cpu kvm64,hv_relaxed,hv_time,hv_synic -m 8G -vga virtio \
-display gtk,=
gl=3Don -audiodev dsound,id=3Dsnd0 -device ich9-intel-hda \
-device hda-=
duplex,audiodev=3Dsnd0 \
-hda "I:\Backup\FreeBSD\FreeBSD-140-zfs.im=
g" \
-drive file=3D\\.\PhysicalDrive8 -rtc base=3Dlocaltime \
-d=
evice usb-ehci,id=3Dusb,bus=3Dpcie.0,addr=3D0x3 -device usb-tablet \
-device usb-kbd -smbios type=3D2 -nodefaults \
-netdev tap,id=3Dmynet0,ifname=3D"OpenVPN-TAP-Windows",script=3Dn=
o,downscript=3Dno \
-device e1000,netdev=3Dmynet0,mac=3D52:55:00:d1:55:01 \
-device ich9-ahc=
i,id=3Dsata \
-bios "I:\OS\qemu\FreeBSD\OSX-KVM-master\OVMF_combine=
d.fd"

=20


as you can see as graphic adapter I've added :


=20 

-vga virtio -display gtk,gl=3Don

That's because I want to use the virtio GPU instead of the VMware= =20 SVGA,but I'm not able to make it work. On FreeBSD 14.0 guest os I did :

=20 


# lspci=20
00:01.0 : VGA compatible controller : Red Hat Inc. Virtio 1.0 GPU (rev. 01)=

=20


and then,I've added on /boot/loader.conf the following kerne= l modules : 


virtio_load=3D"YES"=20
virtio_pci_load=3D"YES"=20
virtio_blk_load=3D"YES"=20
virtio_balloon_load=3D"YES"

=20


I tried to load the virtio kernel modules manually :

=20 


[root@marietto /home/marietto]=3D=3D> kldload virtio kldl=
oad: can't load virtio: module=20
already loaded or in kernel
 =20
[root@marietto /home/marietto]=3D=3D> kldload virtio_pci kldload: can=
9;t load virtio_pci:=20
module already loaded or in kernel =20

[root@marietto /home/marietto]=3D=3D> kldload virtio_blk kldload: can=
9;t load virtio_blk:=20
module already loaded or in kernel =20

[root@marietto /home/marietto]=3D=3D> kldload virtio_balloon kldload: ca=
n't load=20
virtio_balloon: module already loaded or in kernel


At this point,I've tried to use two different xorg.conf file= s to see what happened : 


nano /etc/X11/xorg.conf : =20

Section "Device"=20
Identifier "Card0"=20
Driver "modesetting"=20
BusID "PCI:0:1:0"

Xorg.1.log.modesetting : https://pastebin.ubuntu.com/p/JYbks5yNnV/

=20 


nano /etc/X11/xorg.conf : =20

Section "Device"=20
Identifier "Card0"=20
Driver "virtio"=20
BusID "PCI:0:1:0"

Xorg.1.log.virtio : h=
ttps://pastebin.ubuntu.com/p/tt9Pnd5Zz4/

=20


None of them worked. Can you give some suggestions ?


FULL thread :

https://forums.fr= eebsd.org/threads/how-to-virtualize-freebsd-14-release-as-a-vm-on-top-of-wi= ndows-11-using-qemu-hyperv.93158/#post-652770

=C2=A0
Mario.
--000000000000449a38061713baff-- From nobody Sun Apr 28 19:33:45 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VSGrt32rsz5JqLp for ; Sun, 28 Apr 2024 19:35:26 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "weser.webweaving.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VSGrs1htvz3xsd for ; Sun, 28 Apr 2024 19:35:25 +0000 (UTC) (envelope-from dirkx@webweaving.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=webweaving.org header.s=shared header.b=aV46udmv; dmarc=pass (policy=none) header.from=webweaving.org; spf=pass (mx1.freebsd.org: domain of dirkx@webweaving.org designates 148.251.234.232 as permitted sender) smtp.mailfrom=dirkx@webweaving.org Received: from smtpclient.apple (fiber.static.cbizz.nl [185.142.248.117] (may be forged)) (authenticated bits=0) by weser.webweaving.org (8.17.1/8.17.1) with ESMTPA id 43SJXw4J094640 for ; Sun, 28 Apr 2024 21:34:00 +0200 (CEST) (envelope-from dirkx@webweaving.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=webweaving.org; s=shared; t=1714332841; bh=5Ty3LbfUQu1YQL94eEMdJF9seL8sXOHCyp4VJkil5C4=; h=From:Subject:Date:To; b=aV46udmvsoe80y7knt8wIzpHuNA8l2n0E4R1V8V1gOJqF3XlcOmvlM15lBP5wWqE1 RLkrkLGve0+C1Rtmu/uu6FXzZhKOGDg/1rbV05TUAEe8eqWmHa99pVP/FSzNQEA3wq 7z6rXmRo87b9i9XAdHpXg1pRUobrXY7tFFA9np90= X-Authentication-Warning: weser.webweaving.org: Host fiber.static.cbizz.nl [185.142.248.117] (may be forged) claimed to be smtpclient.apple From: Dirk-Willem van Gulik Content-Type: multipart/alternative; boundary="Apple-Mail=_2C77F2BB-87C7-4F5B-A2B8-5CD33CED02FF" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Multicast & Tunnel devices Message-Id: Date: Sun, 28 Apr 2024 21:33:45 +0200 To: FreeBSD Hackers X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (weser.webweaving.org [148.251.234.232]); Sun, 28 Apr 2024 21:34:01 +0200 (CEST) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.40 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[webweaving.org,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[webweaving.org:s=shared]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ONCE_RECEIVED(0.10)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:24940, ipnet:148.251.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_XAW(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[webweaving.org:+] X-Rspamd-Queue-Id: 4VSGrs1htvz3xsd --Apple-Mail=_2C77F2BB-87C7-4F5B-A2B8-5CD33CED02FF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Would anyone know if there is something special with tunnel devices and = multicast ?=20 I=E2=80=99ve got some code that happily processes multicast packets on a = normal interface; but appears not to do this on a tunnel interface. Tun0 = shows multicast enabled: =09 tun0: flags=3D8043 metric 0 mtu = 1500 Tcpdump on that interface gives the expected thing (here with mDNS): tcpdump -n -i tun0 port 5353 listening on tun0, link-type NULL (BSD loopback), capture size = 262144 bytes 19:26:03.976259 IP 10.31.0.6.5353 > 224.0.0.251.5353: 0 PTR = (QM)? _raop._tcp.local. (34) And code, with a simple IP_ADD_MEMBERSHIP of the MC group on the IP of = the local interface below works on a normal interface (e.g. = igb0/10.0.0.1/24).=20 ./listener 10.0.0.1 224.0.0.251 5353 Received packet, len=3D128 etc But yields no output if ran against above tun0 interface (while tcpdump = on same is fine). Does that ring a bell with anyone ? Dw int main(int argc, char *argv[]) { struct sockaddr_in addr; struct ip_mreq mreq; // skip error trapping command line arguments char* ip =3D argv[1];=20 char* group =3D argv[2];=20 int port =3D atoi(argv[3]); // 0 if error, which is an invalid port memset(&addr, 0, sizeof(addr)); addr.sin_family =3D AF_INET; addr.sin_addr.s_addr =3D htonl(INADDR_ANY); addr.sin_port =3D htons(port); mreq.imr_interface.s_addr =3D inet_addr(ip);=20 mreq.imr_multiaddr.s_addr =3D inet_addr(group); // skip error trapping on inet_addr int fd =3D socket(AF_INET, SOCK_DGRAM, 0); // skip error trapping socket if (bind(fd, (struct sockaddr*) &addr, sizeof(addr)) < 0) { // skip error trapping if (setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, (char*) &mreq, = sizeof(mreq)) < 0 ){ // skip error trapping argumetns while (1) { .. int nbytes =3D recvfrom(fd,msgbuf,MSGBUFSIZE,0,(struct sockaddr = *) &addr,&addrlen); if (nbytes < 0) { perror("recvfrom"); return 1; } printf(=E2=80=9CReceived packet, len=3D%d\n", nbytes); } --Apple-Mail=_2C77F2BB-87C7-4F5B-A2B8-5CD33CED02FF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Would anyone = know if there is something special with tunnel devices and multicast = ? 

I=E2=80=99ve got some code that happily = processes multicast packets on a normal interface; but appears not to do = this on a tunnel interface. Tun0 shows multicast enabled:
=
tun0: = flags=3D8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu = 1500

Tcpdump on that interface gives the expected = thing (here with mDNS):

tcpdump -n -i tun0 port = 5353

listening on tun0, link-type NULL = (BSD loopback), capture size 262144 bytes

19:26:03.976259 IP 10.31.0.6.5353 = > 224.0.0.251.5353: 0 PTR (QM)? _raop._tcp.local. (34)


And code, with a = simple IP_ADD_MEMBERSHIP  of the MC group on the IP of = the local interface below works on a normal interface (e.g. = igb0/10.0.0.1/24). 


./listener 10.0.0.1 = 224.0.0.251 5353

Received = packet, len=3D128

etc


But yields no = output if ran against above tun0 interface (while tcpdump on same is = fine). Does that ring a bell with anyone ?


Dw



int main(int argc, = char *argv[])

{

    struct sockaddr_in = addr;

    struct ip_mreq = mreq;


// skip error trapping command = line arguments


    char* ip =3D = argv[1]; 

    char* group =3D = argv[2]; 

    int port =3D atoi(argv[3]); // 0 if = error, which is an invalid port


    memset(&addr, 0, = sizeof(addr));

    addr.sin_family =3D = AF_INET;

    addr.sin_addr.s_addr =3D = htonl(INADDR_ANY);

    addr.sin_port =3D = htons(port);


    mreq.imr_interface.s_addr =3D = inet_addr(ip); 

  =   mreq.imr_multiaddr.s_addr =3D inet_addr(group);


// skip = error trapping on inet_addr


  =   int fd =3D socket(AF_INET, SOCK_DGRAM, 0);

// skip error trapping = socket


    if (bind(fd, (struct sockaddr*) = &addr, sizeof(addr)) < 0) {

// skip error = trapping


    if (setsockopt(fd, IPPROTO_IP, = IP_ADD_MEMBERSHIP, (char*) &mreq, sizeof(mreq)) < 0 ){

// skip error trapping = argumetns


    while (1) {

..

        int nbytes = =3D recvfrom(fd,msgbuf,MSGBUFSIZE,0,(struct sockaddr *) = &addr,&addrlen);

        if (nbytes < 0) = {

            = perror("recvfrom");

            return = 1;

        }

printf(=E2=80=9CReceived packet, = len=3D%d\n", nbytes);

     }


= --Apple-Mail=_2C77F2BB-87C7-4F5B-A2B8-5CD33CED02FF--