From nobody Tue Sep 16 02:41:38 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cQmPz3t7Pz6857c for ; Tue, 16 Sep 2025 02:41:59 +0000 (UTC) (envelope-from freebsd-current-freebsd-org111@ketas.si.pri.ee) Received: from mail.ketas.si.pri.ee (d004-fea2-0bff-021e-13e8-8437-07d0-2001.dyn.estpak.ee [IPv6:2001:7d0:8437:13e8:21e:bff:fea2:d004]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4cQmPy4yQwz45vx for ; Tue, 16 Sep 2025 02:41:58 +0000 (UTC) (envelope-from freebsd-current-freebsd-org111@ketas.si.pri.ee) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ketas.si.pri.ee header.s=ketas-si-pri-ee-20240416002854-4096 header.b=cKT2+GqT; dmarc=pass (policy=reject) header.from=ketas.si.pri.ee; spf=pass (mx1.freebsd.org: domain of freebsd-current-freebsd-org111@ketas.si.pri.ee designates 2001:7d0:8437:13e8:21e:bff:fea2:d004 as permitted sender) smtp.mailfrom=freebsd-current-freebsd-org111@ketas.si.pri.ee X-Original-To: freebsd-current@freebsd.org DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ketas.si.pri.ee; s=ketas-si-pri-ee-20240416002854-4096; t=1757990499; bh=Q9eK6nkMwaRckw6gXhjB9nmtOGZY/2JYTJv8Z5OE6AM=; h=Date:From:To:Subject:In-Reply-To:References; b=cKT2+GqTn3XCm/3yUYaYCtXzDRyW8l0MKr198nWIj3yIENm9pDb3JG7pOYkJmSK6u 0VWCVEj7EUS+JDbsD5AizkRvjodCzwlaiQ60JKmsqw72+uIEoLFGE8QOrXzX5lfXV+ rY4HBBk8Ct19Ot8Nr0wNkzC1LvmJh38eSOXe5LZsfpvW5pCWcRmF716R5Gqx8KdDsg 37af7XPemAg44yf21JvEeCGFiJMCISWRczTZgToA5GYBuLYFa8LupFw9uHRaMlZ9xM r0/pXjzFdMPQ6AO1yn6tUE+avGc4+ez4nNKDLzi/XcL5XfNxSwOylbBN9BTHnyRHhA YGWusvTrcAkozR/uP5SP8Z/K44MtXbssuFVetyoBlahPcQdLOlb9wO4o82PiBWSN25 B5O4uC4hiUlT3urajgTFYlficbqkRo/WxMN3eqhFjkOH3isdX/4DmED4djXaBRK9x4 jUbpiWFWarkqy8lLrP4DV/JZz00Gd9g67KeSv9XrJuvRy5FJzHkLlJKLYZz8YliA2M hMkmWPsIgj1j91yFLpL6OuWcjyfalbJ/PVukG+1oxPh66jFo8pMsMLuJRxHzCw6+2T LQW5BFaenenkVSOpjn5O6PD2jgExN8vqB3Uaclfz8REZtbw3a0n+TVpFRSVS/DgkRU tY6IYL+9ufB+cFD10nxWDXyw= Received: from ehlo.thunderbird.net (0115-0000-0000-0000-13c8-8437-07d0-2001.dyn.estpak.ee [IPv6:2001:7d0:8437:13c8::115]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.ketas.si.pri.ee (Postfix) with ESMTPSA id 83C8F5A14A4 for ; Tue, 16 Sep 2025 05:41:39 +0300 (EEST) Date: Tue, 16 Sep 2025 05:41:38 +0300 From: Sulev-Madis Silber To: freebsd-current@freebsd.org Subject: Re: Git and buildworld running at the same time User-Agent: K-9 Mail for Android In-Reply-To: References: Message-ID: <6BC7CEA4-D21C-4D53-9013-3F2DBA260362@ketas.si.pri.ee> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Spamd-Result: default: False [-0.60 / 15.00]; HFILTER_HOSTNAME_5(3.00)[d004-fea2-0bff-021e-13e8-8437-07d0-2001.dyn.estpak.ee]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-0.80)[-0.797]; DMARC_POLICY_ALLOW(-0.50)[ketas.si.pri.ee,reject]; R_DKIM_ALLOW(-0.20)[ketas.si.pri.ee:s=ketas-si-pri-ee-20240416002854-4096]; ONCE_RECEIVED(0.20)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:7d0:8437:1300::/56]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[ketas.si.pri.ee:+] X-Rspamd-Queue-Id: 4cQmPy4yQwz45vx in my systems i've had buildworld failures with recent git=2E i didn't even= know it does bg maint and one other thing was that if i let git loose and don't do [core] packedGitWindowSize =3D 1m packedGitLimit =3D 1m preloadIndex =3D false [diff] renameLimit =3D 1 it'll tell kernel to eat up my ram and cause userland to die during some p= articular unknown git operation this is also on zfs and i'm not alone here, while ufs doesn't do this (in = other people's machines) i don't know where to point finger (mmap? slow io?) or how to test it in v= m=2E or with what command=2E i noticed that lot of changes via pull in port= s caused this 100% a time seems like this needs several factors for perfect storm=2E note that git i= s only thing that can do this=2E or perhaps dovecot too, i configured that = thing to not do mmap and / or cache to this day i don't have proper solution to this seems like slow io has part here i could simulate that with geom (gnop maybe?) into a vm but i can't figure= out how to have the proper stress test set for this=2E i tried several (me= mory) stress methods but they never do this unsure what this really is=2E is it a feature or bug or how to limit it i find it totally unacceptable that zfs is able to take any commands from = userland that cause kernel to take over 95% of ram in few seconds=2E it sho= uld start waiting far before this is not arc too=2E something in the kernel is allocating all of my ram= from git fs operations and that sucks maybe this thing is missing from zfs test suite? or maybe noone ran that t= hing on less than ideal machines? funnily i'm able to do anything else on this c2d & 4g machine=2E just need= s time, nothing fails i have no idea how to solve this mystery=2E zfs has many tuning options, a= re some of them related to writes too? i haven't used any of them funnily those options don't seem to slow git down, if you could say it's f= ast before so any ideas? before people tried to test this by lowering ram in vm, but if they had th= ose on fast ssd's, it would never manifest unless i'm mistaken here, this is slow io issue? but how could system accept more that it can do? i don't know how zfs works but can't it like put io operations into pause = if it looks like disks are not keeping up? just like cpu or ram limits are respected and nothing really goes "over th= e edge" i expect this to bite users in future=2E especially corner case hw=2E this= could be purposefully low end hw too i could come back with a test case but i don't really have place where i c= an run things repeatedly into failure