From nobody Sat Aug 8 16:37:05 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hHRWC14tTz6nvfF; Sat, 08 Aug 2026 16:37:11 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4hHRWB4RkFz3vgY; Sat, 08 Aug 2026 16:37:10 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; none Received: from phl-compute-03.internal (phl-compute-03.internal [10.202.2.43]) by mailfout.phl.internal (Postfix) with ESMTP id 49F74EC011B; Sat, 8 Aug 2026 12:37:09 -0400 (EDT) Received: from phl-frontend-04 ([10.202.2.163]) by phl-compute-03.internal (MEProxy); Sat, 08 Aug 2026 12:37:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gojira.at; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1786207029; x=1786293429; bh=ts4cOHkazu+kXGizFdAQk74+yP9WYnqxb1d8+IhjUEw=; b= BMZ+wdlyAbESHKCF6dsotAbbIivR141xyCJwig9Hzj1qv7thqMqJbtl2Vq7iyzsr p/3U+rNs1t9hr9NsUBhOLb8i2SEhxISrVS9f5JM3/7MeuzDwb5iGSu6cMsDpD9E7 8A9+0qmHSSBV2z7Ams4Gk4Znvh0Vb/kb/qM3Fk+0gMWbPjS4vm1mXtevrC8rosNk CcpmFuZ+a8Z1sqMvdsNxh8LaCfQ5fXn0WdK3FkE5g8OPTHySNx3iiOHAKxuNd2iS A1HhyBgd3Tym3u9T43ZrbVLVmzpqOB3sSKulRKHuBg3pHVVOcbG6Wpln95aszyut DzF41m+m07sqlyT4Lt03qA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1786207029; x= 1786293429; bh=ts4cOHkazu+kXGizFdAQk74+yP9WYnqxb1d8+IhjUEw=; b=D qUoQ6XExhnDLFymYktWnbrgeN0nrFS6e1kOuZMsSt3BjHFEbjnvs1pKdkoXdP8lA 5DfXLjiI2peLS8gRLDfeZ01Sszj2IdDYW/q58Gf8gZNvzDXWm5vqtRFiruZuNIZJ GQLW+HCidxvbwiVpBo2fJ4fiMrcTeGVhuqDmW2gK72bZrfJ1MHf9ollGGsEyKCp4 faidbMjmSUwp7JvYxl6JeYmVnFDDeG7vHhbyczj086oIK9+12Ln+6Qfh7jusfCBn Lb8WPU6RFe8HOKxbqvbeDhQq3HOI1TTZFCZGvSZeMmDZlCBVDnheZ05XsH3KzLZ4 kyjPla6x115LbWmuTJ+3A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGebM+RGLVrC39zeSN2iuvy1UW8bBXYE8IyRfd2sp12hVFAWSduSUx1ZqWNXK2Cf7 wHdmKsBpbJy1Rg3zTgjPmYv87kmIPsHFaIoJaJjoe6/Aq6FwG8+DVNZLoRv5XIzqmOR+S3 W3g/Fo1LJDSgnfLY7z/jEVMDrAYJZu7sFn4IJeC3GYnyE/+cIHjX0mEZeN+q2L0JTj5avp y4gekVm9owQEXW2V8+S/T2EJedX8UWlUEvfC7fagxjW/5QpdGbQOu56IPNqebe4LGaONcp xs7QN1XPOQ3xrMT+MO/lnPXWmOCTjUxPJOlhz2iBcPF7wghgBmMG011sWCiPR0Lv/mjlst DkauiLjR3/STFvy2DCwb203yKy+RK785zJBGCPBBM4px0YIEfU2xq/dHf6ZqPItxqXBIz3 BtyiTaV1AoDQHkisCes5V0BrWhJtRwuyBf+TS07CXqQ9cualgQqLXAhtmKz9CvuPLRRH57 Io01DQJ1JIKoTtnTNVacMXwTNL6XfS5jQcORDrpckCDMwPeKozIMkBw99QqsiyRUyp+sOI ztZ0XrwWlvIpvwDmkM3oBHD60Y8Hr0fkRCbJtmUAJ+ESNP9a9CEpCti+mHlh1BMjFb5IY4 Flrt/THaUq+CwbRO8hkO2P1SRt0T0r8y37F/Vn4x8bLN+S8rBqBgnzTFSbwA X-ME-Proxy: Feedback-ID: i64fe486c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 8 Aug 2026 12:37:08 -0400 (EDT) Date: Sat, 08 Aug 2026 18:37:05 +0200 Message-ID: <878q6gd0we.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: Mark Millard Cc: Mark Johnston , Alan Somers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, Jitendra Bhati Subject: Re: git: 4bd01d6ae016 - main - fts: refactor to use fd-relative operations internally In-Reply-To: <8c09ef72-bac2-414e-8670-9fa54a3d06c2@yahoo.com> References: <6a70e844.251cb.6995429f@gitrepo.freebsd.org> <8c09ef72-bac2-414e-8670-9fa54a3d06c2@yahoo.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/32.0 Mule/6.0 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4hHRWB4RkFz3vgY X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:151847, ipnet:103.168.172.0/24, country:AU] X-Spamd-Bar: ---- On Fri, 07 Aug 2026 23:05:17 +0200, Mark Millard wrote: >=20 > On 8/7/26 08:30, Mark Johnston wrote: > > On Thu, Aug 06, 2026 at 04:52:55PM -0600, Alan Somers wrote: > >> On Tue, Aug 4, 2026 at 4:06=E2=80=AFPM Mark Johnston wrote: > >>> On Tue, Aug 04, 2026 at 01:33:40PM -0600, Alan Somers wrote: > >>>> On Tue, Aug 4, 2026 at 1:00=E2=80=AFPM Alan Somers wrote: > >>>>> > >>>>> On Tue, Aug 4, 2026 at 12:58=E2=80=AFPM Mark Johnston wrote: > >>>>>> > >>>>>> On Mon, Aug 03, 2026 at 07:13:08PM +0000, Alan Somers wrote: > >>>>>>> The branch main has been updated by asomers: > >>>>>>> > >>>>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=3D4bd01d6ae01632501b= 63438b8d9a401db9744a78 > >>>>>>> > >>>>>>> commit 4bd01d6ae01632501b63438b8d9a401db9744a78 > >>>>>>> Author: Jitendra Bhati > >>>>>>> AuthorDate: 2026-06-12 17:07:55 +0000 > >>>>>>> Commit: Alan Somers > >>>>>>> CommitDate: 2026-08-03 19:12:28 +0000 > >>>>>>> > >>>>>>> fts: refactor to use fd-relative operations internally > >>>>>>> > >>>>>>> Replace all _open() calls with _openat() in __fts_open(), fts= _read(), > >>>>>>> and fts_children(). Replace statfs() with _fstatfs(). > >>>>>>> > >>>>>>> Add fts_dirfd to struct _ftsent, set to the file descriptor o= f the > >>>>>>> parent directory. Callers can use openat(ent->fts_dirfd, ent-= >fts_name, > >>>>>>> ...) to access files safely without relying on fts_accpath, w= hich > >>>>>>> enables programs in capability mode to open the files describ= ed by > >>>>>>> _ftsent. > >>>>>>> > >>>>>>> This is a preparatory change for fts_openat() which will allo= w callers > >>>>>>> to provide a pre-opened directory fd, enabling fts(3) travers= al inside > >>>>>>> Capsicum capability mode. > >>>>>>> > >>>>>>> Mirror all fts_open() changes to fts_open_b(). > >>>>>>> > >>>>>>> As a result of expanding _ftsend, publish new ELF symbol vers= ions for > >>>>>>> fts_openat and related functions. > >>>>>>> > >>>>>>> Sponsored by: Google LLC (GSoC 2026) > >>>>>>> Reviewed by: asomers > >>>>>>> Pull Request: https://github.com/freebsd/freebsd-src/pull/2= 303 > >>>>>> > >>>>>> Quite a few regression tests seem to be failing after this commit,= e.g., > >>>>>> some of the makefs tests: https://ci.freebsd.org/job/FreeBSD-main-= amd64-test/29201/ > >>>>> > >>>>> I'm testing a fix now. I should have something ready for you soon. > >> > >> The fix is committed now, as 74bd6fb1eb02 . Sorry for the trouble. > >=20 > > Thanks, but I think there are still some problems. I'm still seeing > > vnode exhaustion while running the test suite. > >=20 > > First, fts_build() appears to dup() the directory entry into every > > single(!) child FTS entry. That's slow and means that we'll potentially > > acquire a large number of fds when traversing a wide directory, enough > > to hit per-process fd limits. Second, the return value from dup() is > > not checked, and it's not clear to me whether that's handled elsewhere. > > Finally, I'm fairly sure these dup()ed fds are leaked in various places; > > for instance, isn't fts_lfree() potentially leaking the dirfd of each > > entry it frees? > >=20 > > The first and last issues are probably straightforward to fix, but I'm > > not sure about the second one. I don't think FTS can reasonably consume > > an fd for every entry in a directory, that's going to cause a lot of > > problems. > >=20 > >=20 >=20 > A question showed up on Discord's area for FreeBSD ports : >=20 > QUOTE > Ooo could this also be related to why I=E2=80=99m getting =E2=80=9Ctoo ma= ny open files=E2=80=9D > errors from poudriere when building things like go and rust? > END QUOTE >=20 > This was from someone that had progressed to/past 74bd6fb1eb02 in order > to avoid the prior poudriere(-devel) problems they were having. >=20 >=20 > (I'm still synchronized to before the problems started.) I have multiple BEs: 2x main (one with the change and the other one with both commits reverted). The following applies to both: I can build lang/go126 with poudriere in a stable/14 (base_latest), stable/15 (base_latest) and main (base_weekly) pkgbase jail. But building fails in a main (base_latest) jail with the "too many open files" error.