Date: Fri, 7 Aug 2026 14:05:17 -0700 From: Mark Millard <marklmi@yahoo.com> To: Mark Johnston <markj@freebsd.org>, Alan Somers <asomers@freebsd.org> Cc: dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, Jitendra Bhati <bhatijitendra2022@gmail.com> Subject: Re: git: 4bd01d6ae016 - main - fts: refactor to use fd-relative operations internally Message-ID: <8c09ef72-bac2-414e-8670-9fa54a3d06c2@yahoo.com> In-Reply-To: <anX6LVmcB1H5s_DY@nuc> References: <6a70e844.251cb.6995429f@gitrepo.freebsd.org> <anI2QoB7hj7CSMRq@nuc> <CAOtMX2g_U_mspHXdf3HL%2BFr-kh6XSbxR6JSTmVQ9OHDTqfckYw@mail.gmail.com> <CAOtMX2i9NxYuu=LZwLLP7NErLf4i5A7K=wLC%2B-GywchbwUZacw@mail.gmail.com> <anJiX_r4gThay5bk@nuc> <CAOtMX2j0yN9EF5fbMZ3YH7CgPS-KfNVDtgauDLva0uekkXiYDQ@mail.gmail.com> <anX6LVmcB1H5s_DY@nuc>
index | next in thread | previous in thread | raw e-mail
On 8/7/26 08:30, Mark Johnston wrote: > On Thu, Aug 06, 2026 at 04:52:55PM -0600, Alan Somers wrote: >> On Tue, Aug 4, 2026 at 4:06 PM Mark Johnston <markj@freebsd.org> wrote: >>> On Tue, Aug 04, 2026 at 01:33:40PM -0600, Alan Somers wrote: >>>> On Tue, Aug 4, 2026 at 1:00 PM Alan Somers <asomers@freebsd.org> wrote: >>>>> >>>>> On Tue, Aug 4, 2026 at 12:58 PM Mark Johnston <markj@freebsd.org> wrote: >>>>>> >>>>>> On Mon, Aug 03, 2026 at 07:13:08PM +0000, Alan Somers wrote: >>>>>>> The branch main has been updated by asomers: >>>>>>> >>>>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=4bd01d6ae01632501b63438b8d9a401db9744a78 >>>>>>> >>>>>>> commit 4bd01d6ae01632501b63438b8d9a401db9744a78 >>>>>>> Author: Jitendra Bhati <bhatijitendra2022@gmail.com> >>>>>>> AuthorDate: 2026-06-12 17:07:55 +0000 >>>>>>> Commit: Alan Somers <asomers@FreeBSD.org> >>>>>>> CommitDate: 2026-08-03 19:12:28 +0000 >>>>>>> >>>>>>> fts: refactor to use fd-relative operations internally >>>>>>> >>>>>>> Replace all _open() calls with _openat() in __fts_open(), fts_read(), >>>>>>> and fts_children(). Replace statfs() with _fstatfs(). >>>>>>> >>>>>>> Add fts_dirfd to struct _ftsent, set to the file descriptor of the >>>>>>> parent directory. Callers can use openat(ent->fts_dirfd, ent->fts_name, >>>>>>> ...) to access files safely without relying on fts_accpath, which >>>>>>> enables programs in capability mode to open the files described by >>>>>>> _ftsent. >>>>>>> >>>>>>> This is a preparatory change for fts_openat() which will allow callers >>>>>>> to provide a pre-opened directory fd, enabling fts(3) traversal inside >>>>>>> Capsicum capability mode. >>>>>>> >>>>>>> Mirror all fts_open() changes to fts_open_b(). >>>>>>> >>>>>>> As a result of expanding _ftsend, publish new ELF symbol versions for >>>>>>> fts_openat and related functions. >>>>>>> >>>>>>> Sponsored by: Google LLC (GSoC 2026) >>>>>>> Reviewed by: asomers >>>>>>> Pull Request: https://github.com/freebsd/freebsd-src/pull/2303 >>>>>> >>>>>> Quite a few regression tests seem to be failing after this commit, e.g., >>>>>> some of the makefs tests: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29201/ >>>>> >>>>> I'm testing a fix now. I should have something ready for you soon. >> >> The fix is committed now, as 74bd6fb1eb02 . Sorry for the trouble. > > Thanks, but I think there are still some problems. I'm still seeing > vnode exhaustion while running the test suite. > > First, fts_build() appears to dup() the directory entry into every > single(!) child FTS entry. That's slow and means that we'll potentially > acquire a large number of fds when traversing a wide directory, enough > to hit per-process fd limits. Second, the return value from dup() is > not checked, and it's not clear to me whether that's handled elsewhere. > Finally, I'm fairly sure these dup()ed fds are leaked in various places; > for instance, isn't fts_lfree() potentially leaking the dirfd of each > entry it frees? > > The first and last issues are probably straightforward to fix, but I'm > not sure about the second one. I don't think FTS can reasonably consume > an fd for every entry in a directory, that's going to cause a lot of > problems. > > A question showed up on Discord's area for FreeBSD ports : QUOTE Ooo could this also be related to why I’m getting “too many open files” errors from poudriere when building things like go and rust? END QUOTE This was from someone that had progressed to/past 74bd6fb1eb02 in order to avoid the prior poudriere(-devel) problems they were having. (I'm still synchronized to before the problems started.) -- === Mark Millard marklmi at yahoo.comhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8c09ef72-bac2-414e-8670-9fa54a3d06c2>