From nobody Fri Aug 7 22:47:50 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hGznN52nMz6nv1s; Fri, 07 Aug 2026 22:47:52 +0000 (UTC) (envelope-from markj@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hGznN47C2z3Z5V; Fri, 07 Aug 2026 22:47:52 +0000 (UTC) (envelope-from markj@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786142872; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sfje/2FgIMgfx4rYBK/t263yxsfqK7swnPb/4c0lDWg=; b=K6ADMhsU5Y6iV5wT+TYxyLqxtRN+C4p64mZlIc2yrhMJHVZQavlfvapm0jtz0GqlxYSAtM aUvY43gCc433HhvWwfrDTOOFk4Wiujl2+TCt6BbQvgHcC0mg1Cnn3azLUuXu2BnPmTf4mA bW0uqsAoDCKAho3fGtVON3VR109XG+GY7tsfeTojzcQz57masXjx2rBl7jAOFVbCniI8bG +vfRV8T1wprkP3vISohlis/2cjerIP3/kDFul4IE9odxkhOBxCqUOmPwJDj0rYcJ4gmfIX od76hbmKI9L8hwMAlUanNzBoys4ROPNHUaS9d8OiAzBx34+nmT0GxUyu/Y+IzQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786142872; a=rsa-sha256; cv=none; b=sZ6zKWu7+isU2jwJxY/vn19+V6Eaqtn0InXnOzlq0Bmz6BHHJRz2MbbcG70IYlvd8LlIx2 rt57uskXTaHWmuYFJQy7pO9p/5jvScnEC/9kPh0LI1BzUmCFrZpjxzxaqyhuaBRPD/AsMA QyasXqLBC4Nj/8qsX8VPz5PKlaYeMrCV+fqcsYE9GiUNl7FTQsOSeUlFLkEozHNfxub1YZ vWTOxznjXs7XX2sAiHBBfilvJsrkGxwVgv2S+QnZ9/Ta2mjehUNoG49iQH87m+9ryBiPpl 5mRoUY+3lM0Eq8/6ohS9Gt/npG3TI1/uuA3IZxX3EPTh5fMqE6dEncliZ11kXw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786142872; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sfje/2FgIMgfx4rYBK/t263yxsfqK7swnPb/4c0lDWg=; b=lTj493E2WDchZGX2qM3oPYT+BMdzzNC8phIQBVf/wgztAuxNdpEvKbNCa57S5KNIqM6bs6 TZcGwDm74syyikKFYGPVc46DcFd76JttZEese2N8ig/s3B6scqG0UHRz96hk5I+CnedNKT kXV5xS/bl8jn0oLBdIB4y4c+iZ81AbWsBnJ1yAyozn5QFmCGP//3r+qKAHLgpWYPT/+uyJ SMVCg8OJDC/oaivLMpcK0OPc7fiLSFdUPjK5g8BdtBk/CD7PRn96qDmnEh2qAbIRfqQjWy NTgBXyYnCxuR93HyYx+bZsNoLpP+uHLOwFiJ0ZZthnY1cIz44bOG8AW+beMVrQ== Received: from nuc (192-0-220-237.cpe.teksavvy.com [192.0.220.237]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: markj) by smtp.freebsd.org (Postfix) with ESMTPSA id 4hGznN149lzptR; Fri, 07 Aug 2026 22:47:52 +0000 (UTC) (envelope-from markj@freebsd.org) Date: Fri, 7 Aug 2026 18:47:50 -0400 From: Mark Johnston To: Mark Millard Cc: Alan Somers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, Jitendra Bhati Subject: Re: git: 4bd01d6ae016 - main - fts: refactor to use fd-relative operations internally Message-ID: References: <6a70e844.251cb.6995429f@gitrepo.freebsd.org> <8c09ef72-bac2-414e-8670-9fa54a3d06c2@yahoo.com> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8c09ef72-bac2-414e-8670-9fa54a3d06c2@yahoo.com> On Fri, Aug 07, 2026 at 02:05:17PM -0700, Mark Millard wrote: > On 8/7/26 08:30, Mark Johnston wrote: > > On Thu, Aug 06, 2026 at 04:52:55PM -0600, Alan Somers wrote: > >> On Tue, Aug 4, 2026 at 4:06 PM Mark Johnston wrote: > >>> On Tue, Aug 04, 2026 at 01:33:40PM -0600, Alan Somers wrote: > >>>> On Tue, Aug 4, 2026 at 1:00 PM Alan Somers wrote: > >>>>> > >>>>> On Tue, Aug 4, 2026 at 12:58 PM Mark Johnston wrote: > >>>>>> > >>>>>> On Mon, Aug 03, 2026 at 07:13:08PM +0000, Alan Somers wrote: > >>>>>>> The branch main has been updated by asomers: > >>>>>>> > >>>>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=4bd01d6ae01632501b63438b8d9a401db9744a78 > >>>>>>> > >>>>>>> commit 4bd01d6ae01632501b63438b8d9a401db9744a78 > >>>>>>> Author: Jitendra Bhati > >>>>>>> AuthorDate: 2026-06-12 17:07:55 +0000 > >>>>>>> Commit: Alan Somers > >>>>>>> CommitDate: 2026-08-03 19:12:28 +0000 > >>>>>>> > >>>>>>> fts: refactor to use fd-relative operations internally > >>>>>>> > >>>>>>> Replace all _open() calls with _openat() in __fts_open(), fts_read(), > >>>>>>> and fts_children(). Replace statfs() with _fstatfs(). > >>>>>>> > >>>>>>> Add fts_dirfd to struct _ftsent, set to the file descriptor of the > >>>>>>> parent directory. Callers can use openat(ent->fts_dirfd, ent->fts_name, > >>>>>>> ...) to access files safely without relying on fts_accpath, which > >>>>>>> enables programs in capability mode to open the files described by > >>>>>>> _ftsent. > >>>>>>> > >>>>>>> This is a preparatory change for fts_openat() which will allow callers > >>>>>>> to provide a pre-opened directory fd, enabling fts(3) traversal inside > >>>>>>> Capsicum capability mode. > >>>>>>> > >>>>>>> Mirror all fts_open() changes to fts_open_b(). > >>>>>>> > >>>>>>> As a result of expanding _ftsend, publish new ELF symbol versions for > >>>>>>> fts_openat and related functions. > >>>>>>> > >>>>>>> Sponsored by: Google LLC (GSoC 2026) > >>>>>>> Reviewed by: asomers > >>>>>>> Pull Request: https://github.com/freebsd/freebsd-src/pull/2303 > >>>>>> > >>>>>> Quite a few regression tests seem to be failing after this commit, e.g., > >>>>>> some of the makefs tests: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/29201/ > >>>>> > >>>>> I'm testing a fix now. I should have something ready for you soon. > >> > >> The fix is committed now, as 74bd6fb1eb02 . Sorry for the trouble. > > > > Thanks, but I think there are still some problems. I'm still seeing > > vnode exhaustion while running the test suite. > > > > First, fts_build() appears to dup() the directory entry into every > > single(!) child FTS entry. That's slow and means that we'll potentially > > acquire a large number of fds when traversing a wide directory, enough > > to hit per-process fd limits. Second, the return value from dup() is > > not checked, and it's not clear to me whether that's handled elsewhere. > > Finally, I'm fairly sure these dup()ed fds are leaked in various places; > > for instance, isn't fts_lfree() potentially leaking the dirfd of each > > entry it frees? > > > > The first and last issues are probably straightforward to fix, but I'm > > not sure about the second one. I don't think FTS can reasonably consume > > an fd for every entry in a directory, that's going to cause a lot of > > problems. > > > > > > A question showed up on Discord's area for FreeBSD ports : > > QUOTE > Ooo could this also be related to why I’m getting “too many open files” > errors from poudriere when building things like go and rust? > END QUOTE Yes that seems plausible. See also https://github.com/freebsd/freebsd-src/pull/2360 > > This was from someone that had progressed to/past 74bd6fb1eb02 in order > to avoid the prior poudriere(-devel) problems they were having. > > > (I'm still synchronized to before the problems started.)