Date: Sat, 13 Sep 2025 10:13:19 -0600 From: Warner Losh <imp@bsdimp.com> To: bob prohaska <fbsd@www.zefox.net> Cc: Brad Davis <brd@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Git and buildworld running at the same time Message-ID: <CANCZdfpR7ZEUEOMZio1aXAwOiU_yLbn8TgBGk-3UsamjnRw9tA@mail.gmail.com> In-Reply-To: <aMWW6GpzUwSufCTE@www.zefox.net> References: <aMWGWbI-8TwnAPnP@www.zefox.net> <f9ae18ca-13d9-4fdc-8ee4-6b1870ad67ad@app.fastmail.com> <CANCZdfrNw=AHLUcKxGzYoPQ7Zs=E2Wjp6Kqq4E3aU_LyBh_snA@mail.gmail.com> <e786ed05-5ce5-42dc-92bc-d6cfdd3ccc3d@app.fastmail.com> <aMWW6GpzUwSufCTE@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Sat, Sep 13, 2025 at 10:08 AM bob prohaska <fbsd@www.zefox.net> wrote: > On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davis wrote: > > On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote: > > > > > > > > > On Sat, Sep 13, 2025 at 9:28 AM Brad Davis <brd@freebsd.org> wrote: > > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote: > > >> > Lately I've noticed that sometimes while running buildworld a top > > >> > window reports git running also. Up to now, I've surmised that > > >> > this is intentional, with git providing some housekeeping function. > > >> > > > >> > Yesterday a buildworld session was accompanied by a prolonged > > >> > interval of git running also, with a large memory footprint, > > >> > near 1GB. That seems rather excessive. > > >> > > > >> > At the same time, it dawned on me that my recent habit has been > > >> > to run git pull, immediately followed by buildworld. Might it be > > >> > prudent to wait (how long?) to let git finish any housekeeping > > >> > triggered by the pull command? It seems likely that any overlap > > >> > could readily lead to inconsistencies which might account for > > >> > some of the buildworld problems I've been encountering lately. > > >> > > >> This is part of the normal build process and how the output of uname > -a includes bits like this: main-n280188-2024887abc7d-dirty or > main-n280188-2024887abc7d > > >> > > >> To find out of the src tree is pristine or dirty the build process > uses git to find out. > > > > > > Though that's only for the kernel, not for world builds. Right? > > > > It happens for world builds as well.. newvers.sh runs git (used to be > svnlite) and is called in Makefile.inc1. > > > > Is it normal for git to consume (while buildworld is running) close > to 1 GB of memory for durations exceeding an hour? > > More fundamentally, is it OK to start buildworld immediately after > a git pull finishes interactive output and returns a shell prompt? > It's what I do: % git pull --rebase % make buildworld <args> I don't think it's normal for hours-long git runs, even with just 1GB. That's why I'm so interested in this. Why is git even running during buildworld? It shouldn't. That's a bug in my mind. I sometimes have git take minutes to do garbage collection on a rebase. But that's inline, not in the background. So some more basic questions to help me understand: (1) what root are you running on (media type and filesystem) (2) can you capture the full git command that's taking hours? (3) Any extra, non-default settings? (4) What's the exact build command you are using? Warner [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sat, Sep 13, 2025 at 10:08 AM bob prohaska <<a href="mailto:fbsd@www.zefox.net">fbsd@www.zefox.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, Sep 13, 2025 at 09:53:46AM -0600, Brad Davis wrote:<br> > On Sat, Sep 13, 2025, at 9:40 AM, Warner Losh wrote:<br> > > <br> > > <br> > > On Sat, Sep 13, 2025 at 9:28 AM Brad Davis <<a href="mailto:brd@freebsd.org" target="_blank">brd@freebsd.org</a>> wrote:<br> > >> On Sat, Sep 13, 2025, at 8:57 AM, bob prohaska wrote:<br> > >> > Lately I've noticed that sometimes while running buildworld a top<br> > >> > window reports git running also. Up to now, I've surmised that <br> > >> > this is intentional, with git providing some housekeeping function.<br> > >> ><br> > >> > Yesterday a buildworld session was accompanied by a prolonged<br> > >> > interval of git running also, with a large memory footprint,<br> > >> > near 1GB. That seems rather excessive.<br> > >> ><br> > >> > At the same time, it dawned on me that my recent habit has been<br> > >> > to run git pull, immediately followed by buildworld. Might it be<br> > >> > prudent to wait (how long?) to let git finish any housekeeping<br> > >> > triggered by the pull command? It seems likely that any overlap<br> > >> > could readily lead to inconsistencies which might account for<br> > >> > some of the buildworld problems I've been encountering lately.<br> > >> <br> > >> This is part of the normal build process and how the output of uname -a includes bits like this: main-n280188-2024887abc7d-dirty or main-n280188-2024887abc7d<br> > >> <br> > >> To find out of the src tree is pristine or dirty the build process uses git to find out.<br> > > <br> > > Though that's only for the kernel, not for world builds. Right?<br> > <br> > It happens for world builds as well.. newvers.sh runs git (used to be svnlite) and is called in Makefile.inc1.<br> > <br> <br> Is it normal for git to consume (while buildworld is running) close<br> to 1 GB of memory for durations exceeding an hour?<br> <br> More fundamentally, is it OK to start buildworld immediately after<br> a git pull finishes interactive output and returns a shell prompt?<br></blockquote><div><br></div><div>It's what I do:</div><div><br></div><div>% git pull --rebase</div><div>% make buildworld <args></div><div><br></div><div>I don't think it's normal for hours-long git runs, even with just 1GB. That's why I'm so interested in this. Why is git even running during buildworld? It shouldn't. That's a bug in my mind.</div><div><br></div><div>I sometimes have git take minutes to do garbage collection on a rebase. But that's inline, not in the background.</div><div><br></div><div>So some more basic questions to help me understand:</div><div>(1) what root are you running on (media type and filesystem)</div><div>(2) can you capture the full git command that's taking hours?</div><div>(3) Any extra, non-default settings?</div><div>(4) What's the exact build command you are using?</div><div><br></div><div>Warner</div></div></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpR7ZEUEOMZio1aXAwOiU_yLbn8TgBGk-3UsamjnRw9tA>