Date: Sun, 14 Sep 2025 10:35:27 -0600 From: Warner Losh <imp@bsdimp.com> To: bob prohaska <fbsd@www.zefox.net> Cc: Brad Davis <brd@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Git and buildworld running at the same time Message-ID: <CANCZdfpq3TYYG_-13fkV_ad7y2M6w1ehrz8N1uwXo50=0rvFvA@mail.gmail.com> In-Reply-To: <aMbsfSvVt6xp8JbW@www.zefox.net> References: <aMWGWbI-8TwnAPnP@www.zefox.net> <f9ae18ca-13d9-4fdc-8ee4-6b1870ad67ad@app.fastmail.com> <CANCZdfrNw=AHLUcKxGzYoPQ7Zs=E2Wjp6Kqq4E3aU_LyBh_snA@mail.gmail.com> <e786ed05-5ce5-42dc-92bc-d6cfdd3ccc3d@app.fastmail.com> <aMWW6GpzUwSufCTE@www.zefox.net> <CANCZdfpR7ZEUEOMZio1aXAwOiU_yLbn8TgBGk-3UsamjnRw9tA@mail.gmail.com> <aMbsfSvVt6xp8JbW@www.zefox.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Sun, Sep 14, 2025 at 10:25 AM bob prohaska <fbsd@www.zefox.net> wrote: > On Sat, Sep 13, 2025 at 10:13:19AM -0600, Warner Losh wrote: > > On Sat, Sep 13, 2025 at 10:08 AM bob prohaska <fbsd@www.zefox.net> > wrote: > > > > > Is it normal for git to consume (while buildworld is running) close > > > to 1 GB of memory for durations exceeding an hour? > > > > > > More fundamentally, is it OK to start buildworld immediately after > > > a git pull finishes interactive output and returns a shell prompt? > > > > > > > It's what I do: > > > > % git pull --rebase > > % make buildworld <args> > > After running make cleandir twice, buildworld still failed with > the same "failed assertion" error. > > Running > git pull --rebase > reported > Updating 63bd2416ccd..455426da078 with no apparent errors > and a return to the shell prompt. Meanwhile, top continues > to show git activity, some minutes later amounting to over > one GB of memory use, 66MB of swap use and ~80% of one core > in use. After a few more minutes memory use grew to 1.2 GB, > but %CPU is lower, ~10-20%. > > > Top does not show any flags applied to git, and a > ps -aux | grep git > reports only the "grep git" process, nothing else. > > It's very likely that at some point I started buildworld > while git was similarly active. > > Am I correct thinking this would be a bad thing? > Yes. It can be. RPi2, IIRC, has USB2, so having root on usb would mean it's a relatively high latency connection. That's fine, in general, but causes two problems for you. (1) git is very sensitive to high latency I/O path. This means git will run for much longer on such a system and (2) git can take a lot of memory, which is bad during buildworld which itself creates a lot of memory pressure. The two together means that buildworld has a lot less memory to work with and will be slower or may even fail. I'm surprised at the git activity. top only reports the command. It must be in tear-down still (which is weird, because the shell does a wait() for the process and the process is completely torn down before that returns... So I'm super confused... I like the more plausible explanation of 'git was running in another session when I started world' but it does seem like that might not be the case. Warner [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sun, Sep 14, 2025 at 10:25 AM bob prohaska <<a href="mailto:fbsd@www.zefox.net">fbsd@www.zefox.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, Sep 13, 2025 at 10:13:19AM -0600, Warner Losh wrote:<br> > On Sat, Sep 13, 2025 at 10:08 AM bob prohaska <<a href="mailto:fbsd@www.zefox.net" target="_blank">fbsd@www.zefox.net</a>> wrote:<br> > <br> > > Is it normal for git to consume (while buildworld is running) close<br> > > to 1 GB of memory for durations exceeding an hour?<br> > ><br> > > More fundamentally, is it OK to start buildworld immediately after<br> > > a git pull finishes interactive output and returns a shell prompt?<br> > ><br> > <br> > It's what I do:<br> > <br> > % git pull --rebase<br> > % make buildworld <args><br> <br> After running make cleandir twice, buildworld still failed with<br> the same "failed assertion" error.<br> <br> Running<br> git pull --rebase<br> reported <br> Updating 63bd2416ccd..455426da078 with no apparent errors<br> and a return to the shell prompt. Meanwhile, top continues<br> to show git activity, some minutes later amounting to over<br> one GB of memory use, 66MB of swap use and ~80% of one core<br> in use. After a few more minutes memory use grew to 1.2 GB,<br> but %CPU is lower, ~10-20%. <br> <br> <br> Top does not show any flags applied to git, and a<br> ps -aux | grep git<br> reports only the "grep git" process, nothing else.<br> <br> It's very likely that at some point I started buildworld <br> while git was similarly active. <br> <br> Am I correct thinking this would be a bad thing?<br></blockquote><div><br></div><div>Yes. It can be. RPi2, IIRC, has USB2, so having root on usb would mean it's a relatively high latency connection. That's fine, in general, but causes two problems for you. (1) git is very sensitive to high latency I/O path. This means git will run for much longer on such a system and (2) git can take a lot of memory, which is bad during buildworld which itself creates a lot of memory pressure. The two together means that buildworld has a lot less memory to work with and will be slower or may even fail.</div><div><br></div><div>I'm surprised at the git activity. top only reports the command. It must be in tear-down still (which is weird, because the shell does a wait() for the process and the process is completely torn down before that returns... So I'm super confused...</div><div><br></div><div>I like the more plausible explanation of 'git was running in another session when I started world' but it does seem like that might not be the case.<br><br>Warner</div></div></div>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpq3TYYG_-13fkV_ad7y2M6w1ehrz8N1uwXo50=0rvFvA>