Date: Mon, 26 Jun 2023 14:49:55 +0300 From: Vitaliy Gusev <gusev.vitaliy@gmail.com> To: Aryeh Friedman <aryeh.friedman@gmail.com> Cc: FreeBSD virtualization <freebsd-virtualization@freebsd.org>, FreeBSD Mailing List <freebsd-questions@freebsd.org> Subject: Re: bhyve: how to keep the host from starving the guest Message-ID: <48D9FE17-B885-40AC-8958-EA90EEB9D8B6@gmail.com> In-Reply-To: <CAGBxaXkcBuu70SjjW4ci-6Lhv24gqowf6T2Rj4Aig4sZLt-Tvg@mail.gmail.com> References: <CAGBxaXkcBuu70SjjW4ci-6Lhv24gqowf6T2Rj4Aig4sZLt-Tvg@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Hi Aryeh, Have you wired a guest memory with bhyve's -S option? -S Wire guest memory Anyway, OS does not have another choice than kill a process to free some memory when RAM+swap is fully used (assume kernel already scanned Inactive memory). As recommendation: Look at an another memory consumers like ZFS, another processes Increase swap Tune vm.overcommit sysctl. See tuning(7) for details. So in short, there is no good way to run applications that fully use 10 GB memory on a systems with just 1 GB RAM + 1 GB swap. You should have enough resources to do that. And it would be nice, if you provide more information and metrics for your system, total RAM, memory assigned for VM, additional statistics for all intensive processes (SIZE, RES), etc. — Vitaliy Gusev > On 26 Jun 2023, at 13:44, Aryeh Friedman <aryeh.friedman@gmail.com> wrote: > > I am a 12 core machine that I want allocate only 4 CPU's to the host > and 8 to a VM (the host is my desktop FreeBSD machine and the guest is > debian 11 used for playing around with learning AI model making).... I > have passed my GeForce 1030 (bottom of the line GPU for AI work it > seems) but since it appears that no one can get tensorflow, pytorch or > anything else that runs ANN's on a GPU to work on FreeBSD (I have > tracked down to the fact nvidia never ported CUDA to FreeBSD)... the > problem is sometimes during heavy work on the guest then the host > slows down and if the the host is doing heavy work (especially > resource intensive things like compiling lang/rust) that the host will > kill the guest if it runs out of available memory+swap (speaking of > that does wiring the memory at least prevent this?) > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > [-- Attachment #2 --] <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Hi Aryeh,<br><div><br></div><div>Have you wired a guest memory with bhyve's -S option?</div><div><br></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><span style="font-family: Menlo; font-size: 12px; font-variant-ligatures: no-common-ligatures;"><b>-S</b></span><span style="font-family: Menlo; font-size: 12px; font-variant-ligatures: no-common-ligatures;"> Wire guest memory</span></div></blockquote><div><br></div><div>Anyway, OS does not have another choice than kill a process to free some memory</div><div>when RAM+swap is fully used (assume kernel already scanned Inactive memory).</div><div><br></div><div>As recommendation:</div><div><ul class="MailOutline"><li>Look at an another memory consumers like ZFS, another processes</li><li>Increase swap</li><li>Tune <span style="background-color: rgb(255, 255, 255);"><span style="font-family: Menlo; font-size: 12px; text-decoration: underline; font-variant-ligatures: no-common-ligatures;">vm.</span><span style="font-family: Menlo; font-size: 12px; text-decoration: underline; font-variant-ligatures: no-common-ligatures;">over</span><span style="font-family: Menlo; font-size: 12px; text-decoration: underline; font-variant-ligatures: no-common-ligatures;">commit</span><span style="font-variant-ligatures: no-common-ligatures;"> sysctl. </span></span>See <u>tuning</u>(7) for details.</li></ul><div><br></div><div>So in short, there is no good way to run applications that fully use 10 GB</div><div>memory on a systems with just 1 GB RAM + 1 GB swap. You should have</div><div>enough resources to do that.</div></div><div><br></div><div>And it would be nice, if you provide more information and metrics for your</div><div>system, total RAM, memory assigned for VM, additional statistics for all</div><div>intensive processes (SIZE, RES), etc.</div><div><br></div><div>—</div><div>Vitaliy Gusev</div><div><br></div><div><blockquote type="cite"><div>On 26 Jun 2023, at 13:44, Aryeh Friedman <aryeh.friedman@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><div>I am a 12 core machine that I want allocate only 4 CPU's to the host<br>and 8 to a VM (the host is my desktop FreeBSD machine and the guest is<br>debian 11 used for playing around with learning AI model making).... I<br>have passed my GeForce 1030 (bottom of the line GPU for AI work it<br>seems) but since it appears that no one can get tensorflow, pytorch or<br>anything else that runs ANN's on a GPU to work on FreeBSD (I have<br>tracked down to the fact nvidia never ported CUDA to FreeBSD)... the<br>problem is sometimes during heavy work on the guest then the host<br>slows down and if the the host is doing heavy work (especially<br>resource intensive things like compiling lang/rust) that the host will<br>kill the guest if it runs out of available memory+swap (speaking of<br>that does wiring the memory at least prevent this?)<br><br>-- <br>Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org<br><br></div></div></blockquote></div><br></body></html>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48D9FE17-B885-40AC-8958-EA90EEB9D8B6>