Date: Thu, 10 Mar 2022 12:34:07 +0200 From: Ze Dupsys <zedupsys@gmail.com> To: Brian Buhrow <buhrow@nfbcal.org> Cc: freebsd-xen@freebsd.org Subject: Re: ZFS + FreeBSD XEN dom0 panic Message-ID: <1937e88f-159a-5324-db8f-687e84a0bea9@gmail.com> In-Reply-To: <202203022007.222K7Y7I013387@nfbcal.org> References: <202203022007.222K7Y7I013387@nfbcal.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2022.03.02. 22:07, Brian Buhrow wrote: > Hello. Given Roger's message and the presentation of the errors you're seeing, I think > we're both in agreement that you're running out of memory. Below I've listed some suggestions > on how you might go about making your system more stable, and, possibly figuring out where the > trouble is, exactly. When I was building my infrastructure, I ran into a bunch of edge > conditions and weird bugs as well, so some ideas that may not seem obvious, may, in fact, work, > simply because they cause you to skirt some latent bug in the system. > > Here's what my setup looks like, in case it helps. > > > Hope these notes are helpful. Hello Brian, Yes, of course those notes are helpful. Knowing that somebody has a working and stable setup and how his configuration/setup is, is a help. Thank you for the detailed information. I did not respond right away, because i wanted to follow those steps before writing anything and i did not have a spare HDD for "new" testing system. Wanted to still keep already set-up current version 13.0 for testing as well. For testing i made somewhat simplified setup of your described one: 1. Installed FreeBSD-12.1, UFS ada0p1, brand new HDD, basic configuration just for network, resolve.conf, etc. 2. 8GB RAM Dom0, 16GB swap on ada0p2 3. i did not use gmirror for now, since it is a test setup and on this new HDD problems should not happen so soon, 4. bridge0 created, no vlans, just a simple bridge, no firewall not even physical NIC bridged, just for VMs 5. creating zpool on ada1, mountpoint=none, here though i got a question: When i create ZFS partition on disk, i use "gpart add -t freebsd-zfs .." and similar for UFS, but what gpt type should be used for XEN DomU passthrough partition? I mean, if i passed through whole disk to DomU, and VM formated it as ZFS, in case of physical system reboot, BIOS could get confused and maybe even boot from that previously passed-through-HDD. AFAIK FreeBSD boot loader prefers ZFS over UFS if both available. If gpt type is freebsd-zfs, how can i ensure that Dom0 does not try to taste it or mount it? In a way i know that in VM the given block device will be formatted with MBR or GPT, but still, what gpt type should be used in that case, so that Dom0 is satisfied and it is somewhat secure. For ZFS volumes for now i use volmode=dev. 6. well, to crash sooner i created 1 VM with 1 HDD, and other with 5 HDDs, passsed trhough ZVOLs. After approx. 10 hours of running given scripts, on/off 2 VMs and load on ZFS, machine panic'ed, so my hopes for stable setup were shattered. The only difference with your written mail was that on FreeBSD 12.1 RELEASE pkg repo, there is only xen-kernel-4.12 available. I will try to run with VMs with 1 and 2 HDDs only, but historically i know that FreeBSD 12.2 with 6GB RAM crashed with those setups as well, it just took longer to happen. Another difference is hardware used. Do you remember if you have tuned some BIOS settings? I'm thinking to poke around there as well. Could you share a config template for one of DomU VMs? Maybe i use some bad syntax for HDD devices or something. Thank you!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1937e88f-159a-5324-db8f-687e84a0bea9>