Date: Sat, 23 Dec 2023 19:36:00 +0100 From: Mario Marietto <marietto2008@gmail.com> To: Warner Losh <imp@bsdimp.com> Cc: Stefano Stabellini <sstabellini@kernel.org>, Stanislav Silnicki <stanislav.silnicki@mailgate.us>, freebsd-arm@freebsd.org, Michal.Orzel@amd.com, xen-devel@lists.xenproject.org, Artem_Mygaiev@epam.com Subject: Re: How to boot FreeBSD for arm 32 bit as DomU with u-boot on my ARM Chromebook Message-ID: <CA%2B1FSigREr=aSHOBMbEK9CaoKzNnKC-6sE=Ui8ovVwnMaZgZhQ@mail.gmail.com> In-Reply-To: <CANCZdfpRJxJw2_ma0cCcoGGmAat4KVj5sOfdnGMR9w%2BOQCYxaQ@mail.gmail.com> References: <CA%2B1FSiit5NJ_W7f5mssivkGGLevXh_XmSB7ZxZbQe1dgev0_Pg@mail.gmail.com> <97aa980b9b44.6eb7f9d5c54e7@mailgate.us> <CA%2B1FSiheSwrO7Tz3F3Nmu4vcC2E=ONRX8nF3bhQGV0UDOFTrww@mail.gmail.com> <da024d1687f3a.2f0c601bba502@mailgate.us> <CA%2B1FSih7C0HEun99LHEwvyYRiH0PaLwwgnoQLHB69TWz2ykDZg@mail.gmail.com> <CA%2B1FSijoPkRZRaU6tujaMJZeFd3CSyUN%2Bh4y0_NrrB5VWLgwpA@mail.gmail.com> <CA%2B1FSiib3uu9Ky8N5rSSt%2BH1nfzUnChKhGs2ERWmsbp2T4=2uA@mail.gmail.com> <CA%2B1FSihtb3%2Bbm-pSqa7JGBjy3JUXYD6i0c6J2KHy%2BEO8rmZ5PA@mail.gmail.com> <alpine.DEB.2.22.394.2312191131290.3175268@ubuntu-linux-20-04-desktop> <CA%2B1FSigh=fc07o-RSMZ%2Bhf9-YBUudAwnGcN9k8bP1RnPBP7tRA@mail.gmail.com> <CANCZdfpRJxJw2_ma0cCcoGGmAat4KVj5sOfdnGMR9w%2BOQCYxaQ@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] I've added this parameter to bootxen.source : guest_loglvl=all bootxen.source : mmc dev 1 ext2load mmc 1:3 0x42000000 zImage-5.4.261-iommu-dma-on-xen ext2load mmc 1:3 0x51000000 xen-4.17-armhf-armmp-0x51004000.ub ext2load mmc 1:3 0x5ffec000 exynos5250-snow.dtb fdt addr 0x5ffec000 fdt resize 1024 fdt set /chosen \#address-cells <0x2> fdt set /chosen \#size-cells <0x2> fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0 dom0_mem=1152M dom0_max_vcpus=2 bootscrub=0 vwfi=native guest_loglvl=all" fdt mknod /chosen dom0 fdt set /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module" "multiboot,module" fdt set /chosen/dom0 reg <0x0 0x42000000 0x0 0x49F9A8 > fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 rw rootwait clk_ignore_unused --no-log" bootm 0x51000000 - 0x5ffec000 but when I try to boot FreeBSD I don't get more informations than usual : root@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen# ./start-freebsd Parsing config from freebsd.cfg xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found: Invalid kernel libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 1:cannot (re-)build domain: -3 libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 1:Non-existent domain libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable to destroy guest libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction of domain failed freebsd is an invalid domain identifier (rc=-6) Are you aware about a new parameter that I can use to have more detailed debug information ? On Wed, Dec 20, 2023 at 5:52 AM Warner Losh <imp@bsdimp.com> wrote: > I'd think you'd need the right virtualization loader. I'm not entirely > sure the u-boot.bin you've been creating is for a dom-u.. > If I misunderstood, then the below isn't good advice. Chain booting the > u-boot, the first u-boot initializes things so you want > to start with stage after the SPL. But the different error messages > suggest that it's trying to reboot with kexec, which > isn't supported on armv7 at the moment. > > If you could boot in kvm, I think that the following would work.... > Though I'm not entirely sure how to > specify the two .fd files in your setup. The use of qemu is to have an > easy env to debug things... I don't > have a chromebook to try... > > My first instinct would be to try qemu on x86 (this is the first step of > many to get to your destination). > > If you could boot the GENERIC_SD image that we produce using qemu + > edk2-arm-code.fd that would > be a huge first step. This will give you the boot loader, I believe, to > boot in the VM that you need better > than going via the u-boot route. Since you are booting in a virtualized > environment, I think it wouldn't > matter which one :). > > So, I did the following to boot the virtualized armv7 FreeBSD environment, > following a post on the forums I found and knew to have the right recipe: > > https://forums.freebsd.org/threads/run-boot-freebsd-arm-32bit-image-in-qemu.80765/ > > 1. pkg install qemu > 2. mkdir qemu-armv7-env > 3. cd qemu-armv7-env > 4. fetch > https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/14.0/FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD.img.xz > 5. xz -d -T 0 FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD.img.xz > 6. dd if=/dev/zero of=pflash0.img bs=1m count=64 > 7. dd if=/dev/zero of=pflash1.img bs=1m count=64 > 8. dd if=/usr/local/share/qemu/edk2-arm-code.fd of=pflash0.img conv=notrunc > 9. dd if=/usr/local/share/qemu/edk2-arm-vars.fd of=pflash1.img conv=notrunc > 10. cat > start-freebsd-arm.sh > #!/bin/sh > qemu-system-arm \ > -M virt \ > -m 1024 \ > -drive file=pflash0.img,format=raw,if=pflash,readonly=on \ > -drive file=pflash1.img,format=raw,if=pflash \ > -drive file=$1.img,if=virtio,cache=writethrough \ > -nographic \ > -serial mon:stdio > ^D > 11. chmod +x start-freebsd-arm.sh > 12. ./start-freebsd-arm.sh FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD > > But I hit a snag with this on qemu 8.1.2 and 8.1.3 with both 13.2 and 14.0: > > Starting devd. > Starting dhclient. > DHCPDISCOVER on vtnet0 to 255.255.255.255 port 67 interval 7 > Fatal kernel mode data abort: 'Alignment Fault' on read > trapframe: 0xc4b36a60 > FSR=00000001, FAR=dd96701a, spsr=20000013 > r0 =00000000, r1 =00000001, r2 =00000001, r3 =c4b36b4c > r4 =00000014, r5 =d6618800, r6 =dd96702e, r7 =0000022c > r8 =00000000, r9 =0000022c, r10=dd96701a, r11=c4b36b90 > r12=4300ffff, ssp=c4b36af0, slr=c04a9728, pc =c04a9750 > > panic: Fatal abort > cpuid = 0 > time = 1680843057 > KDB: stack backtrace: > #0 0xc035786c at kdb_backtrace+0x48 > #1 0xc02fdd20 at vpanic+0x140 > #2 0xc02fdbe0 at vpanic+0 > #3 0xc06304ac at abort_align+0 > #4 0xc063052c at abort_align+0x80 > #5 0xc063017c at abort_handler+0x480 > #6 0xc060f480 at exception_exit+0 > #7 0xc04a9750 at udp_input+0x288 > #8 0xc0473f54 at ip_input+0x1e0 > #9 0xc04447c0 at netisr_dispatch_src+0xf8 > #10 0xc043bf2c at ether_demux+0x1a4 > #11 0xc043d5e4 at ether_nh_input+0x480 > #12 0xc04447c0 at netisr_dispatch_src+0xf8 > #13 0xc043c404 at ether_input+0x50 > #14 0xc01c0838 at vtnet_rx_vq_process+0x880 > #15 0xc01b70d0 at vtpci_intx_intr+0xac > #16 0xc02b87f0 at ithread_loop+0x2ec > #17 0xc02b465c at fork_exit+0xc0 > Uptime: 19s > > I don't know if this is a problem with qemu or FreeBSD's kernel... > > Warner > > On Tue, Dec 19, 2023 at 3:25 PM Mario Marietto <marietto2008@gmail.com> > wrote: > >> I've asked some help on the channel #arm on Reddit and someone replied : >> >> >> https://www.reddit.com/r/arm/comments/18mcir8/i_cant_boot_freebsd_for_arm32_bit_as_domu_with/ >> >> Maybe his answer can be useful to understand why it does not work. >> >> On Tue, Dec 19, 2023 at 8:33 PM Stefano Stabellini < >> sstabellini@kernel.org> wrote: >> >>> +Michal >>> >>> Hi Mario, >>> >>> I am not sure about booting FreeBSD, but I am certain that u-boot works >>> fine as DomU kernel on ARMv8 (not sure about ARMv7). With this config >>> file: >>> >>> name="test" >>> kernel="u-boot.bin" >>> extra = "console=hvc0" >>> memory=256 >>> vcpus=1 >>> disk = [ 'FreeBSD-13.2-RELEASE-armv7.img,raw,xvda' ] >>> >>> I don't know for sure if you can boot FreeBSD but you should definitely >>> be able to see the u-boot command line prompt. The fact that you are >>> getting this message: >>> >>> xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader >>> found: Invalid kernel >>> >>> Means that something is not right in the u-boot configuration or u-boot >>> build. Michal and Artem (CCed) might know more. From what I recall, >>> there was nothing special required to get u-boot.bin to boot as domU >>> kernel, so now I wonder if it is an ARMv7 vs. ARMv8 issue. >>> >>> Cheers, >>> >>> Stefano >>> >>> >>> On Tue, 19 Dec 2023, Mario Marietto wrote: >>> > ....I see that some other interesting files have been produced by >>> u-boot when I have compiled it : >>> > >>> > u-boot >>> > u-boot.lds >>> > u-boot.bin >>> > u-boot.map >>> > u-boot-nodtb.bin >>> > u-boot.dtb >>> > u-boot.srec >>> > u-boot-dtb.bin >>> > u-boot.sym >>> > >>> > So,maybe I should use a different u-boot* file for booting FreeBSD ? >>> > >>> > >>> > On Tue, Dec 19, 2023 at 4:28 PM Mario Marietto <marietto2008@gmail.com> >>> wrote: >>> > Hello to everyone. >>> > >>> > I have compiled the needed u-boot.bin from scratch using this >>> procedure : >>> > >>> > # git clone https://github.com/u-boot/u-boot.git >>> > # cd u-boot >>> > # ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make snow_defconfig : >>> this line generates the file .config >>> > # nano .config and I've added these parameters : >>> > >>> > CONFIG_ARMV7_NONSEC=n >>> > CONFIG_EFI_GRUB_ARM32_WORKAROUND=y >>> > >>> > the uboot-bin file is generated with this command : >>> > >>> > # ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make >>> > >>> > At this point,I took a look inside the .config file and I saw that the >>> parameter "CONFIG_ARMV7_NONSEC=n" has been removed. So,for >>> > some reason,it is not accepted and this could be a problem.... >>> > >>> > These are the xen config files that I've used : >>> > >>> > nano freebsd.cfg >>> > >>> > name="test" >>> > kernel="u-boot.bin" >>> > extra = "console=hvc0" >>> > memory=256 >>> > vcpus=1 >>> > disk = [ 'FreeBSD-13.2-RELEASE-armv7.img,raw,xvda' ] >>> > >>> > nano start-freebsd >>> > >>> > xl create freebsd.cfg >>> > xl console freebsd >>> > >>> > This is what happens when I launch the vm : >>> > >>> > # ./start-freebsd >>> > >>> > Parsing config from freebsd.cfg >>> > xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader >>> found: Invalid kernel >>> > libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image >>> failed >>> > libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain >>> 1:cannot (re-)build domain: -3 >>> > libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain >>> 1:Non-existent domain >>> > libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain >>> 1:Unable to destroy guest >>> > libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain >>> 1:Destruction of domain failed >>> > freebsd is an invalid domain identifier (rc=-6) >>> > >>> > >>> > On Mon, Dec 18, 2023 at 12:39 PM Mario Marietto < >>> marietto2008@gmail.com> wrote: >>> > So,ok,I should have said "the second u-boot" ; since the first >>> u-boot binary is the "u-boot binary located in the RO >>> > memory" of the Chromebook". Sorry for the confusion. >>> > >>> > On Mon, Dec 18, 2023 at 12:35 PM Mario Marietto < >>> marietto2008@gmail.com> wrote: >>> > ---> There are no specific options in u-boot devoted to FreeBSD >>> > >>> > This is an important factor. So,what about if,instead of compiling a >>> new version of u-boot on the partition 2,I will >>> > recompile the u-boot customized version created by the virtual open >>> system in 2014,that should be installed on the first >>> > partition ? It could work if there are no differences between the >>> u-boot that should boot Linux and the u-boot that >>> > should boot FreeBSD. >>> > >>> > Can you give a look at the u-boot source code created by virtual open >>> systems ? You can find it on my google drive : >>> > >>> > >>> https://drive.google.com/file/d/1eAaZMfd6CU0xiqQfH7sq5wGVzzO09BRm/view?usp=sharing >>> > >>> > I need to understand if I can recompile it without problem so that it >>> can satisfy my needs (the ability of the file >>> > u-boot.bin to boot FreeBSD as domU under Xen,as explained by Stefano >>> Stabellini,the xen developer that suggested to me >>> > what I could do to have FreeBSD virtualized under Xen on my Arm >>> Chromebook) ; otherwise the risk is to find later >>> > problems that will make me troubles and that I will not able to fix. >>> > >>> > I gave a look at the virtual open system u-boot and I didn't see any >>> arndale_defconfig inside. So,If I have understood >>> > correctly,I should put that file inside the root of the u-boot source >>> code,let's say here : >>> > >>> > marietto:/home/marietto/Desktop/Files/u-boot_FreeBSD/u-boot-vos # ls >>> > >>> > .checkpatch.conf README doc >>> net >>> > .git api drivers >>> onenand_ipl >>> > .gitignore arch dts >>> post >>> > COPYING board examples >>> rules.mk >>> > CREDITS boards.cfg fs >>> scripts >>> > MAINTAINERS common include >>> snapshot.commit >>> > MAKEALL config.mk lib >>> spl >>> > Makefile cros mkconfig >>> test >>> > PRESUBMIT.cfg disk nand_spl >>> tools >>> > >>> > and I should do : make and make install ? and the file I >>> need,u-boot.bin will be generated ? >>> > >>> > I didn't find any pre made configuration file inside : >>> > >>> > u-boot-vos # find . -type f -name "exynos*" >>> > >>> > ./include/exynos-fb.h >>> > ./include/configs/exynos5-common.h >>> > ./doc/device-tree-bindings/spi/exynos-spi.txt >>> > ./doc/device-tree-bindings/usb/exynos-usb.txt >>> > ./drivers/power/exynos-tmu.c >>> > ./drivers/power/exynos-cpufreq.c >>> > ./drivers/video/exynos-fb.c >>> > ./drivers/spi/exynos_spi.c >>> > ./board/samsung/dts/exynos5250-spring.dts >>> > ./board/samsung/dts/exynos5250-smdk5250.dts >>> > ./board/samsung/dts/exynos5250-snow.dts >>> > ./board/samsung/dts/exynos5250-daisy.dts >>> > ./arch/arm/include/asm/arch-exynos5/exynos-cpufreq.h >>> > ./arch/arm/include/asm/arch-exynos5/exynos-tmu.h >>> > ./arch/arm/dts/exynos5250.dtsi >>> > ./arch/arm/dts/exynos-periph-id.dtsi >>> > ./arch/arm/cpu/armv7/exynos5/exynos_cache.c >>> > >>> > u-boot-vos # find . -type f -name "arndale*" >>> > >>> > For sure I can't use a newer version of u-boot because otherwise the >>> patches needed to bypass the bootloader protections >>> > of the Arm Chromebook (such as a lot of different patches needed to >>> boot correctly Linux) will be broken ; anyway,since >>> > it works,I don't need to use an updated version of u-boot. >>> > >>> > ----> As per my experience, you have to respect these two options, >>> compiling u-boot for >>> > FreeBSD: >>> https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment >>> > >>> > It says that I should use these parameters : >>> > >>> > CONFIG_ARMV7_NONSEC=n >>> > CONFIG_EFI_GRUB_ARM32_WORKAROUND=y >>> > >>> > These are the parameters used to configure a Linux kernel. I don't >>> understand what's the relation between the compilation >>> > of a linux kernel and u-boot. In the past I tried to recompile >>> u-boot,but I didn't have the need to set up those >>> > parameters,so I don't know how to do it (but I know how to recompile a >>> Linux kernel). >>> > >>> > ---> I'm not sure that I'm getting you right, as I don't understand >>> what you mean under "the first u-boot". >>> > >>> > >>> > I'm talking about first u-boot because the whole procedure to boot >>> Linux on the ARM Chromebook,that's explained here : >>> > >>> > >>> http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/ >>> > >>> > >>> > at some point they say : >>> > >>> > >>> > To be able to run KVM on ARM platforms, the kernel has to be booted in >>> hypervisor mode. Because of this relatively recent >>> > requirement (due to the introduction of the virtualization >>> extensions), up until now all booting methods would boot the >>> > kernel in the standard Supervisor mode. >>> > >>> > For the ARM Chromebook the default boot procedure doesn't allow us to >>> boot in hypervisor mode. Although the laptop's boot >>> > mechanism is based on the frequently used u-boot, the binary is >>> located in RO memory. Fortunately, a chained u-boot >>> > mechanism can be used (i.e. starting another u-boot after the >>> original). We can then enter hypervisor mode from our >>> > custom iteration of u-boot and subsequently load our kernel and >>> userspace. >>> > >>> > So,the first u-boot is the u-boot provided by virtual open >>> systems,that's able to chainload the "u-boot binary located in >>> > RO memory" , that does not boot Chrome OS in hypervisor mode. We don't >>> need it if we want to boot Linux with kvm or xen >>> > enabled. >>> > >>> > >>> > On Sun, Dec 17, 2023 at 1:28 AM Stanislav Silnicki < >>> stanislav.silnicki@mailgate.us> wrote: >>> > I'm not an expert in the topic, I only know, that ARM has >>> divided hardware into two worlds - Secure and >>> > Not-So, strictly limiting any software, running in non-secure >>> world with access to functions and >>> > resources. >>> https://developer.arm.com/documentation/den0013/d/Security/TrustZone-hardware-architecture?lang=en >>> <https://developerarm.com/documentation/den0013/d/Security/TrustZone-hardware-architecture?lang=en>; >>> > >>> > I'm not sure, that I'm getting you right, as I don't understand what >>> you mean under "the first u-boot". >>> > >>> > As I understand, virtualization (HYP) is running in non-secure world( >>> https://developer.arm.com/documentation/ddi0406/c/System-Level-Architecture/The-System-Level-Programmers--Model/The-Virtualization-Extens >>> > ions), so my guess (only guess!!!), virtualization software has to >>> prepare (configure) HW platform in the way, >>> > that FreeBSD kernel will not lack any resources, required to configure >>> MPU, VA, etc. >>> > So, if you lucky to boot virtualizer, which is aware of target OS, >>> that maybe you can boot the kernel. Although, I >>> > doubt, that you need to boot 'second' u-boot to boot the kernel - >>> there is simply ubldr, which you can hook somehow >>> > from virtualizer.... >>> > >>> > Stan >>> > >>> > >>> > >>> > Mario Marietto wrote: >>> > >>> > >>> > ---> As I understand, it makes sure that u-boot keeps in secure >>> mode during boot and passes control to >>> > ubldr, which boots FreeBSD kernel, in that mode. >>> > >>> > Can you elaborate your sentence more ? I know that the bootloader >>> secure mode is bypassed by the virtual open >>> > systems u-boot. Are you saying that when the control passes to the >>> second u-boot,it will happen in secure >>> > mode,so that the bypass that happened loading the first u-boot,is >>> annulled ? If this is true,maybe can I boot >>> > FreeBSD using the virtual-open-system custom u-boot ? Is this >>> compatible with FreeBSD ? Where can I find the >>> > u-boot.bin that the xen developer talked about ? thanks bro'. >>> > >>> > >>> > >>> > On Sun, Dec 17, 2023 at 12:35 AM Stanislav Silnicki < >>> stanislav.silnicki@mailgate.us> wrote: >>> > Hi Mario, >>> > >>> > U-Boot beast is hiding in this den: >>> https://source.denx.de/u-boot/u-boot.git >>> > I took a brief look at your post and it seems to me, that >>> option CONFIG_CMO_BY_VA_ONLY is irrelevant to >>> > your target armv7 32 bit >>> > platform: >>> https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/cpu/armv8/Kconfig?ref_type=heads#L3 >>> > >>> > As for compiling the u-boot, it is a doable task, given that you >>> understand what you are doing. There >>> > are no specific options in u-boot devoted to FreeBSD. It is a boot >>> loader, whose mission to make basic >>> > hardware initialization, read you kernel file from some media into RAM >>> and then pass it control. >>> > >>> > Basically, you can grab some defconfig, prepared for any other >>> Exynos5250 based board (say, this one: >>> > >>> https://source.denx.de/u-boot/u-boot/-/blob/master/configs/arndale_defconfig?ref_type=heads) >>> and adopt >>> > it somehow. >>> > >>> > As per my experience, you have to respect these two options, compiling >>> u-boot for >>> > FreeBSD: >>> https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment >>> > >>> > As I understand, it makes sure, that u-boot keeps in secure mode >>> during boot and passes control to >>> > ubldr, which boots FreBSD kernel, in that mode. Otherwise, there a lot >>> of surprises you may realize. >>> > >>> > Hope, this will help to progress you tasks >>> > Stan >>> > >>> > Mario Marietto wrote: >>> > >>> > >>> > Hello. >>> > >>> > I'm trying to boot FreeBSD for arm32 bit as DomU on my ARM >>> Chromebook. Basically there are >>> > two ways to accomplish this task : >>> > >>> > 1) to write a patch that allows the FreeBSD kernel to boot as a >>> zImage file. This could be >>> > accomplished applying this patch to a specific file that's on >>> the source code of FreeBSD : >>> > >>> > >>> > >>> https://xenbits.xen.org/gitweb/?p=p...8;hb=0782e25d98cc1391472717035f986c979edef0c9 >>> > >>> > >>> > This patch was written by Julien Grall a lot of time ago and now >>> it does not work anymore. >>> > This is the reason : >>> > >>> > >>> > It appears FreeBSD-CURRENT removed the last step >>> converting the kernel file to >>> > kernel.bin. The patch can be readily rebased, but without >>> kernel.bin that >>> > doesn't do too much >>> > >>> > >>> > >>> > So,without a rebase of that patch the first option is not applicable. >>> And I'm not able to fix it. >>> > >>> > 2) booting FreeBSD using U-Boot,as explained to me by a xen developer : >>> > >>> > >>> > I was trying to explain why and how Julien's patch works so that >>> you could be the one >>> > to re-do something similar or fix the patch on the FreeBSD >>> kernel that you are >>> > working with. I am happy to help review and write patches but I >>> don't work with the >>> > FreeBSD kernel so I wouldn't be able to help you quickly. >>> However, I might have a >>> > suggestion. Do you know if FreeBSD can be booted by U-Boot ? >>> Because U-Boot >>> > definitely boots as Xen on ARM guest firmware/bootloader. You >>> should be able to build >>> > U-Boot and use the U-Boot binary as Xen guest kernel, then >>> U-Boot could load FreeBSD >>> > from disk or network and start it. For instance as domU config >>> file: >>> > >>> > kernel="/home/petalinux/u-boot.bin" >>> > disk = [ '/home/petalinux/test.img,raw,xvda' ] >>> > >>> > I know it is important to build u-boot with the following config >>> to make it work on >>> > Xen. >>> > >>> > CONFIG_CMO_BY_VA_ONLY=y >>> > >>> > >>> > >>> > This option seems more doable to me according to my knowledge. But I >>> need to understand how to do >>> > it. >>> > >>> > Well,let's say that on the ARM Chromebook I'm forced to use and >>> install a customized version of >>> > u-boot,created by virtual open systems,because it is the only one that >>> allows bypassing its >>> > bootloader protection. You can find more information here : >>> > >>> > >>> http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/?vos=tech >>> > >>> > This is the relevant section to read : >>> > >>> > >>> > Bootloader : >>> > >>> > If you wish to skip this chapter you can download a pre-compiled >>> binary of the >>> > bootloader: >>> > >>> > >>> > $ wget >>> > >>> http://www.virtualopensystems.com/downloads/guides/kvm_on_chromebook/nv_u-boot-snow.kpart >>> > >>> > >>> > To be able to run KVM on ARM platforms, the kernel has to be >>> booted in hypervisor >>> > mode. Because of this relatively recent requirement (due to the >>> introduction of the >>> > virtualization extensions), up until now all booting methods >>> would boot the kernel in >>> > the standard Supervisor mode. For the ARM Chromebook the default >>> boot procedure >>> > doesn't allow us to boot in hypervisor mode. Although the >>> laptop's boot mechanism is >>> > based on the frequently used u-boot, the binary is located in RO >>> memory. Fortunately, >>> > a chained u-boot mechanism can be used (i.e. starting another >>> u-boot after the >>> > original). We can then enter hypervisor mode from our custom >>> iteration of u-boot and >>> > subsequently load our kernel and userspace. >>> > >>> > Checkout the needed u-boot code : >>> > >>> > >>> > $ git clone git://github.com/virtualopensystems/u-boot.git$ cd >>> u-boot$ >>> > ./scripts/build.sh >>> > >>> > >>> > If successful, a message about how to copy the bootloader on the >>> USB flash disk or SD >>> > card will appear. We will use it later when preparing the boot >>> medium to start our >>> > system. If you have followed the Setting up the boot medium >>> chapter and you have a >>> > prepared boot device, then you can update u-boot by running : >>> > >>> > >>> > $ sudo dd if=nv_uboot-snow.kpart of=/dev/sdX1 >>> > >>> > >>> > >>> > so,the needed u-boot that we must use should be installed on the first >>> partition of the sd card. >>> > >>> > There is another relevant section to read : >>> > >>> > >>> > Setting up the boot medium >>> > >>> > Now it is time to copy all the relevant files that we created in >>> the previous >>> > chapters,and use them to boot Chromebook with a different kernel >>> and OS. In all these >>> > examples the device /dev/sdX is used. Take extra care to change >>> the examples to the >>> > device that you have attached. Insert the boot medium on your >>> workstation and >>> > carefully execute the following step. First we need to properly >>> format the boot >>> > medium. >>> > >>> > In the uboot source directory : >>> > >>> > >>> > $ sudo ./scripts/sdcard.sh /dev/sdX >>> > >>> > >>> > This will erase all data and create 4 partitions in the medium, >>> along with copying >>> > the u-boot binary to the first partition: >>> > >>> > >>> > Partition 1 = ChromeOS signed binary (V.O.S chained u-boot) >>> > Partition 2 = not used >>> > Partition 3 = EXT2 partition for u-boot files (uImage and >>> exynos5250-snow.dtb) >>> > Partition 4 = EXT4 partition for userspace files >>> > >>> > >>> > With u-boot being copied, next is the kernel image and DTB file. >>> From the kernel >>> > source execute : >>> > >>> > >>> > $ mkdir ../mnt/ >>> > $ sudo mount /dev/sdX3 ../mnt/ >>> > $ sudo cp arch/arm/boot/uImage ../mnt/ >>> > $ sudo cp arch/arm/boot/dts/exynos5250-snow.dtb ../mnt/ >>> > $ sudo umount /dev/sdX3 >>> > >>> > >>> > Finally, we have to copy the Ubuntu userspace filesystem that we >>> created earlier: >>> > >>> > >>> > $ sudo mount /dev/sdX4 mnt/$ sudo cp -a ./precise/* mnt/$ sudo >>> umount /dev/sdX4 >>> > >>> > >>> > >>> > Now,my idea is to chainload the already chain loaded u-boot created by >>> V.O.S to the new u-boot >>> > that we need for booting FreeBSD and that can be installed in the >>> partition n.2,as shown in this >>> > scheme,because it is not used : >>> > >>> > >>> > Partition 1 = ChromeOS signed binary (V.O.S chained u-boot) >>> > Partition 2 = not used (maybe we can install the u-boot for arm 32 >>> bit,compatible with FreeBSD on >>> > this partition) >>> > Partition 3 = EXT2 partition for u-boot files (uImage and >>> exynos5250-snow.dtb) >>> > Partition 4 = EXT4 partition for userspace files >>> > >>> > >>> > Take in consideration that default boot string is hardcoded here,in >>> the snow.h file of the custom >>> > u-boot created by VOS : >>> > >>> > >>> > >>> https://github.com/virtualopensyste...18a39b6c177dff58a/include/configs/snow.h#L101 >>> > >>> > >>> > and it needs to be recompiled because it should point to the partition >>> n.2,where I will install >>> > the u-boot files as explained here : >>> > >>> > >>> > https://wiki.freebsd.org/arm/Chromebook >>> > >>> > >>> > I have some questions to ask before I start working on this. >>> > >>> > 1) The xen developer said : >>> > >>> > >>> > You should be able to build U-Boot and use the U-Boot binary as >>> Xen guest kernel... >>> > >>> > >>> > >>> > where is the u-boot binary,according to this document ? >>> > >>> > https://wiki.freebsd.org/arm/Chromebook >>> > >>> > I don't see it. >>> > >>> > >>> > 2) where is the source code of the file that I can get here : >>> > >>> > >>> http://commondatastorage.googleapis.com/chromeos-localmirror/distfiles/nv_uboot-snow-simplefb.kpart.bz2 >>> > >>> > I need the source code if I want to recompile u-boot so that it can >>> point to the partition 4. >>> > >>> > Maybe it can be found on this link : >>> > >>> > http://linux-exynos.org/dist/chromebook/nv_uboot/ >>> > >>> > but it can't be opened.... >>> > >>> > >>> > 3) in this specific scenario the source code of u-boot should run on >>> arm 32 bit,not on arm >>> > 64,because I have the Samsung Chromebook "SNOW" model XE303C12,that's >>> powered by a Samsung Exynos >>> > 5250 (ARMv7 32 bit Cortex A15) Soc. >>> > >>> > >>> > 4) I'm not sure if I can chainload the customized u-boot created by >>> V.O.S that should be >>> > installed on the first partition with the u-boot tailored for booting >>> FreeBSD that should be >>> > installed on the partition 2.... >>> > >>> > >>> > 5) the xen developer said that u-boot should be compiled enabling this >>> option : >>> > >>> > >>> > Code: >>> > >>> > CONFIG_CMO_BY_VA_ONLY=y >>> > >>> > >>> > Well,can you provide some good source that can help me to understand >>> how I can recompile u-boot >>> > for FreeBSD ? thanks. >>> > >>> > -- >>> > Mario. >>> > >>> > >>> > >>> > -- >>> > Mario. >>> > >>> > >>> > >>> > -- >>> > Mario. >>> > >>> > >>> > >>> > -- >>> > Mario. >>> > >>> > >>> > >>> > -- >>> > Mario. >>> > >>> > >>> > >>> > -- >>> > Mario. >>> > >>> > >> >> >> >> -- >> Mario. >> > -- Mario. [-- Attachment #2 --] <div dir="ltr"><div><span style="font-family:monospace">I've added this parameter to bootxen.source :</span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">guest_loglvl=all</span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">bootxen.source :</span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">mmc dev 1 <br>ext2load mmc 1:3 0x42000000 zImage-5.4.261-iommu-dma-on-xen <br>ext2load mmc 1:3 0x51000000 xen-4.17-armhf-armmp-0x51004000.ub <br>ext2load mmc 1:3 0x5ffec000 exynos5250-snow.dtb <br>fdt addr 0x5ffec000 <br>fdt resize 1024 <br>fdt set /chosen \#address-cells <0x2> <br>fdt set /chosen \#size-cells <0x2> <br>fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0 dom0_mem=1152M dom0_max_vcpus=2 bootscrub=0 vwfi=native guest_loglvl=all"<br>fdt mknod /chosen dom0 <br>fdt set /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module" "multiboot,module" <br>fdt set /chosen/dom0 reg <0x0 0x42000000 0x0 0x49F9A8 > <br>fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 rw rootwait clk_ignore_unused --no-log" <br>bootm 0x51000000 - 0x5ffec000<br> <br></span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">but when I try to boot FreeBSD I don't get more informations than usual :<br></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">root@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen# ./start-freebsd </span></span></div><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">Parsing config from freebsd.cfg </span></div><div><span style="font-family:monospace"></span></div><span style="font-family:monospace">xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found: Invalid kernel <br>libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed <br>libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 1:cannot (re-)build domain: -3 <br>libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 1:Non-existent domain <br>libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable to destroy guest <br>libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction of domain failed <br></span><div><span style="font-family:monospace">freebsd is an invalid domain identifier (rc=-6)</span></div><br><div>Are you aware about a new parameter that I can use to have more detailed debug information ?<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 20, 2023 at 5:52 AM Warner Losh <<a href="mailto:imp@bsdimp.com">imp@bsdimp.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>I'd think you'd need the right virtualization loader. I'm not entirely sure the u-boot.bin you've been creating is for a dom-u.. </div><div>If I misunderstood, then the below isn't good advice. Chain booting the u-boot, the first u-boot initializes things so you want</div><div>to start with stage after the SPL. But the different error messages suggest that it's trying to reboot with kexec, which</div><div>isn't supported on armv7 at the moment.<br></div><div><br></div><div>If you could boot in kvm, I think that the following would work.... Though I'm not entirely sure how to</div><div>specify the two .fd files in your setup. The use of qemu is to have an easy env to debug things... I don't</div><div>have a chromebook to try...<br></div><div><br></div><div>My first instinct would be to try qemu on x86 (this is the first step of many to get to your destination).</div><div><br></div><div>If you could boot the GENERIC_SD image that we produce using qemu + edk2-arm-code.fd that would</div><div>be a huge first step. This will give you the boot loader, I believe, to boot in the VM that you need better</div><div>than going via the u-boot route. Since you are booting in a virtualized environment, I think it wouldn't</div><div>matter which one :).</div><div><br></div><div>So, I did the following to boot the virtualized armv7 FreeBSD environment, following a post on the forums I found and knew to have the right recipe:</div><div><a href="https://forums.freebsd.org/threads/run-boot-freebsd-arm-32bit-image-in-qemu.80765/" target="_blank">https://forums.freebsd.org/threads/run-boot-freebsd-arm-32bit-image-in-qemu.80765/</a></div><div><br></div><div>1. pkg install qemu</div><div>2. mkdir qemu-armv7-env</div><div>3. cd qemu-armv7-env</div><div>4. fetch <a href="https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/14.0/FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD.img.xz" target="_blank">https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/14.0/FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD.img.xz</a></div><div>5. xz -d -T 0 FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD.img.xz</div><div>6. dd if=/dev/zero of=pflash0.img bs=1m count=64<br>7. dd if=/dev/zero of=pflash1.img bs=1m count=64<br>8. dd if=/usr/local/share/qemu/edk2-arm-code.fd of=pflash0.img conv=notrunc<br>9. dd if=/usr/local/share/qemu/edk2-arm-vars.fd of=pflash1.img conv=notrunc</div><div>10. cat > start-freebsd-arm.sh</div><div>#!/bin/sh<br>qemu-system-arm \<br> -M virt \<br> -m 1024 \<br> -drive file=pflash0.img,format=raw,if=pflash,readonly=on \<br> -drive file=pflash1.img,format=raw,if=pflash \<br> -drive file=$1.img,if=virtio,cache=writethrough \<br> -nographic \<br> -serial mon:stdio</div><div>^D</div><div>11. chmod +x start-freebsd-arm.sh</div><div>12. ./start-freebsd-arm.sh FreeBSD-14.0-RELEASE-arm-armv7-GENERICSD</div><div><br></div><div>But I hit a snag with this on qemu 8.1.2 and 8.1.3 with both 13.2 and 14.0:</div><div><br></div><div>Starting devd.<br>Starting dhclient.<br>DHCPDISCOVER on vtnet0 to 255.255.255.255 port 67 interval 7<br>Fatal kernel mode data abort: 'Alignment Fault' on read<br>trapframe: 0xc4b36a60<br>FSR=00000001, FAR=dd96701a, spsr=20000013<br>r0 =00000000, r1 =00000001, r2 =00000001, r3 =c4b36b4c<br>r4 =00000014, r5 =d6618800, r6 =dd96702e, r7 =0000022c<br>r8 =00000000, r9 =0000022c, r10=dd96701a, r11=c4b36b90<br>r12=4300ffff, ssp=c4b36af0, slr=c04a9728, pc =c04a9750<br><br>panic: Fatal abort<br>cpuid = 0<br>time = 1680843057<br>KDB: stack backtrace:<br>#0 0xc035786c at kdb_backtrace+0x48<br>#1 0xc02fdd20 at vpanic+0x140<br>#2 0xc02fdbe0 at vpanic+0<br>#3 0xc06304ac at abort_align+0<br>#4 0xc063052c at abort_align+0x80<br>#5 0xc063017c at abort_handler+0x480<br>#6 0xc060f480 at exception_exit+0<br>#7 0xc04a9750 at udp_input+0x288<br>#8 0xc0473f54 at ip_input+0x1e0<br>#9 0xc04447c0 at netisr_dispatch_src+0xf8<br>#10 0xc043bf2c at ether_demux+0x1a4<br>#11 0xc043d5e4 at ether_nh_input+0x480<br>#12 0xc04447c0 at netisr_dispatch_src+0xf8<br>#13 0xc043c404 at ether_input+0x50<br>#14 0xc01c0838 at vtnet_rx_vq_process+0x880<br>#15 0xc01b70d0 at vtpci_intx_intr+0xac<br>#16 0xc02b87f0 at ithread_loop+0x2ec<br>#17 0xc02b465c at fork_exit+0xc0<br>Uptime: 19s</div><div><br></div><div>I don't know if this is a problem with qemu or FreeBSD's kernel...</div><div><br></div><div>Warner<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 19, 2023 at 3:25 PM Mario Marietto <<a href="mailto:marietto2008@gmail.com" target="_blank">marietto2008@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I've asked some help on the channel #arm on Reddit and someone replied :</div><div><br></div><div><a href="https://www.reddit.com/r/arm/comments/18mcir8/i_cant_boot_freebsd_for_arm32_bit_as_domu_with/" target="_blank">https://www.reddit.com/r/arm/comments/18mcir8/i_cant_boot_freebsd_for_arm32_bit_as_domu_with/</a></div><div><br></div><div>Maybe his answer can be useful to understand why it does not work. <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 19, 2023 at 8:33 PM Stefano Stabellini <<a href="mailto:sstabellini@kernel.org" target="_blank">sstabellini@kernel.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">+Michal<br> <br> Hi Mario,<br> <br> I am not sure about booting FreeBSD, but I am certain that u-boot works<br> fine as DomU kernel on ARMv8 (not sure about ARMv7). With this config<br> file:<br> <br> name="test"<br> kernel="u-boot.bin"<br> extra = "console=hvc0"<br> memory=256<br> vcpus=1<br> disk = [ 'FreeBSD-13.2-RELEASE-armv7.img,raw,xvda' ]<br> <br> I don't know for sure if you can boot FreeBSD but you should definitely<br> be able to see the u-boot command line prompt. The fact that you are<br> getting this message:<br> <br> xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found: Invalid kernel<br> <br> Means that something is not right in the u-boot configuration or u-boot<br> build. Michal and Artem (CCed) might know more. From what I recall,<br> there was nothing special required to get u-boot.bin to boot as domU<br> kernel, so now I wonder if it is an ARMv7 vs. ARMv8 issue.<br> <br> Cheers,<br> <br> Stefano<br> <br> <br> On Tue, 19 Dec 2023, Mario Marietto wrote:<br> > ....I see that some other interesting files have been produced by u-boot when I have compiled it :<br> > <br> > u-boot<br> > u-boot.lds<br> > u-boot.bin<br> > u-boot.map<br> > u-boot-nodtb.bin<br> > u-boot.dtb<br> > u-boot.srec<br> > u-boot-dtb.bin<br> > u-boot.sym<br> > <br> > So,maybe I should use a different u-boot* file for booting FreeBSD ?<br> > <br> > <br> > On Tue, Dec 19, 2023 at 4:28 PM Mario Marietto <<a href="mailto:marietto2008@gmail.com" target="_blank">marietto2008@gmail.com</a>> wrote:<br> > Hello to everyone.<br> > <br> > I have compiled the needed u-boot.bin from scratch using this procedure :<br> > <br> > # git clone <a href="https://github.com/u-boot/u-boot.git" rel="noreferrer" target="_blank">https://github.com/u-boot/u-boot.git</a><br>; > # cd u-boot<br> > # ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make snow_defconfig : this line generates the file .config<br> > # nano .config and I've added these parameters :<br> > <br> > CONFIG_ARMV7_NONSEC=n<br> > CONFIG_EFI_GRUB_ARM32_WORKAROUND=y<br> > <br> > the uboot-bin file is generated with this command :<br> > <br> > # ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make<br> > <br> > At this point,I took a look inside the .config file and I saw that the parameter "CONFIG_ARMV7_NONSEC=n" has been removed. So,for<br> > some reason,it is not accepted and this could be a problem....<br> > <br> > These are the xen config files that I've used :<br> > <br> > nano freebsd.cfg<br> > <br> > name="test"<br> > kernel="u-boot.bin"<br> > extra = "console=hvc0"<br> > memory=256<br> > vcpus=1<br> > disk = [ 'FreeBSD-13.2-RELEASE-armv7.img,raw,xvda' ]<br> > <br> > nano start-freebsd<br> > <br> > xl create freebsd.cfg<br> > xl console freebsd<br> > <br> > This is what happens when I launch the vm :<br> > <br> > # ./start-freebsd<br> > <br> > Parsing config from freebsd.cfg<br> > xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found: Invalid kernel<br> > libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed<br> > libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 1:cannot (re-)build domain: -3<br> > libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 1:Non-existent domain<br> > libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable to destroy guest<br> > libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction of domain failed<br> > freebsd is an invalid domain identifier (rc=-6)<br> > <br> > <br> > On Mon, Dec 18, 2023 at 12:39 PM Mario Marietto <<a href="mailto:marietto2008@gmail.com" target="_blank">marietto2008@gmail.com</a>> wrote:<br> > So,ok,I should have said "the second u-boot" ; since the first u-boot binary is the "u-boot binary located in the RO<br> > memory" of the Chromebook". Sorry for the confusion.<br> > <br> > On Mon, Dec 18, 2023 at 12:35 PM Mario Marietto <<a href="mailto:marietto2008@gmail.com" target="_blank">marietto2008@gmail.com</a>> wrote:<br> > ---> There are no specific options in u-boot devoted to FreeBSD<br> > <br> > This is an important factor. So,what about if,instead of compiling a new version of u-boot on the partition 2,I will<br> > recompile the u-boot customized version created by the virtual open system in 2014,that should be installed on the first<br> > partition ? It could work if there are no differences between the u-boot that should boot Linux and the u-boot that<br> > should boot FreeBSD.<br> > <br> > Can you give a look at the u-boot source code created by virtual open systems ? You can find it on my google drive :<br> > <br> > <a href="https://drive.google.com/file/d/1eAaZMfd6CU0xiqQfH7sq5wGVzzO09BRm/view?usp=sharing" rel="noreferrer" target="_blank">https://drive.google.com/file/d/1eAaZMfd6CU0xiqQfH7sq5wGVzzO09BRm/view?usp=sharing</a><br>; > <br> > I need to understand if I can recompile it without problem so that it can satisfy my needs (the ability of the file<br> > u-boot.bin to boot FreeBSD as domU under Xen,as explained by Stefano Stabellini,the xen developer that suggested to me<br> > what I could do to have FreeBSD virtualized under Xen on my Arm Chromebook) ; otherwise the risk is to find later<br> > problems that will make me troubles and that I will not able to fix.<br> > <br> > I gave a look at the virtual open system u-boot and I didn't see any arndale_defconfig inside. So,If I have understood<br> > correctly,I should put that file inside the root of the u-boot source code,let's say here :<br> > <br> > marietto:/home/marietto/Desktop/Files/u-boot_FreeBSD/u-boot-vos # ls<br> > <br> > .checkpatch.conf README doc net<br> > .git api drivers onenand_ipl<br> > .gitignore arch dts post<br> > COPYING board examples <a href="http://rules.mk" rel="noreferrer" target="_blank">rules.mk</a><br> > CREDITS boards.cfg fs scripts<br> > MAINTAINERS common include snapshot.commit<br> > MAKEALL <a href="http://config.mk" rel="noreferrer" target="_blank">config.mk</a> lib spl<br> > Makefile cros mkconfig test<br> > PRESUBMIT.cfg disk nand_spl tools<br> > <br> > and I should do : make and make install ? and the file I need,u-boot.bin will be generated ? <br> > <br> > I didn't find any pre made configuration file inside :<br> > <br> > u-boot-vos # find . -type f -name "exynos*" <br> > <br> > ./include/exynos-fb.h<br> > ./include/configs/exynos5-common.h<br> > ./doc/device-tree-bindings/spi/exynos-spi.txt<br> > ./doc/device-tree-bindings/usb/exynos-usb.txt<br> > ./drivers/power/exynos-tmu.c<br> > ./drivers/power/exynos-cpufreq.c<br> > ./drivers/video/exynos-fb.c<br> > ./drivers/spi/exynos_spi.c<br> > ./board/samsung/dts/exynos5250-spring.dts<br> > ./board/samsung/dts/exynos5250-smdk5250.dts<br> > ./board/samsung/dts/exynos5250-snow.dts<br> > ./board/samsung/dts/exynos5250-daisy.dts<br> > ./arch/arm/include/asm/arch-exynos5/exynos-cpufreq.h<br> > ./arch/arm/include/asm/arch-exynos5/exynos-tmu.h<br> > ./arch/arm/dts/exynos5250.dtsi<br> > ./arch/arm/dts/exynos-periph-id.dtsi<br> > ./arch/arm/cpu/armv7/exynos5/exynos_cache.c <br> > <br> > u-boot-vos # find . -type f -name "arndale*"<br> > <br> > For sure I can't use a newer version of u-boot because otherwise the patches needed to bypass the bootloader protections<br> > of the Arm Chromebook (such as a lot of different patches needed to boot correctly Linux) will be broken ; anyway,since<br> > it works,I don't need to use an updated version of u-boot.<br> > <br> > ----> As per my experience, you have to respect these two options, compiling u-boot for<br> > FreeBSD: <a href="https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment" rel="noreferrer" target="_blank">https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment</a><br>; > <br> > It says that I should use these parameters :<br> > <br> > CONFIG_ARMV7_NONSEC=n<br> > CONFIG_EFI_GRUB_ARM32_WORKAROUND=y<br> > <br> > These are the parameters used to configure a Linux kernel. I don't understand what's the relation between the compilation<br> > of a linux kernel and u-boot. In the past I tried to recompile u-boot,but I didn't have the need to set up those<br> > parameters,so I don't know how to do it (but I know how to recompile a Linux kernel).<br> > <br> > ---> I'm not sure that I'm getting you right, as I don't understand what you mean under "the first u-boot".<br> > <br> > <br> > I'm talking about first u-boot because the whole procedure to boot Linux on the ARM Chromebook,that's explained here :<br> > <br> > <a href="http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/" rel="noreferrer" target="_blank">http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/</a><br>; > <br> > <br> > at some point they say :<br> > <br> > <br> > To be able to run KVM on ARM platforms, the kernel has to be booted in hypervisor mode. Because of this relatively recent<br> > requirement (due to the introduction of the virtualization extensions), up until now all booting methods would boot the<br> > kernel in the standard Supervisor mode.<br> > <br> > For the ARM Chromebook the default boot procedure doesn't allow us to boot in hypervisor mode. Although the laptop's boot<br> > mechanism is based on the frequently used u-boot, the binary is located in RO memory. Fortunately, a chained u-boot<br> > mechanism can be used (i.e. starting another u-boot after the original). We can then enter hypervisor mode from our<br> > custom iteration of u-boot and subsequently load our kernel and userspace.<br> > <br> > So,the first u-boot is the u-boot provided by virtual open systems,that's able to chainload the "u-boot binary located in<br> > RO memory" , that does not boot Chrome OS in hypervisor mode. We don't need it if we want to boot Linux with kvm or xen<br> > enabled.<br> > <br> > <br> > On Sun, Dec 17, 2023 at 1:28 AM Stanislav Silnicki <<a href="mailto:stanislav.silnicki@mailgate.us" target="_blank">stanislav.silnicki@mailgate.us</a>> wrote:<br> > I'm not an expert in the topic, I only know, that ARM has divided hardware into two worlds - Secure and<br> > Not-So, strictly limiting any software, running in non-secure world with access to functions and<br> > resources. <a href="https://developerarm.com/documentation/den0013/d/Security/TrustZone-hardware-architecture?lang=en" rel="noreferrer" target="_blank">https://developer.arm.com/documentation/den0013/d/Security/TrustZone-hardware-architecture?lang=en</a><br>; > <br> > I'm not sure, that I'm getting you right, as I don't understand what you mean under "the first u-boot".<br> > <br> > As I understand, virtualization (HYP) is running in non-secure world(<a href="https://developer.arm.com/documentation/ddi0406/c/System-Level-Architecture/The-System-Level-Programmers--Model/The-Virtualization-Extens" rel="noreferrer" target="_blank">https://developer.arm.com/documentation/ddi0406/c/System-Level-Architecture/The-System-Level-Programmers--Model/The-Virtualization-Extens</a><br>; > ions), so my guess (only guess!!!), virtualization software has to prepare (configure) HW platform in the way,<br> > that FreeBSD kernel will not lack any resources, required to configure MPU, VA, etc.<br> > So, if you lucky to boot virtualizer, which is aware of target OS, that maybe you can boot the kernel. Although, I<br> > doubt, that you need to boot 'second' u-boot to boot the kernel - there is simply ubldr, which you can hook somehow<br> > from virtualizer....<br> > <br> > Stan<br> > <br> > <br> > <br> > Mario Marietto wrote:<br> > <br> > <br> > ---> As I understand, it makes sure that u-boot keeps in secure mode during boot and passes control to<br> > ubldr, which boots FreeBSD kernel, in that mode.<br> > <br> > Can you elaborate your sentence more ? I know that the bootloader secure mode is bypassed by the virtual open<br> > systems u-boot. Are you saying that when the control passes to the second u-boot,it will happen in secure<br> > mode,so that the bypass that happened loading the first u-boot,is annulled ? If this is true,maybe can I boot<br> > FreeBSD using the virtual-open-system custom u-boot ? Is this compatible with FreeBSD ? Where can I find the<br> > u-boot.bin that the xen developer talked about ? thanks bro'.<br> > <br> > <br> > <br> > On Sun, Dec 17, 2023 at 12:35 AM Stanislav Silnicki <<a href="mailto:stanislav.silnicki@mailgate.us" target="_blank">stanislav.silnicki@mailgate.us</a>> wrote:<br> > Hi Mario,<br> > <br> > U-Boot beast is hiding in this den: <a href="https://source.denx.de/u-boot/u-boot.git" rel="noreferrer" target="_blank">https://source.denx.de/u-boot/u-boot.git</a><br>; > I took a brief look at your post and it seems to me, that option CONFIG_CMO_BY_VA_ONLY is irrelevant to<br> > your target armv7 32 bit<br> > platform: <a href="https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/cpu/armv8/Kconfig?ref_type=heads#L3" rel="noreferrer" target="_blank">https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/cpu/armv8/Kconfig?ref_type=heads#L3</a><br>; > <br> > As for compiling the u-boot, it is a doable task, given that you understand what you are doing. There<br> > are no specific options in u-boot devoted to FreeBSD. It is a boot loader, whose mission to make basic<br> > hardware initialization, read you kernel file from some media into RAM and then pass it control.<br> > <br> > Basically, you can grab some defconfig, prepared for any other Exynos5250 based board (say, this one:<br> > <a href="https://source.denx.de/u-boot/u-boot/-/blob/master/configs/arndale_defconfig?ref_type=heads" rel="noreferrer" target="_blank">https://source.denx.de/u-boot/u-boot/-/blob/master/configs/arndale_defconfig?ref_type=heads</a>) and adopt<br> > it somehow.<br> > <br> > As per my experience, you have to respect these two options, compiling u-boot for<br> > FreeBSD: <a href="https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment" rel="noreferrer" target="_blank">https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment</a><br>; > <br> > As I understand, it makes sure, that u-boot keeps in secure mode during boot and passes control to<br> > ubldr, which boots FreBSD kernel, in that mode. Otherwise, there a lot of surprises you may realize.<br> > <br> > Hope, this will help to progress you tasks<br> > Stan<br> > <br> > Mario Marietto wrote:<br> > <br> > <br> > Hello.<br> > <br> > I'm trying to boot FreeBSD for arm32 bit as DomU on my ARM Chromebook. Basically there are<br> > two ways to accomplish this task :<br> > <br> > 1) to write a patch that allows the FreeBSD kernel to boot as a zImage file. This could be<br> > accomplished applying this patch to a specific file that's on the source code of FreeBSD :<br> > <br> > <br> > <a href="https://xenbits.xen.org/gitweb/?p=p...8;hb=0782e25d98cc1391472717035f986c979edef0c9" rel="noreferrer" target="_blank">https://xenbits.xen.org/gitweb/?p=p...8;hb=0782e25d98cc1391472717035f986c979edef0c9</a><br>; > <br> > <br> > This patch was written by Julien Grall a lot of time ago and now it does not work anymore.<br> > This is the reason :<br> > <br> > <br> > It appears FreeBSD-CURRENT removed the last step converting the kernel file to<br> > kernel.bin. The patch can be readily rebased, but without kernel.bin that<br> > doesn't do too much<br> > <br> > <br> > <br> > So,without a rebase of that patch the first option is not applicable. And I'm not able to fix it.<br> > <br> > 2) booting FreeBSD using U-Boot,as explained to me by a xen developer :<br> > <br> > <br> > I was trying to explain why and how Julien's patch works so that you could be the one<br> > to re-do something similar or fix the patch on the FreeBSD kernel that you are<br> > working with. I am happy to help review and write patches but I don't work with the<br> > FreeBSD kernel so I wouldn't be able to help you quickly. However, I might have a<br> > suggestion. Do you know if FreeBSD can be booted by U-Boot ? Because U-Boot<br> > definitely boots as Xen on ARM guest firmware/bootloader. You should be able to build<br> > U-Boot and use the U-Boot binary as Xen guest kernel, then U-Boot could load FreeBSD<br> > from disk or network and start it. For instance as domU config file:<br> > <br> > kernel="/home/petalinux/u-boot.bin"<br> > disk = [ '/home/petalinux/test.img,raw,xvda' ]<br> > <br> > I know it is important to build u-boot with the following config to make it work on<br> > Xen.<br> > <br> > CONFIG_CMO_BY_VA_ONLY=y<br> > <br> > <br> > <br> > This option seems more doable to me according to my knowledge. But I need to understand how to do<br> > it.<br> > <br> > Well,let's say that on the ARM Chromebook I'm forced to use and install a customized version of<br> > u-boot,created by virtual open systems,because it is the only one that allows bypassing its<br> > bootloader protection. You can find more information here :<br> > <br> > <a href="http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/?vos=tech" rel="noreferrer" target="_blank">http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/?vos=tech</a><br>; > <br> > This is the relevant section to read :<br> > <br> > <br> > Bootloader :<br> > <br> > If you wish to skip this chapter you can download a pre-compiled binary of the<br> > bootloader:<br> > <br> > <br> > $ wget<br> > <a href="http://www.virtualopensystems.com/downloads/guides/kvm_on_chromebook/nv_u-boot-snow.kpart" rel="noreferrer" target="_blank">http://www.virtualopensystems.com/downloads/guides/kvm_on_chromebook/nv_u-boot-snow.kpart</a><br>; > <br> > <br> > To be able to run KVM on ARM platforms, the kernel has to be booted in hypervisor<br> > mode. Because of this relatively recent requirement (due to the introduction of the<br> > virtualization extensions), up until now all booting methods would boot the kernel in<br> > the standard Supervisor mode. For the ARM Chromebook the default boot procedure<br> > doesn't allow us to boot in hypervisor mode. Although the laptop's boot mechanism is<br> > based on the frequently used u-boot, the binary is located in RO memory. Fortunately,<br> > a chained u-boot mechanism can be used (i.e. starting another u-boot after the<br> > original). We can then enter hypervisor mode from our custom iteration of u-boot and<br> > subsequently load our kernel and userspace.<br> > <br> > Checkout the needed u-boot code :<br> > <br> > <br> > $ git clone git://<a href="http://github.com/virtualopensystems/u-boot.git$" rel="noreferrer" target="_blank">github.com/virtualopensystems/u-boot.git$</a> cd u-boot$<br> > ./scripts/build.sh<br> > <br> > <br> > If successful, a message about how to copy the bootloader on the USB flash disk or SD<br> > card will appear. We will use it later when preparing the boot medium to start our<br> > system. If you have followed the Setting up the boot medium chapter and you have a<br> > prepared boot device, then you can update u-boot by running :<br> > <br> > <br> > $ sudo dd if=nv_uboot-snow.kpart of=/dev/sdX1<br> > <br> > <br> > <br> > so,the needed u-boot that we must use should be installed on the first partition of the sd card.<br> > <br> > There is another relevant section to read :<br> > <br> > <br> > Setting up the boot medium<br> > <br> > Now it is time to copy all the relevant files that we created in the previous<br> > chapters,and use them to boot Chromebook with a different kernel and OS. In all these<br> > examples the device /dev/sdX is used. Take extra care to change the examples to the<br> > device that you have attached. Insert the boot medium on your workstation and<br> > carefully execute the following step. First we need to properly format the boot<br> > medium.<br> > <br> > In the uboot source directory :<br> > <br> > <br> > $ sudo ./scripts/sdcard.sh /dev/sdX<br> > <br> > <br> > This will erase all data and create 4 partitions in the medium, along with copying<br> > the u-boot binary to the first partition:<br> > <br> > <br> > Partition 1 = ChromeOS signed binary (V.O.S chained u-boot)<br> > Partition 2 = not used<br> > Partition 3 = EXT2 partition for u-boot files (uImage and exynos5250-snow.dtb)<br> > Partition 4 = EXT4 partition for userspace files<br> > <br> > <br> > With u-boot being copied, next is the kernel image and DTB file. From the kernel<br> > source execute :<br> > <br> > <br> > $ mkdir ../mnt/<br> > $ sudo mount /dev/sdX3 ../mnt/<br> > $ sudo cp arch/arm/boot/uImage ../mnt/<br> > $ sudo cp arch/arm/boot/dts/exynos5250-snow.dtb ../mnt/<br> > $ sudo umount /dev/sdX3<br> > <br> > <br> > Finally, we have to copy the Ubuntu userspace filesystem that we created earlier:<br> > <br> > <br> > $ sudo mount /dev/sdX4 mnt/$ sudo cp -a ./precise/* mnt/$ sudo umount /dev/sdX4<br> > <br> > <br> > <br> > Now,my idea is to chainload the already chain loaded u-boot created by V.O.S to the new u-boot<br> > that we need for booting FreeBSD and that can be installed in the partition n.2,as shown in this<br> > scheme,because it is not used :<br> > <br> > <br> > Partition 1 = ChromeOS signed binary (V.O.S chained u-boot)<br> > Partition 2 = not used (maybe we can install the u-boot for arm 32 bit,compatible with FreeBSD on<br> > this partition)<br> > Partition 3 = EXT2 partition for u-boot files (uImage and exynos5250-snow.dtb)<br> > Partition 4 = EXT4 partition for userspace files<br> > <br> > <br> > Take in consideration that default boot string is hardcoded here,in the snow.h file of the custom<br> > u-boot created by VOS :<br> > <br> > <br> > <a href="https://github.com/virtualopensyste...18a39b6c177dff58a/include/configs/snow.h#L101" rel="noreferrer" target="_blank">https://github.com/virtualopensyste...18a39b6c177dff58a/include/configs/snow.h#L101</a><br>; > <br> > <br> > and it needs to be recompiled because it should point to the partition n.2,where I will install<br> > the u-boot files as explained here :<br> > <br> > <br> > <a href="https://wiki.freebsd.org/arm/Chromebook" rel="noreferrer" target="_blank">https://wiki.freebsd.org/arm/Chromebook</a><br>; > <br> > <br> > I have some questions to ask before I start working on this.<br> > <br> > 1) The xen developer said :<br> > <br> > <br> > You should be able to build U-Boot and use the U-Boot binary as Xen guest kernel...<br> > <br> > <br> > <br> > where is the u-boot binary,according to this document ?<br> > <br> > <a href="https://wiki.freebsd.org/arm/Chromebook" rel="noreferrer" target="_blank">https://wiki.freebsd.org/arm/Chromebook</a><br>; > <br> > I don't see it.<br> > <br> > <br> > 2) where is the source code of the file that I can get here :<br> > <br> > <a href="http://commondatastorage.googleapis.com/chromeos-localmirror/distfiles/nv_uboot-snow-simplefb.kpart.bz2" rel="noreferrer" target="_blank">http://commondatastorage.googleapis.com/chromeos-localmirror/distfiles/nv_uboot-snow-simplefb.kpart.bz2</a><br>; > <br> > I need the source code if I want to recompile u-boot so that it can point to the partition 4.<br> > <br> > Maybe it can be found on this link :<br> > <br> > <a href="http://linux-exynos.org/dist/chromebook/nv_uboot/" rel="noreferrer" target="_blank">http://linux-exynos.org/dist/chromebook/nv_uboot/</a><br>; > <br> > but it can't be opened....<br> > <br> > <br> > 3) in this specific scenario the source code of u-boot should run on arm 32 bit,not on arm<br> > 64,because I have the Samsung Chromebook "SNOW" model XE303C12,that's powered by a Samsung Exynos<br> > 5250 (ARMv7 32 bit Cortex A15) Soc.<br> > <br> > <br> > 4) I'm not sure if I can chainload the customized u-boot created by V.O.S that should be<br> > installed on the first partition with the u-boot tailored for booting FreeBSD that should be<br> > installed on the partition 2....<br> > <br> > <br> > 5) the xen developer said that u-boot should be compiled enabling this option :<br> > <br> > <br> > Code:<br> > <br> > CONFIG_CMO_BY_VA_ONLY=y<br> > <br> > <br> > Well,can you provide some good source that can help me to understand how I can recompile u-boot<br> > for FreeBSD ? thanks.<br> > <br> > --<br> > Mario.<br> > <br> > <br> > <br> > --<br> > Mario.<br> > <br> > <br> > <br> > --<br> > Mario.<br> > <br> > <br> > <br> > --<br> > Mario.<br> > <br> > <br> > <br> > --<br> > Mario.<br> > <br> > <br> > <br> > --<br> > Mario.<br> > <br> > </blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Mario.<br></div> </blockquote></div> </div> </blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Mario.<br></div>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSigREr=aSHOBMbEK9CaoKzNnKC-6sE=Ui8ovVwnMaZgZhQ>