From nobody Wed Jan 10 23:25:21 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T9P7F4xHQz56SNr for ; Wed, 10 Jan 2024 23:26:01 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T9P7D5295z45FB for ; Wed, 10 Jan 2024 23:26:00 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=G15LrjOy; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::631 as permitted sender) smtp.mailfrom=marietto2008@gmail.com Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-a2c179aa5c4so84190966b.0 for ; Wed, 10 Jan 2024 15:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704929158; x=1705533958; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=B/fQfOTSnAufInRRWatE4A3GVNi3arCudtxIfRWCYuI=; b=G15LrjOyJcEML0itBy7O2GCg+5xIFvQsYtjhHSq4SWYH2O+dDmrnBADtEl6Y2x4eeR ca4/UxHM6qE1j+nIsshjDZ9IJgbk55FS+HyVPP3MYGI33JRZheHkF37mfsiMBu210OTI HnatP0PcJ8snht8xBLS6qv54CGWOyAqQKE1Bb7iTjUeRZypwGOrsdET1KvwoTCEdMcED AU8U1X6IhiTGrESgqORXaHRT/J+njqfisB9mDciTVFf+4eQQ5POt7Dg5vKRzPBV7VSNW 1iNa+i/uVtIDo9rEOMkeXNShpOeF47B5KKgMoYmLBr5FIZfRqEjH19uP/fN2kCkUkOdl nXcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704929158; x=1705533958; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=B/fQfOTSnAufInRRWatE4A3GVNi3arCudtxIfRWCYuI=; b=I3FuIIOBzsf9ZJBkpqxQOk/SSYdC6fn5pFvz6Rt/cpaHfjJPgGV1F/69qKUpNkYQvP NZ4PlNDo8fXh5cNdheWKOtq8N+VyJX33Qr/i6xF1QCGbXNUoaplmDh+iR4uMoyWfNTot 6KMQFPyd1IvfpWf6O85Il3P8/HQe1EfiDSvZgpjQr7s9Sg0BthynZ/KyLd7MXmH9jd6p V/E96T4U1f1HgFy42QRZIY23PBROR1XiV7784PDQrHo5UEbY2Kt0r4kDO5xZnFCpyf6C L342uAKQlsY5U2nSVeHv94Mu40guXC8dMFlomglP1jVqATJJ+jPRLe5uWPIRjZSwrANB YjrQ== X-Gm-Message-State: AOJu0YzzG3w3NchaNyyjNeEAqwjygxOlZqrN74x6keZZv2JQiamP4q9C g2LoOGLAPM2ZtPfmlomKqilRD/NfLhcEsdyakIviH/vsVIY= X-Google-Smtp-Source: AGHT+IFeFfxA6sGN9P4X3fXH9+kzWMBzsfjlp5C1qdneFwbUKqo9PnXVW5OhCxQ6AR6+bvR6Ruv1fB01qw4PtDP71qg= X-Received: by 2002:a17:906:16d6:b0:a2b:63ca:cee7 with SMTP id t22-20020a17090616d600b00a2b63cacee7mr105385ejd.130.1704929158235; Wed, 10 Jan 2024 15:25:58 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 From: Mario Marietto Date: Thu, 11 Jan 2024 00:25:21 +0100 Message-ID: Subject: unknown option "XENHVM" for Arm 32 ; missing code detected. To: freebsd-arm Content-Type: multipart/alternative; boundary="000000000000d80951060e9fc0bd" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.83 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; NEURAL_HAM_LONG(-0.84)[-0.836]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::631:from] X-Rspamd-Queue-Id: 4T9P7D5295z45FB --000000000000d80951060e9fc0bd Content-Type: text/plain; charset="UTF-8" Hello. I'm trying to boot FreeBSD for arm32 bit as DomU on my ARM Chromebook. Basically,with the help of the xen developers I have patched the FreeBSD kernel and I've been able to boot it as if it was a zImage format file. I've enabled all the xen options inside the Linux kernel file used for the domU and this is what happened when I tried to boot the FreeBSD kernel file : mario@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/freebsd-xen/domU-freebsd# ./start-freebsd Parsing config from freebsd.cfg libxl: debug: libxl_create.c:2081:do_domain_create: ao 0x442780: create: how=(nil) callback=(nil) poller=0x43cc50 libxl: detail: libxl_create.c:662:libxl__domain_make: passthrough: disabled libxl: debug: libxl_arm.c:148:libxl__arch_domain_prepare_config: Configure the domain libxl: debug: libxl_arm.c:151:libxl__arch_domain_prepare_config: - Allocate 0 SPIs libxl: debug: libxl_device.c:415:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=unknown specification=xen libxl: debug: libxl_device.c:452:libxl__device_disk_set_backend: Disk vdev=xvda, using backend phy libxl: debug: libxl_create.c:1342:initiate_domain_create: Domain 1:running bootloader libxl: debug: libxl_bootloader.c:417:libxl__bootloader_run: Domain 1:no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=0x43d8f0: deregister unregistered domainbuilder: detail: xc_dom_allocate: cmdline="console=hvc0", features="" domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x40008+0x966 at 0xb5972000 domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x40008000 -> 0x4096e000 (pfn 0x40008 + 0x966 pages) domainbuilder: detail: xc_dom_load_zimage_kernel: called domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 0x40008000-0x4096e000 domainbuilder: detail: xc_dom_load_zimage_kernel: copy 9851212 bytes from blob 0xb62d8000 to dst 0xb5972000 domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x48000+0x1 at 0xb6f0a000 domainbuilder: detail: xc_dom_alloc_segment: devicetree : 0x48000000 -> 0x48001000 (pfn 0x48000 + 0x1 pages) domainbuilder: detail: alloc_magic_pages: called domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x48001000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x0 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: bootearly: doing nothing domainbuilder: detail: start_info_arm: called domainbuilder: detail: domain builder memory footprint domainbuilder: detail: allocated domainbuilder: detail: malloc : 58 kB domainbuilder: detail: anon mmap : 0 bytes domainbuilder: detail: mapped domainbuilder: detail: file mmap : 9620 kB domainbuilder: detail: domU mmap : 9628 kB domainbuilder: detail: vcpu_arm32: called domainbuilder: detail: Initial state CPSR 0x400001d3 PC 0x40008000 domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[0] -> d0 0x39000 domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[1] -> d0 0x39001 domainbuilder: detail: xc_dom_release: called libxl: debug: libxl_device.c:415:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=phy specification=xen libxl: debug: libxl_event.c:812:libxl__ev_xswatch_register: watch w=0x4410e4 wpath=/local/domain/0/backend/vbd/1/51712/state token=3/0: register slotnum=3 libxl: debug: libxl_create.c:2120:do_domain_create: ao 0x442780: inprogress: poller=0x43cc50, flags=i libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x4410e4 wpath=/local/domain/0/backend/vbd/1/51712/state token=3/0: event epath=/local/domain/0/backend/vbd/1/51712/state libxl: debug: libxl_event.c:1054:devstate_callback: backend /local/domain/0/backend/vbd/1/51712/state wanted state 2 still waiting state 1 libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x4410e4 wpath=/local/domain/0/backend/vbd/1/51712/state token=3/0: event epath=/local/domain/0/backend/vbd/1/51712/state libxl: debug: libxl_event.c:1051:devstate_callback: backend /local/domain/0/backend/vbd/1/51712/state wanted state 2 ok libxl: debug: libxl_event.c:849:libxl__ev_xswatch_deregister: watch w=0x4410e4 wpath=/local/domain/0/backend/vbd/1/51712/state token=3/0: deregister slotnum=3 libxl: debug: libxl_device.c:1150:device_backend_callback: Domain 1:calling device_backend_cleanup libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=0x4410e4: deregister unregistered libxl: debug: libxl_linux.c:194:libxl__hotplug_disk: Domain 1:Args and environment ready libxl: debug: libxl_device.c:1251:device_hotplug: Domain 1:calling hotplug script: /etc/xen/scripts/block add libxl: debug: libxl_device.c:1252:device_hotplug: Domain 1:extra args: libxl: debug: libxl_device.c:1260:device_hotplug: Domain 1:env: libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: script: /etc/xen/scripts/block libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_TYPE: vbd libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_PATH: backend/vbd/1/51712 libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_BASE_PATH: backend libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execute: /etc/xen/scripts/block add libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=0x441168: deregister unregistered libxl: debug: libxl_linux.c:211:libxl__get_hotplug_script_info: Domain 1:num_exec 1, not running hotplug scripts libxl: debug: libxl_device.c:1236:device_hotplug: Domain 1:No hotplug script to execute libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=0x441168: deregister unregistered libxl: debug: libxl_event.c:812:libxl__ev_xswatch_register: watch w=0x43e7cc wpath=/local/domain/1/console/tty token=3/1: register slotnum=3 libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x43e7cc wpath=/local/domain/1/console/tty token=3/1: event epath=/local/domain/1/console/tty libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x43e7cc wpath=/local/domain/1/console/tty token=3/1: event epath=/local/domain/1/console/tty libxl: debug: libxl_event.c:2403:libxl__ao_progress_report: ao 0x442780: progress report: ignored libxl: debug: libxl_event.c:849:libxl__ev_xswatch_deregister: watch w=0x43e7cc wpath=/local/domain/1/console/tty token=3/1: deregister slotnum=3 libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=0x43e7cc: deregister unregistered libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x442780: complete, rc=0 libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x442780: destroy libxl: debug: libxl_domain.c:704:libxl_domain_unpause: Domain 1:ao 0x442780: create: how=(nil) callback=(nil) poller=0x43cc50 libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x442780: complete, rc=0 libxl: debug: libxl_domain.c:712:libxl_domain_unpause: Domain 1:ao 0x442780: inprogress: poller=0x43cc50, flags=ic libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x442780: destroy xencall:buffer: debug: total allocations:98 total releases:98 xencall:buffer: debug: current allocations:0 maximum allocations:3 xencall:buffer: debug: cache current size:3 xencall:buffer: debug: cache hits:86 misses:3 toobig:9 xencall:buffer: debug: total allocations:0 total releases:0 xencall:buffer: debug: current allocations:0 maximum allocations:0 xencall:buffer: debug: cache current size:0 xencall:buffer: debug: cache hits:0 misses:0 toobig:0 FROZEN. I see these two processes enabled...but I don't know how to use FreeBSD... # ps ax 2606 ? Ssl 0:00 xl -vvvv create freebsd.cfg 2607 pts/0 Sl+ 0:00 /usr/lib/xen-4.17/bin/xenconsole 1 --num 0 --type pv now the fun begins...because,based on what say the xen developers : ... the ps output, it seems that ``xl create`` completed and you have the console open. So the freeze you mention is just because your FreeBSD guest is not outputting anything. As mentioned earlier, I don't think a lot of testing has been done for 32-bit Arm FreeBSD. So it is quite possible that there are some pieces of code missing. The first step is to check if the FreeBSD kernel was built with Xen options. If they are, then you will need to find out where FreeBSD is stuck (or why the console is not enabled). Unfortunately, we don't have any support to use GDB on the guest kernel. So you will have to modify FreeBSD a bit to check where it freezes. Assuming you have a debug build of the hypervisor, then you can sprinkle the FreeBSD boot code with the assembly instruction 'hvc 0xfffc'. When this is reached, this will issue an hypercall that will print on the Xen console that the given instruction has reached (the PC will be printed). There are other useful 'hvc' calls implemented by Xen for low level debugging. You can look at do_debug_trap() in Xen code. I've added this to GENERIC kernel of FreeBSD guest : # Xen HVM Guest Optimizations # NOTE: XENHVM depends on xenpci. They must be added or removed together. options XENHVM # Xen HVM kernel infrastructure device xenpci # Xen HVM Hypervisor services driver Unfortunately,it is not accepted : unknown option "XENHVM" I've found these parameters ONLY on the file /usr/src/sys/conf/files.amd64 : x86/xen/pv.c optional xenhvm x86/xen/pvcpu_enum.c optional xenhvm x86/xen/xen_pci_bus.c optional xenhvm Look at the prefix x86. It is not for arm. It seems there is a piece of code that hasn't been written. Is there a chance to add it ? Does anyone want to help ? No arch/arm64 does not contain Xen in KERNCONF either. Surprisingly files.x86 invokes a different set of files. I don't know what build it is. x86/xen/hvm.c optional xenhvm x86/xen/xen_intr.c optional xenhvm x86/xen/xen_apic.c optional xenhvm x86/xen/xenpv.c optional xenhvm x86/xen/xen_msi.c optional xenhvm x86/xen/xen_nexus.c optional xenhvm -- Mario. --000000000000d80951060e9fc0bd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello.

I'm trying to boot FreeBSD for arm32 bit as DomU on my ARM Chromebook. = Basically,with the help of the xen developers I have patched the FreeBSD ke= rnel and I've been able to boot it as if it was a zImage format file. <= br>

I've enabled all the xen options inside th= e Linux kernel file used for the domU and this is what happened when I trie= d to boot the FreeBSD kernel file :


mario@devuan-bunsen:/mn=
t/zroot2/zroot2/OS/Chromebook/freebsd-xen/domU-freebsd# ./start-freebsd

Parsing config from freebsd.cfg

libxl: debug: libxl_create.c:2081:do_domain_create: ao 0x442780: create: ho=
w=3D(nil) callback=3D(nil) poller=3D0x43cc50

libxl: detail: libxl_create.c:662:libxl__domain_make: passthrough: disabled
libxl: debug: libxl_arm.c:148:libxl__arch_domain_prepare_config: Configure =
the domain
libxl: debug: libxl_arm.c:151:libxl__arch_domain_prepare_config: - Allocate=
 0 SPIs

libxl: debug: libxl_device.c:415:libxl__device_disk_set_backend: Disk vdev=
=3Dxvda spec.backend=3Dunknown specification=3Dxen

libxl: debug: libxl_device.c:452:libxl__device_disk_set_backend: Disk vdev=
=3Dxvda, using backend phy
libxl: debug: libxl_create.c:1342:initiate_domain_create: Domain 1:running =
bootloader
libxl: debug: libxl_bootloader.c:417:libxl__bootloader_run: Domain 1:no boo=
tloader configured, using user supplied kernel

libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=3D0x4=
3d8f0: deregister unregistered domainbuilder: detail: xc_dom_allocate: cmdl=
ine=3D"console=3Dhvc0", features=3D""

domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4000=
8+0x966 at 0xb5972000
domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0x40008000 -&=
gt; 0x4096e000  (pfn 0x40008 + 0x966 pages)
domainbuilder: detail: xc_dom_load_zimage_kernel: called
domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 0x40008000-0x4=
096e000
domainbuilder: detail: xc_dom_load_zimage_kernel: copy 9851212 bytes from b=
lob 0xb62d8000 to dst 0xb5972000
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4800=
0+0x1 at 0xb6f0a000
domainbuilder: detail: xc_dom_alloc_segment:   devicetree   : 0x48000000 -&=
gt; 0x48001000  (pfn 0x48000 + 0x1 pages)
domainbuilder: detail: alloc_magic_pages: called
domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0x48001000
domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0x0
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: bootearly: doing nothing
domainbuilder: detail: start_info_arm: called
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail:    allocated
domainbuilder: detail:       malloc             : 58 kB
domainbuilder: detail:       anon mmap          : 0 bytes
domainbuilder: detail:    mapped
domainbuilder: detail:       file mmap          : 9620 kB
domainbuilder: detail:       domU mmap          : 9628 kB
domainbuilder: detail: vcpu_arm32: called
domainbuilder: detail: Initial state CPSR 0x400001d3 PC 0x40008000
domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[0] -> d0 0x39000
domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[1] -> d0 0x39001
domainbuilder: detail: xc_dom_release: called

libxl: debug: libxl_device.c:415:libxl__device_disk_set_backend: Disk vdev=
=3Dxvda spec.backend=3Dphy specification=3Dxen

libxl: debug: libxl_event.c:812:libxl__ev_xswatch_register: watch w=3D0x441=
0e4 wpath=3D/local/domain/0/backend/vbd/1/51712/state token=3D3/0: register=
 slotnum=3D3

libxl: debug: libxl_create.c:2120:do_domain_create: ao 0x442780: inprogress=
: poller=3D0x43cc50, flags=3Di

libxl: debug: libxl_event.c:750:watchfd_callback: watch w=3D0x4410e4 wpath=
=3D/local/domain/0/backend/vbd/1/51712/state token=3D3/0: event epath=3D/lo=
cal/domain/0/backend/vbd/1/51712/state

libxl: debug: libxl_event.c:1054:devstate_callback: backend /local/domain/0=
/backend/vbd/1/51712/state wanted state 2 still waiting state 1

libxl: debug: libxl_event.c:750:watchfd_callback: watch w=3D0x4410e4 wpath=
=3D/local/domain/0/backend/vbd/1/51712/state token=3D3/0: event epath=3D/lo=
cal/domain/0/backend/vbd/1/51712/state

libxl: debug: libxl_event.c:1051:devstate_callback: backend /local/domain/0=
/backend/vbd/1/51712/state wanted state 2 ok

libxl: debug: libxl_event.c:849:libxl__ev_xswatch_deregister: watch w=3D0x4=
410e4 wpath=3D/local/domain/0/backend/vbd/1/51712/state token=3D3/0: deregi=
ster slotnum=3D3

libxl: debug: libxl_device.c:1150:device_backend_callback: Domain 1:calling=
 device_backend_cleanup
libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=3D0x4=
410e4: deregister unregistered
libxl: debug: libxl_linux.c:194:libxl__hotplug_disk: Domain 1:Args and envi=
ronment ready
libxl: debug: libxl_device.c:1251:device_hotplug: Domain 1:calling hotplug =
script: /etc/xen/scripts/block add
libxl: debug: libxl_device.c:1252:device_hotplug: Domain 1:extra args:
libxl: debug: libxl_device.c:1260:device_hotplug: Domain 1:env:
libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: script: /etc/xe=
n/scripts/block
libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_TYPE: vb=
d
libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_PATH: ba=
ckend/vbd/1/51712
libxl: debug: libxl_device.c:1267:device_hotplug: Domain 1: XENBUS_BASE_PAT=
H: backend
libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execu=
te: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=3D0x4=
41168: deregister unregistered
libxl: debug: libxl_linux.c:211:libxl__get_hotplug_script_info: Domain 1:nu=
m_exec 1, not running hotplug scripts
libxl: debug: libxl_device.c:1236:device_hotplug: Domain 1:No hotplug scrip=
t to execute
libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=3D0x4=
41168: deregister unregistered

libxl: debug: libxl_event.c:812:libxl__ev_xswatch_register: watch w=3D0x43e=
7cc wpath=3D/local/domain/1/console/tty token=3D3/1: register slotnum=3D3

libxl: debug: libxl_event.c:750:watchfd_callback: watch w=3D0x43e7cc wpath=
=3D/local/domain/1/console/tty token=3D3/1: event epath=3D/local/domain/1/c=
onsole/tty

libxl: debug: libxl_event.c:750:watchfd_callback: watch w=3D0x43e7cc wpath=
=3D/local/domain/1/console/tty token=3D3/1: event epath=3D/local/domain/1/c=
onsole/tty

libxl: debug: libxl_event.c:2403:libxl__ao_progress_report: ao 0x442780: pr=
ogress report: ignored

libxl: debug: libxl_event.c:849:libxl__ev_xswatch_deregister: watch w=3D0x4=
3e7cc wpath=3D/local/domain/1/console/tty token=3D3/1: deregister slotnum=
=3D3

libxl: debug: libxl_event.c:863:libxl__ev_xswatch_deregister: watch w=3D0x4=
3e7cc: deregister unregistered
libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x442780: complete,=
 rc=3D0
libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x442780: destroy

libxl: debug: libxl_domain.c:704:libxl_domain_unpause: Domain 1:ao 0x442780=
: create: how=3D(nil) callback=3D(nil) poller=3D0x43cc50

libxl: debug: libxl_event.c:2067:libxl__ao_complete: ao 0x442780: complete,=
 rc=3D0

libxl: debug: libxl_domain.c:712:libxl_domain_unpause: Domain 1:ao 0x442780=
: inprogress: poller=3D0x43cc50, flags=3Dic

libxl: debug: libxl_event.c:2036:libxl__ao__destroy: ao 0x442780: destroy

xencall:buffer: debug: total allocations:98 total releases:98
xencall:buffer: debug: current allocations:0 maximum allocations:3
xencall:buffer: debug: cache current size:3
xencall:buffer: debug: cache hits:86 misses:3 toobig:9
xencall:buffer: debug: total allocations:0 total releases:0
xencall:buffer: debug: current allocations:0 maximum allocations:0
xencall:buffer: debug: cache current size:0
xencall:buffer: debug: cache hits:0 misses:0 toobig:0

FROZEN.

I see these two processes enabled...but I don't know how to use FreeBSD= ...

=09 =09 
# ps ax
2606 ?        Ssl    0:00 xl -vvvv create freebsd.cfg
2607 pts/0    Sl+    0:00 /usr/lib/xen-4.17/bin/xenconsole 1 --num 0 --type=
 pv

now the fun begins...because,based on what say the xen developers :

=09
=09
... the ps output, it seems that ``xl create`` completed and you have the console open. So the freeze you mention is just because your=20 FreeBSD guest is not outputting anything. As mentioned earlier, I don't= =20 think a lot of testing has been done for 32-bit Arm FreeBSD. So it is=20 quite possible that there are some pieces of code missing. The first=20 step is to check if the FreeBSD kernel was built with Xen=20 options. If they are, then you will need to find out where FreeBSD is=20 stuck (or why the console is not enabled). Unfortunately, we don't have= =20 any support to use GDB on the guest kernel. So you will have to modify=20 FreeBSD a bit to check where it freezes. Assuming you have a debug=20 build of the hypervisor, then you can sprinkle the FreeBSD boot code=20 with the assembly instruction 'hvc 0xfffc'. When this is reached, t= his=20 will issue an hypercall that will print on the Xen console that the given= =20 instruction has reached (the PC will be printed). There are other useful=20 'hvc' calls implemented by Xen for low level debugging. You can loo= k at=20 do_debug_trap() in Xen code.
=09

I've added this to GENERIC kernel of FreeBS= D guest :

=09 =09 
# Xen HVM Guest Optimizations

# NOTE: XENHVM depends on xenpci.  They must be added or removed togeth=
er.

options     XENHVM            # Xen HVM kernel infrastructure
device      xenpci            # Xen HVM Hypervisor services driver

<=
/code>
Unfortunately,it is not accepted :

unknown option "XENHVM"


I've found these parameters ONLY on the file /usr/src/sys/conf/files.amd64 :

<=
/code>

		
x86/xen/pv.c           =
 optional    xenhvm
x86/xen/pvcpu_enum.c    optional    xenhvm
x86/xen/xen_pci_bus.c   optional    xenhvm

Look at the prefix x86. It is not for arm. It seems th= ere is a piece of code that hasn't been written. Is there a chance to a= dd it ? Does anyone want to help ?

No arch/arm64 d= oes not contain Xen in KERNCONF either.

Surprisingly files.x86 invokes a diff= erent set of files. I don't know what build it is.

=09 =09 
x86/xen/hvm.c          =
  optional    xenhvm
x86/xen/xen_intr.c       optional    xenhvm
x86/xen/xen_apic.c       optional    xenhvm
x86/xen/xenpv.c          optional    xenhvm
x86/xen/xen_msi.c        optional    xenhvm
x86/xen/xen_nexus.c      optional    xenhvm

--
=
Mario.
--000000000000d80951060e9fc0bd--