Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 11 Aug 2020 10:23:12 +0900
From:      KIRIYAMA Kazuhiko <kiri@truefc.org>
To:        freebsd-virtualization@freebsd.org
Cc:        kiri@truefc.org
Subject:   Why qemu-system-aarch64 so late to process ?
Message-ID:  <202008110123.07B1NChc032219@kx.truefc.org>

next in thread | raw e-mail | index | archive | help
Hi, all

I'm working on qemu-system-aarch64 and try to build
packages, but it's too late to compile stuffs. I've compare
to bhyve package building procedure, then about 70 times
later than bhyve one. I don't know why so much late on
qemu-system-aarch64.

Following results are devel/binutils package building
process between bhyve and qemu-system-aarch64.

=== devel/binutils package building (bhyve) ===

VM environments:

root@vm:~ # cat /vm/jdtpkxb/jdtpkxb.conf
loader="bhyveload"
cpu=4
memory=8GB
network0_type="virtio-net"
network0_switch="local"
disk0_type="virtio-blk"
disk0_name="disk0.img"
storage=32G
memory=8G
cpu=4
uuid="200bdead-cd7e-11ea-8cb7-0cc47ab3cfd4"
network0_mac="58:9c:fc:02:0f:18"
root@vm:~ # tail -35 /vm/jdtpkxb/vm-bhyve.log
Jul 27 06:24:14:  [primary disk dev: file]
Jul 27 06:24:14: initialising network device tap1
Jul 27 06:24:14: setting mtu of tap1 to 9000
Jul 27 06:24:14: adding tap1 -> vm-local (local addm)
Jul 27 06:24:14: bring up tap1 -> vm-local (local addm)
Jul 27 06:24:14: booting
Jul 27 06:24:14: bhyveload -c /dev/nmdm-jdtpkxb.1A -m 8GB -e autoboot_delay=3 -d /vm/jdtpkxb/disk0.img jdtpkxb
Jul 27 06:24:18:  [bhyve options: -c 4 -m 8GB -AHP -U 200bdead-cd7e-11ea-8cb7-0cc47ab3cfd4 -u]
Jul 27 06:24:18:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/vm/jdtpkxb/disk0.img -s 5:0,virtio-net,tap1,mac=58:9c:fc:02:0f:18]
Jul 27 06:24:18:  [bhyve console: -l com1,/dev/nmdm-jdtpkxb.1A]
Jul 27 06:24:18: starting bhyve (run 1)
Jul 28 21:44:41: bhyve exited with status 1
Jul 28 21:44:41: destroying network device tap1
Jul 28 21:44:42: stopped
Aug 10 09:58:51: initialising
Aug 10 09:58:51:  [loader: bhyveload]
Aug 10 09:58:51:  [cpu: 4]
Aug 10 09:58:51:  [memory: 8GB]
Aug 10 09:58:51:  [hostbridge: standard]
Aug 10 09:58:51:  [com ports: com1]
Aug 10 09:58:51:  [uuid: 200bdead-cd7e-11ea-8cb7-0cc47ab3cfd4]
Aug 10 09:58:51:  [utctime: yes]
Aug 10 09:58:51:  [debug mode: no]
Aug 10 09:58:51:  [primary disk: disk0.img]
Aug 10 09:58:51:  [primary disk dev: file]
Aug 10 09:58:51: initialising network device tap2
Aug 10 09:58:51: setting mtu of tap2 to 9000
Aug 10 09:58:51: adding tap2 -> vm-local (local addm)
Aug 10 09:58:51: bring up tap2 -> vm-local (local addm)
Aug 10 09:58:51: booting
Aug 10 09:58:51: bhyveload -c /dev/nmdm-jdtpkxb.1A -m 8GB -e autoboot_delay=3 -d /vm/jdtpkxb/disk0.img jdtpkxb
Aug 10 09:58:54:  [bhyve options: -c 4 -m 8GB -AHP -U 200bdead-cd7e-11ea-8cb7-0cc47ab3cfd4 -u]
Aug 10 09:58:54:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/vm/jdtpkxb/disk0.img -s 5:0,virtio-net,tap2,mac=58:9c:fc:02:0f:18]
Aug 10 09:58:54:  [bhyve console: -l com1,/dev/nmdm-jdtpkxb.1A]
Aug 10 09:58:54: starting bhyve (run 1)
root@vm:~ # 

Package building process in VM:

root@jdtpkxb:~ # top -bd1
last pid: 54340;  load averages:  2.91,  1.23,  0.61  up 0+05:28:59    15:27:58
37 processes:  5 running, 32 sleeping
CPU:  0.3% user,  0.0% nice,  0.2% system,  0.1% interrupt, 99.4% idle
Mem: 208M Active, 125M Inact, 838M Wired, 692M Buf, 6749M Free
Swap: 3071M Total, 3071M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
54299 root          1  75    0   154M   108M RUN      0   0:01  10.99% cc
54315 root          1  74    0   124M    84M CPU3     3   0:01   8.98% cc
54323 root          1  74    0   157M   113M CPU2     2   0:01   0.00% cc
16066 root          1  20    0    21M  9828K select   0   0:01   0.00% sshd
  624 ntpd          1  20    0    21M  6420K select   2   0:01   0.00% ntpd
54339 root          1  73    0   132M    87M CPU1     1   0:01   0.00% cc
17898 root          1  52    0    19M  7540K wait     2   0:00   0.00% sh
16070 root          1  52    0    19M  7372K wait     1   0:00   0.00% sh
  665 root          1  20    0    18M  7036K select   0   0:00   0.00% sendmail
52975 root          1  52    0    13M  3596K select   0   0:00   0.00% gmake
  672 root          1  20    0    13M  2832K nanslp   0   0:00   0.00% cron
  495 root          1  20    0    13M  2836K select   1   0:00   0.00% syslogd
25104 root          1  52    0    15M  4592K wait     3   0:00   0.00% gmake
25091 root          1  52    0    15M  4332K wait     3   0:00   0.00% gmake
23169 root          1  52    0    13M  3200K wait     2   0:00   0.00% make
  761 root          1  20    0    14M  4448K pause    3   0:00   0.00% tcsh
52886 root          1  52    0    13M  3492K wait     0   0:00   0.00% gmake
  728 root          1  39    0    21M  9440K select   3   0:00   0.00% sshd

root@jdtpkxb:~ # uname -a
FreeBSD jdtpkxb 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r363381M: Thu Jul 23 22:48:19 JST 2020     root@msrvkxb:/usr/obj/usr/src/amd64.amd64/sys/XIJ  amd64
root@jdtpkxb:~ # svnlite info /usr/ports
Path: /usr/ports
Working Copy Root Path: /usr/ports
URL: http://svn.freebsd.org/ports/head
Relative URL: ^/head
Repository Root: http://svn.freebsd.org/ports
Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
Revision: 542063
Node Kind: directory
Schedule: normal
Last Changed Author: glewis
Last Changed Rev: 542063
Last Changed Date: 2020-07-12 11:13:27 +0900 (Sun, 12 Jul 2020)

root@jdtpkxb:~ # 

Package building result:

:===> Building binutils-2.33.1_2,1 start  2020.08.10 15:25:33           
Updating database digests format: .......... done
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 233 packages (of 0 packages in the universe):
  :
(snip)
  :
install  -m 0644 /var/ports/work/var/ports/jdtpkxb/devel/binutils/work-native/binutils-2.33.1/includ
e/plugin-api.h /var/ports/work/var/ports/jdtpkxb/devel/binutils/work-native/stage/usr/local/include/
/bin/rm -f /var/ports/work/var/ports/jdtpkxb/devel/binutils/work-native/stage/usr/local/include/ansi
decl.h
/usr/bin/sed -i.bak '/#include "ansidecl.h"/d' /var/ports/work/var/ports/jdtpkxb/devel/binutils/work
-native/stage/usr/local/include/bfd.h
====> Compressing man pages (compress-man)
===>  Building package for binutils-2.33.1_2,1
:<=== Building binutils-2.33.1_2,1 end    2020.08.10 15:32:44 (00:07:11)


=== devel/binutils package building (qemu-system-aarch64) ===

VM environments:

root@vm:~ # cat /vm/lpbkpkx/lpbkpkx.conf
loader="qemu-system-aarch64"
machine=arm64
machine_arc=aarch64
cpu=4
memory=8GB
network0_type="virtio-net-device"
network0_switch="local"
disk0_type="virtio-blk-device"
disk0_name="disk0.img"
storage=32G
qemu_telnet_port="5044"
qemu_monitor_port="5045"
ssh_host="192.168.1.233"
ssh_port="22"
network0_mac="58:9c:fc:0b:1d:a6"
uuid="7f22b8ab-c0bd-11ea-a00d-0cc47ab3cfd4"
root@vm:~ # tail -35 /vm/lpbkpkx/vm-bhyve.log 
Aug 10 09:58:38: setting mtu of tap1 to 9000
Aug 10 09:58:38: adding tap1 -> vm-local (local addm)
Aug 10 09:58:38: bring up tap1 -> vm-local (local addm)
Aug 10 09:58:38: booting
Aug 10 09:58:38:  [qemu standard options: -machine virt -cpu cortex-a57 -smp 4 -m 8G -name lpbkpkx -uuid 7f22b8ab-c0bd-11ea-a00d-0cc47ab3cfd4]
Aug 10 09:58:38:  [qemu block device options: -device virtio-blk-device,drive=hd0 -drive if=none,format=raw,file=/vm/lpbkpkx/disk0.img,id=hd0]
Aug 10 09:58:38:  [qemu display options: -nographic]
Aug 10 09:58:38:  [qemu network options: -netdev tap,id=net0,ifname=tap1 -device virtio-net-device,netdev=net0,mac=58:9c:fc:0b:1d:a6]
Aug 10 09:58:38:  [qemu debug/expert options: -monitor telnet::5045,server,nowait,nodelay -serial telnet::5044,server,nowait,nodelay -bios /vm/.config/QEMU_EFI.fd]
Aug 10 09:58:38: starting qemu (run 1)
Aug 10 10:04:16: qemu exited with status 0
Aug 10 10:04:16: destroying network device tap1
Aug 10 10:04:16: stopped
Aug 10 10:04:29: initialising
Aug 10 10:04:29:  [uefi: QEMU_EFI.fd]
Aug 10 10:04:29:  [cpu: cortex-a57]
Aug 10 10:04:29:  [smp: 4]
Aug 10 10:04:29:  [memory: 8GB]
Aug 10 10:04:29:  [hostbridge: standard]
Aug 10 10:04:29:  [uuid: 7f22b8ab-c0bd-11ea-a00d-0cc47ab3cfd4]
Aug 10 10:04:29:  [utctime: yes]
Aug 10 10:04:29:  [debug mode: no]
Aug 10 10:04:29:  [primary disk: disk0.img]
Aug 10 10:04:29:  [primary disk dev: file]
Aug 10 10:04:29: initialising network device tap1
Aug 10 10:04:29: setting mtu of tap1 to 9000
Aug 10 10:04:29: adding tap1 -> vm-local (local addm)
Aug 10 10:04:29: bring up tap1 -> vm-local (local addm)
Aug 10 10:04:29: booting
Aug 10 10:04:29:  [qemu standard options: -machine virt -cpu cortex-a57 -smp 4 -m 8G -name lpbkpkx -uuid 7f22b8ab-c0bd-11ea-a00d-0cc47ab3cfd4]
Aug 10 10:04:29:  [qemu block device options: -device virtio-blk-device,drive=hd0 -drive if=none,format=raw,file=/vm/lpbkpkx/disk0.img,id=hd0]
Aug 10 10:04:29:  [qemu display options: -nographic]
Aug 10 10:04:29:  [qemu network options: -netdev tap,id=net0,ifname=tap1 -device virtio-net-device,netdev=net0,mac=58:9c:fc:0b:1d:a6]
Aug 10 10:04:29:  [qemu debug/expert options: -monitor telnet::5045,server,nowait,nodelay -serial telnet::5044,server,nowait,nodelay -bios /vm/.config/QEMU_EFI.fd]
Aug 10 10:04:29: starting qemu (run 1)
root@vm:~ # 

Package building process in VM:

root@lpbkpkx:~ # top -bd1
last pid: 31159;  load averages:  4.48,  4.58,  4.37  up 0+06:07:42    16:12:45
36 processes:  5 running, 31 sleeping
CPU:  4.3% user,  0.0% nice,  7.4% system,  0.9% interrupt, 87.5% idle
Mem: 33M Active, 329M Inact, 406M Wired, 302M Buf, 7203M Free
Swap: 2812M Total, 2812M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
31123 root          1  85    0     0B    16K CPU1     1   0:04  43.46% cc
30320 root          1  80    0    13M  3180K RUN      2   0:17  30.18% sh
31137 root          1  81    0    87M    51M CPU0     0   0:03  29.49% cc
31109 root          1  52    0    85M    49M wait     3   0:02   8.79% cc
25892 root          1  52    0    13M  3760K wait     2   0:39   3.66% sh
  873 admin         1  21    0    20M  9224K select   2   0:01   1.76% sshd
26808 root          1  52    0    13M  3640K wait     2   0:36   1.37% sh
  878 root          1  21    0    14M  4376K pause    0   0:02   1.07% tcsh
 8761 root          1  20    0    20M  9652K select   0   0:22   0.88% sshd
28889 root          1  52    0    13M  3100K select   0   0:05   0.88% gmake
  769 ntpd          1  20    0    21M  5896K select   3   0:34   0.10% ntpd
18670 root          1  52    0    13M  3408K wait     1   0:51   0.00% sh
10643 root          1  52    0    16M  7076K wait     0   0:16   0.00% sh
 8765 root          1  52    0    16M  6928K wait     3   0:11   0.00% sh
  810 root          1  20    0    17M  6440K select   3   0:09   0.00% sendmail
  817 root          1  20    0    13M  2712K nanslp   0   0:03   0.00% cron
  575 root          1  20    0    13M  2712K select   1   0:02   0.00% syslogd
15754 root          1  35    0    15M  4456K select   2   0:02   0.00% gmake

root@lpbkpkx:~ # uname -a
FreeBSD lpbkpkx 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r362853: Thu Jul  2 04:21:45 UTC 2020     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC  arm64
root@lpbkpkx:~ # 

Package building result:

:===> Building binutils-2.33.1_2,1 start  2020.08.10 15:25:20           
Updating database digests format: .......... done
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 16 packages (of 0 packages in the universe):
  :
(snip)
  :
install  -m 0644 /var/ports/work/var/ports/lpbkpkx/devel/binutils/work-native/binutils-2.33.1/include/plugin-api.h /var/ports/work/var/ports/lpbkpkx/devel/binutils/work-native/stage/usr/local/include/
/bin/rm -f /var/ports/work/var/ports/lpbkpkx/devel/binutils/work-native/stage/usr/local/include/ansidecl.h
/usr/bin/sed -i.bak '/#include "ansidecl.h"/d' /var/ports/work/var/ports/lpbkpkx/devel/binutils/work-native/stage/usr/local/include/bfd.h
====> Compressing man pages (compress-man)
===>  Building package for binutils-2.33.1_2,1
:<=== Building binutils-2.33.1_2,1 end    2020.08.10 23:26:11 (08:00:51)


Best Regards

---
Kazuhiko Kiriyama <kiri@truefc.org>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202008110123.07B1NChc032219>