From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 7 07:24:57 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A4DF2376 for ; Thu, 7 Feb 2013 07:24:57 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ia0-x22d.google.com (ia-in-x022d.1e100.net [IPv6:2607:f8b0:4001:c02::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 6C361695 for ; Thu, 7 Feb 2013 07:24:57 +0000 (UTC) Received: by mail-ia0-f173.google.com with SMTP id h37so2630657iak.32 for ; Wed, 06 Feb 2013 23:24:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=xZ2xjIf4tgrpV/jFv1GaMMn8SbRKnr9Slmk4alyHpbg=; b=pT3EhYKBohMekMrQOeiqaTC1a8O038cJ6XuHRZjYRrEayCE/f2nmVaa+GXSqDKJ/Ws Qp3SDjpvLSNQSGRrp3rQce6q1DaG9jMlEvqKC1Imd877EdfwRFa4rrDUzzgonIWMcB/6 p+GwNRHrI75CYo6KyTB9FXJhRQNMtVYxqFjKKZEAtGjFwG2SdLMVRGncMB5H1g04FPfZ mWZkm+Rj7uL8FNKRMxqs/eJjEMBk6OIxXElyrY6d2PjV3aVwUsqP8VlKPDqjelEnlJED W1aAS6Eg7VRKHwKJuP6k0EXt/nlVT8/tKPUrBxoKgjjgxLkCsrOvAZgpFfP0wixsMRf4 +2vA== MIME-Version: 1.0 X-Received: by 10.50.170.69 with SMTP id ak5mr583438igc.56.1360221896792; Wed, 06 Feb 2013 23:24:56 -0800 (PST) Received: by 10.42.23.132 with HTTP; Wed, 6 Feb 2013 23:24:56 -0800 (PST) In-Reply-To: <51132599.60506@gmail.com> References: <51132599.60506@gmail.com> Date: Wed, 6 Feb 2013 23:24:56 -0800 Message-ID: Subject: Re: bhyve hangs on going multi user From: Neel Natu To: "Sevan / Venture37" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 07:24:57 -0000 Hi Sevan, At the risk of stating the obvious: > pid 17 (stty), uid 0: exited on signal 4 > Illegal instruction > pid 19 (ps), uid 0: exited on signal 4 > Illegal instruction The userland processes launched by /etc/rc are getting a SIGILL but it is not clear why. Could you do some experiments on your end to narrow this down a bit more: - Boot to single user and then execute a variety of commands to figure out whether the problem happens randomly or does is follow a particular executable (for e.g. execute "ps", "stty", "fsck", "find", "ls" etc). - Did this problem start happening only recently? - If you have a different host system does it happen there too? If you have a different virtual machine does it happen there as well? best Neel On Wed, Feb 6, 2013 at 7:55 PM, Sevan / Venture37 wrote: > Hi, > Attempting to start a VM with the following command results in a hang when > the system tries to go multiuser > sudo bhyveload -m 768 -M 8192 -d ./diskdev guest0 && sudo bhyve -c 1 -a -A > -m 768 -M 8192 -I -H -g 0 -s 0:0,hostbridge -s 1:0,virtio-blk,diskdev -S > 31,uart,stdio gues0 > > > /boot/loader.conf > autoboot_delay=5 > kernel="kernel" > > virtio_load="YES" > if_vtnet_load="YES" > virtio_pci_load="YES" > virtio_blk_load="YES" > > # Avoid clock drift > kern.timecounter.hardware="TSC" > kern.timecounter.invariant_tsc="1" > # Tell the guest it is virtualized > #smbios.bios.vendor="Bochs" > smbios.bios.vendor="BHYVE" > > console="userboot" > bootverbose="1" > > > /etc/fstab > /dev/vtbd0s1a / ufs rw 1 1 > > > > > Booting... > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > SMAP type=01 base=0000000000000000 len=00000000000a0000 > SMAP type=01 base=0000000000100000 len=000000002ff00000 > SMAP type=01 base=0000000100000000 len=0000000200000000 > Table 'APIC' at 0xf0500 > APIC: Found table at 0xf0500APIC: Using the MADT enumerator. > MADT: Found CPU APIC ID 0 ACPI ID 0: enabled > SMP: Added CPU 0 (AP) > Copyright (c) 1992-2013 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #0 r246446: Thu Feb 7 02:36:17 UTC 2013 > you@dev.foo.local:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 > WARNING: WITNESS option enabled, expect reduced performance. > Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff81822000. > Hypervisor: Origin = "BHyVBHyVBHyV" > Calibrating TSC clock ... TSC clock: 3399738220 Hz > CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3399.74-MHz K8-class CPU) > Origin = "GenuineIntel" Id = 0x306a9 Family = 0x6 Model = 0x3a Stepping > = 9 > > Features=0x8fa3ab7f > > Features2=0xe3bae257 > AMD Features=0x28100800 > AMD Features2=0x1 > TSC: P-state invariant > real memory = 12884901888 (12288 MB) > Physical memory chunk(s): > 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) > 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) > 0x000000000184f000 - 0x000000002fffffff, 779816960 bytes (190385 pages) > 0x0000000100000000 - 0x00000002e96e7fff, 8211300352 bytes (2004712 pages) > avail memory = 8932638720 (8518 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: > x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 > x86bios: SSEG 0x001000-0x001fff at 0xffffff8000223000 > x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 > APIC: CPU 0 has ACPI ID 0 > random device not loaded; using insecure entropy > ULE: setup cpu 0 > ACPI: RSDP 0xf0400 00024 (v02 BHYVE ) > ACPI: XSDT 0xf0480 00034 (v01 BHYVE BVXSDT 00000001 INTL 20130117) > ACPI: APIC 0xf0500 0004A (v01 BHYVE BVMADT 00000001 INTL 20130117) > ACPI: FACP 0xf0600 0010C (v05 BHYVE BVFACP 00000001 INTL 20130117) > ACPI: DSDT 0xf0800 000F2 (v02 BHYVE BVDSDT 00000001 INTL 20130117) > ACPI: FACS 0xf0780 00040 > MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000 > ioapic0: Changing APIC ID to 1 > ioapic0: Routing external 8259A's -> intpin 0 > MADT: Interrupt override: source 9, irq 9 > ioapic0: intpin 9 trigger: level > ioapic0: intpin 9 polarity: low > ioapic0 irqs 0-16 on motherboard > cpu0 BSP: > ID: 0x00000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff > lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff > timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 > snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] > feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 > feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 > wlan: <802.11 Link Layer> > nfslock: pseudo-device > null: > random: > module_register_init: MOD_LOAD (vesa, 0xffffffff80c24d10, 0) error 19 > io: > kbd0 at kbdmux0 > ctl: CAM Target Layer loaded > acpi0: on motherboard > ACPI: All ACPI Tables successfully acquired > ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 > atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 > atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment > 0.500000000s) > ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 > Event timer "RTC" frequency 32768 Hz quality 0 > ACPI timer: 1/5 1/4 1/5 1/4 1/3 1/4 1/5 1/4 1/5 1/5 -> 10 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pcib0: decoding 4 range 0-0xcf7 > pcib0: decoding 4 range 0xd00-0xffff > pcib0: could not get PCI interrupt routing table for \_SB_.PCI0 - > AE_NOT_FOUND > pci0: on pcib0 > pci0: domain=0, physical bus=0 > found-> vendor=0x1275, dev=0x1275, revid=0x00 > domain=0, bus=0, slot=0, func=0 > class=06-00-00, hdrtype=0x01, mfdev=0 > cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > found-> vendor=0x1af4, dev=0x1001, revid=0x00 > domain=0, bus=0, slot=1, func=0 > class=01-00-00, hdrtype=0x00, mfdev=0 > cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > MSI-X supports 2 messages in map 0x14 > map[10]: type I/O Port, range 32, base 0x2000, size 6, enabled > pcib0: allocated type 4 (0x2000-0x203f) for rid 10 of pci0:0:1:0 > map[14]: type Memory, range 32, base 0x30000000, size 13, enabled > found-> vendor=0x131f, dev=0x2000, revid=0x00 > domain=0, bus=0, slot=31, func=0 > class=07-00-00, hdrtype=0x00, mfdev=0 > cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > intpin=a, irq=4 > map[10]: type I/O Port, range 32, base 0x3f8, size 3, enabled > pcib0: allocated type 4 (0x3f8-0x3ff) for rid 10 of pci0:0:31:0 > pcib0: no PRT entry for 0.31.INTA > virtio_pci0: port 0x2000-0x203f mem > 0x30000000-0x30001fff at device 1.0 on pci0 > vtblk0: on virtio_pci0 > virtio_pci0: host features: 0x10000004 > virtio_pci0: negotiated features: 0x10000004 > virtio_pci0: attempting to allocate 2 MSI-X vectors (2 supported) > msi: routing MSI-X IRQ 256 to local APIC 0 vector 50 > msi: routing MSI-X IRQ 257 to local APIC 0 vector 51 > virtio_pci0: using IRQs 256-257 for MSI-X > virtio_pci0: using per VQ MSIX interrupts > vtblk0: 20480MB (41943040 512 byte sectors) > uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0 > ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 52 > uart2: fast interrupt > uart2: console (9600,n,8,1) > acpi0: wakeup code va 0xffffff8000299000 pa 0x4000 > Device configuration finished. > procfs registered > lapic: Divisor 2, Frequency 1700037000 Hz > Timecounters tick every 10.000 msec > vlan: initialized, using hash tables with chaining > tcp_init: net.inet.tcp.tcbhashsize auto tuned to 131072 > lo0: bpf attached > hptrr: no controller detected. > hpt27xx: no controller detected. > GEOM: new disk vtbd0 > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > TSC timecounter discards lower 1 bit(s) > Timecounter "TSC-low" frequency 1699869110 Hz quality 1000 > WARNING: WITNESS option enabled, expect reduced performance. > Trying to mount root from ufs:/dev/vtbd0s1a [rw]... > start_init: trying /sbin/init > pid 17 (stty), uid 0: exited on signal 4 > Illegal instruction > pid 19 (ps), uid 0: exited on signal 4 > Illegal instruction > Setting hostuuid: 524a76a9-70d9-11e2-9241-9f81272ed682. > Setting hostid: 0x7c395229. > No suitable dump device was found. > Entropy harvesting:. > Starting file system checks: > /dev/vtbd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS > pid 50 (fsck_ufs), uid 0: exited on signal 4 > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT > fsck: /dev/vtbd0s1a: Illegal instruction > Unknown error; help! > ERROR: ABORTING BOOT (sending SIGTERM to parent)! > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org"