From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 8 11:06:54 2013 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E2B97259 for ; Mon, 8 Apr 2013 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id D01D0370 for ; Mon, 8 Apr 2013 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r38B6sa3057435 for ; Mon, 8 Apr 2013 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r38B6s11057432 for freebsd-virtualization@FreeBSD.org; Mon, 8 Apr 2013 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Apr 2013 11:06:54 GMT Message-Id: <201304081106.r38B6s11057432@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2013 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/170096 virtualization[vimage] Dynamically-attached network interface will c o kern/169991 virtualization[run] [vimage] panic after device plugged in o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF/IPFilter + VIMAGE k a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail a kern/141696 virtualization[rum] [vimage] [panic] rum(4)+ vimage = kernel panic 10 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 10 06:05:45 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B19614BA for ; Wed, 10 Apr 2013 06:05:45 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ia0-x22e.google.com (mail-ia0-x22e.google.com [IPv6:2607:f8b0:4001:c02::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 63DD962E for ; Wed, 10 Apr 2013 06:05:45 +0000 (UTC) Received: by mail-ia0-f174.google.com with SMTP id r13so97220iar.33 for ; Tue, 09 Apr 2013 23:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=7nnQSIdnWY3Qg7JbEhkrTXG7P3RbxTmS3O0rA7VpO2I=; b=cmuJYQo5I3RqyRIaphxU70RWPOZYIG9FBix1BurFQngtNHK1OiCu1GUvcFK/figHZe Hx9LOmgtTlGqdjRGnuYqtwZxnxpmSjEW0bwCNTZxcGCsG0UCd8P40F1idshdr+l2Du07 EdJSHW9QhJpVCZ99u+N8Vdln0p3Yzyx4pslvl/MsnisURCgJO05qWwK3Dfi37AmL6ab5 0P28H/Nf8DQ4UZiASVOIz8O4x48NdyV4JcDQsDPB5LHn0Jn7k8pfSaQaLyESlfsjbu+D P5CbBXtWzdsjbfCvh40vQHcJdUpknYeyXeGiNu+8tLgHtufHalLoraxVPz2Axbr3hayb KdjQ== MIME-Version: 1.0 X-Received: by 10.50.100.167 with SMTP id ez7mr12195053igb.3.1365573944691; Tue, 09 Apr 2013 23:05:44 -0700 (PDT) Received: by 10.43.9.138 with HTTP; Tue, 9 Apr 2013 23:05:44 -0700 (PDT) In-Reply-To: References: <20130328155008.B5464@coco.macktronics.com> <515514E2.1080905@freebsd.org> <20130329140714.L12104@coco.macktronics.com> <20130329143127.W12104@coco.macktronics.com> <20130329172023.L15559@coco.macktronics.com> <20130330125954.G17063@coco.macktronics.com> <20130330141141.B17063@coco.macktronics.com> Date: Tue, 9 Apr 2013 23:05:44 -0700 Message-ID: Subject: Re: bhyve tty / login problems / panic From: Neel Natu To: Dan Mack Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Apr 2013 06:05:45 -0000 Hi, We did some more debugging on Dan's system and root caused the issue to unsynchronized TSCs on the host system. A fix has been committed for this issue: http://svnweb.freebsd.org/base?view=revision&revision=249324 best Neel On Sat, Mar 30, 2013 at 9:03 PM, Neel Natu wrote: > Hi Dan, > > On Sat, Mar 30, 2013 at 12:17 PM, Dan Mack wrote: > >> Here you go: >> >> - uni-processor output: >> >> >> root@olive:~ # bhyvectl --vm=coco --get-stats --cpu=0 >> vcpu0 >> vm exits due to external interrupt 1330 >> number of times hlt was intercepted 676 >> number of times %cr access was intercepted 1 >> number of times rdmsr was intercepted 0 >> number of times wrmsr was intercepted 4 >> number of monitor trap exits 0 >> number of times pause was intercepted 428499 >> vm exits due to interrupt window opening 1231 >> vm exits due to nmi window opening 0 >> number of times in/out was intercepted 423985 >> number of times cpuid was intercepted 22 >> vm exits due to nested page fault 7523 >> number of vm exits for unknown reason 0 >> number of times astpending at exit 38 >> number of vm exits handled in userspace 853290 >> >> number of NMIs delivered to vcpu 0 >> vcpu total runtime 6954314530 >> number of ticks vcpu was idle 59572 >> vcpu migration across host cpus 29 >> >> number of times hlt was ignored 0 >> total number of vm exits 863271 >> >> And here is when running with vcpu==2: >> >> >> root@olive:~ # bhyvectl --vm=coco --get-stats --cpu=0 >> vcpu0 >> vm exits due to external interrupt 19234 >> number of times hlt was intercepted 1965 >> number of times %cr access was intercepted 1 >> number of times rdmsr was intercepted 0 >> number of times wrmsr was intercepted 4 >> number of monitor trap exits 0 >> number of times pause was intercepted 3907709 >> vm exits due to interrupt window opening 8105 >> vm exits due to nmi window opening 0 >> number of times in/out was intercepted 457998 >> number of times cpuid was intercepted 50 >> vm exits due to nested page fault 940909 >> number of vm exits for unknown reason 0 >> number of times astpending at exit 102 >> number of vm exits handled in userspace 4367825 >> >> number of NMIs delivered to vcpu 0 >> vcpu total runtime 51834598990 >> number of ticks vcpu was idle 887 >> vcpu migration across host cpus 25 >> >> number of times hlt was ignored 0 >> total number of vm exits 5335975 >> >> root@olive:~ # bhyvectl --vm=coco --get-stats --cpu=1 >> vcpu1 >> vm exits due to external interrupt 51249 >> number of times hlt was intercepted 3676 >> number of times %cr access was intercepted 2 >> number of times rdmsr was intercepted 0 >> number of times wrmsr was intercepted 4 >> number of monitor trap exits 0 >> number of times pause was intercepted 4667527 >> vm exits due to interrupt window opening 32453 >> vm exits due to nmi window opening 0 >> number of times in/out was intercepted 57278 >> number of times cpuid was intercepted 63 >> vm exits due to nested page fault 266091 >> number of vm exits for unknown reason 0 >> number of times astpending at exit 164 >> number of vm exits handled in userspace 4728481 >> >> number of NMIs delivered to vcpu 0 >> vcpu total runtime 53833466076 >> number of ticks vcpu was idle 965 >> vcpu migration across host cpus 16 >> >> number of times hlt was ignored 0 >> total number of vm exits 5078343 >> >> The panic comes eventually like this: >> >> login: panic: deadlkres: possible deadlock detected for >> 0xfffffe0090fb5490, blocked for 217798 ticks >> >> cpuid = 0 >> Uptime: 255d6h29m56s >> Automatic reboot in 15 seconds - press a key on the console to abort >> >> > Thanks for capturing that. > > In the SMP case the aberrant source of VM exits is the nested page table > faults - which means local apic accesses since these are being entirely > handled in the kernel. They are an order of magnitude more than in the UP > case - even after adjusting for the difference in the run times in the two > cases. > > Also, the uptime message (255d6h29m56s) is indicates something funky > happening with the time keeping. > > I have added one more stat to vmm.ko that counts the number of times a > local apic timer interrupt is generated for each vcpu. > > Also, I would like to install your release.iso if you are able to upload > it. I very much doubt, it but want to rule out those specific bits as the > source of the problem. > > Thanks again for helping to debug this. > > best > Neel > > >> >> Hope this helps, and FYI, here's a dmesg from the host: >> >> >> Copyright (c) 1992-2013 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 10.0-CURRENT #7 r248935: Sat Mar 30 13:19:43 CDT 2013 >> root@olive.example.com:/usr/**obj/usr/src/sys/MACKGEN amd64 >> >> FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 >> CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz (3200.19-MHz K8-class CPU) >> >> Origin = "GenuineIntel" Id = 0x206d7 Family = 0x6 Model = 0x2d >> Stepping = 7 >> >> Features=0xbfebfbff> APIC,SEP,MTRR,PGE,MCA,CMOV,**PAT,PSE36,CLFLUSH,DTS,ACPI,** >> MMX,FXSR,SSE,SSE2,SS,HTT,TM,**PBE> >> >> Features2=0x1fbee3bf> VMX,EST,TM2,SSSE3,CX16,xTPR,**PDCM,PCID,DCA,SSE4.1,SSE4.2,** >> x2APIC,POPCNT,TSCDLT,AESNI,**XSAVE,OSXSAVE,AVX> >> AMD Features=0x2c100800 >> AMD Features2=0x1 >> TSC: P-state invariant, performance statistics >> real memory = 68719476736 (65536 MB) >> avail memory = 63123800064 (60199 MB) >> Event timer "LAPIC" quality 600 >> ACPI APIC Table: >> FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs >> FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 SMT threads >> >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> cpu2 (AP): APIC ID: 2 >> cpu3 (AP): APIC ID: 3 >> cpu4 (AP): APIC ID: 4 >> cpu5 (AP): APIC ID: 5 >> cpu6 (AP): APIC ID: 6 >> cpu7 (AP): APIC ID: 7 >> cpu8 (AP): APIC ID: 8 >> cpu9 (AP): APIC ID: 9 >> cpu10 (AP): APIC ID: 10 >> cpu11 (AP): APIC ID: 11 >> ioapic1 irqs 24-47 on motherboard >> ioapic0 irqs 0-23 on motherboard >> lapic0: Forcing LINT1 to edge trigger >> kbd1 at kbdmux0 >> acpi0: on motherboard >> acpi0: Power Button (fixed) >> acpi0: reservation of f0000000, 18000000 (3) failed >> cpu0: on acpi0 >> cpu1: on acpi0 >> cpu2: on acpi0 >> cpu3: on acpi0 >> cpu4: on acpi0 >> cpu5: on acpi0 >> cpu6: on acpi0 >> cpu7: on acpi0 >> cpu8: on acpi0 >> cpu9: on acpi0 >> cpu10: on acpi0 >> cpu11: on acpi0 >> atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 >> >> Event timer "RTC" frequency 32768 Hz quality 0 >> attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> Event timer "i8254" frequency 1193182 Hz quality 100 >> hpet0: iomem 0xfed00000-0xfed03fff on acpi0 >> Timecounter "HPET" frequency 14318180 Hz quality 950 >> Event timer "HPET" frequency 14318180 Hz quality 350 >> Event timer "HPET1" frequency 14318180 Hz quality 340 >> Event timer "HPET2" frequency 14318180 Hz quality 340 >> Event timer "HPET3" frequency 14318180 Hz quality 340 >> Event timer "HPET4" frequency 14318180 Hz quality 340 >> Event timer "HPET5" frequency 14318180 Hz quality 340 >> Event timer "HPET6" frequency 14318180 Hz quality 340 >> Event timer "HPET7" frequency 14318180 Hz quality 340 >> >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> acpi_button0: on acpi0 >> >> pcib0: port 0xcf8-0xcff on acpi0 >> pci0: on pcib0 >> pcib1: at device 1.0 on pci0 >> pci1: on pcib1 >> pcib2: at device 1.1 on pci0 >> pci2: on pcib2 >> xhci0: mem 0xe3400000-0xe3401fff irq >> 16 at device 0.0 on pci2 >> usbus0: waiting for BIOS to give up control >> xhci0: 32 byte context size. >> usbus0 on xhci0 >> pcib3: at device 2.0 on pci0 >> pci3: on pcib3 >> vgapci0: port 0x3000-0x307f mem >> 0xe2000000-0xe2ffffff,**0xd0000000-0xdfffffff,**0xe0000000-0xe1ffffff >> irq 16 at device 0.0 on pci3 >> pci3: at device 0.1 (no driver attached) >> pcib4: at device 3.0 on pci0 >> pci4: on pcib4 >> pci0: at device 5.0 (no driver attached) >> pci0: at device 5.2 (no driver attached) >> pcib5: at device 17.0 on pci0 >> pci5: on pcib5 >> pci0: at device 22.0 (no driver attached) >> em0: port 0x4040-0x405f mem >> 0xe3500000-0xe351ffff,**0xe3521000-0xe3521fff irq 20 at device 25.0 on >> pci0 >> em0: Using an MSI interrupt >> em0: Ethernet address: 4c:72:b9:d1:b2:d2 >> ehci0: mem 0xe3601000-0xe36013ff irq >> 16 at device 26.0 on pci0 >> usbus1: EHCI version 1.0 >> usbus1 on ehci0 >> pcib6: at device 28.0 on pci0 >> pci6: on pcib6 >> pcib7: at device 28.2 on pci0 >> pci7: on pcib7 >> pci7: at device 0.0 (no driver attached) >> pcib8: at device 28.3 on pci0 >> pci8: on pcib8 >> xhci1: mem 0xe3200000-0xe3201fff irq >> 19 at device 0.0 on pci8 >> usbus2: waiting for BIOS to give up control >> xhci1: 32 byte context size. >> usbus2 on xhci1 >> pcib9: at device 28.5 on pci0 >> pci9: on pcib9 >> em1: port 0x1000-0x101f mem >> 0xe3100000-0xe311ffff,**0xe3120000-0xe3123fff irq 17 at device 0.0 on >> pci9 >> em1: Using MSIX interrupts with 3 vectors >> em1: Ethernet address: 4c:72:b9:d1:b2:d3 >> ehci1: mem 0xe3602000-0xe36023ff irq >> 23 at device 29.0 on pci0 >> usbus3: EHCI version 1.0 >> usbus3 on ehci1 >> pcib10: at device 30.0 on pci0 >> pci10: on pcib10 >> isab0: at device 31.0 on pci0 >> isa0: on isab0 >> ahci0: port >> 0x4068-0x406f,0x4074-0x4077,**0x4060-0x4067,0x4070-0x4073,**0x4020-0x403f >> mem 0xe3523000-0xe35237ff irq 18 at device 31.2 on pci0 >> ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported >> ahcich0: at channel 0 on ahci0 >> ahcich1: at channel 1 on ahci0 >> ahcich2: at channel 2 on ahci0 >> ahcich3: at channel 3 on ahci0 >> ahcich4: at channel 4 on ahci0 >> ahcich5: at channel 5 on ahci0 >> ahciem0: on ahci0 >> pci0: at device 31.3 (no driver attached) >> orm0: at iomem 0xd1000-0xd1fff,0xd2000-**0xd2fff on >> isa0 >> sc0: at flags 0x100 on isa0 >> sc0: VGA <16 virtual consoles, flags=0x300> >> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 >> atkbdc0: at port 0x60,0x64 on isa0 >> atkbd0: irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> ppc0: cannot reserve I/O port range >> est0: on cpu0 >> p4tcc0: on cpu0 >> est1: on cpu1 >> p4tcc1: on cpu1 >> est2: on cpu2 >> p4tcc2: on cpu2 >> est3: on cpu3 >> p4tcc3: on cpu3 >> est4: on cpu4 >> p4tcc4: on cpu4 >> est5: on cpu5 >> p4tcc5: on cpu5 >> est6: on cpu6 >> p4tcc6: on cpu6 >> est7: on cpu7 >> p4tcc7: on cpu7 >> est8: on cpu8 >> p4tcc8: on cpu8 >> est9: on cpu9 >> p4tcc9: on cpu9 >> est10: on cpu10 >> p4tcc10: on cpu10 >> est11: on cpu11 >> p4tcc11: on cpu11 >> ZFS filesystem version: 5 >> ZFS storage pool version: features support (5000) >> Timecounters tick every 1.000 msec >> usbus0: 5.0Gbps Super Speed USB v3.0 >> usbus1: 480Mbps High Speed USB v2.0 >> usbus2: 5.0Gbps Super Speed USB v3.0 >> usbus3: 480Mbps High Speed USB v2.0 >> ugen1.1: at usbus1 >> uhub0: on usbus1 >> ugen0.1: <0x1033> at usbus0 >> uhub1: <0x1033 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 >> ugen3.1: at usbus3 >> uhub2: on usbus3 >> ugen2.1: <0x1033> at usbus2 >> uhub3: <0x1033 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2 >> ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 >> ses0: SEMB S-E-S 2.00 device >> ses0: SEMB SES Device >> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 >> ada0: ATA-9 SATA 3.x device >> ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) >> ada0: Command Queueing enabled >> ada0: 122104MB (250069680 512 byte sectors: 16H 63S/T 16383C) >> ada0: Previously was known as ad4 >> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 >> ada1: ATA-9 SATA 3.x device >> ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) >> ada1: Command Queueing enabled >> ada1: 114473MB (234441648 512 byte sectors: 16H 63S/T 16383C) >> ada1: Previously was known as ad6 >> ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 >> ada2: ATA-8 SATA 3.x device >> ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) >> ada2: Command Queueing enabled >> ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) >> ada2: Previously was known as ad8 >> ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 >> ada3: ATA-8 SATA 3.x device >> ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) >> ada3: Command Queueing enabled >> ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) >> ada3: Previously was known as ad10 >> ada4 at ahcich4 bus 0 scbus4 target 0 lun 0 >> ada4: ATA-8 SATA 3.x device >> ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) >> ada4: Command Queueing enabled >> ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) >> ada4: Previously was known as ad12 >> ada5 at ahcich5 bus 0 scbus5 target 0 lun 0 >> ada5: ATA-8 SATA 3.x device >> ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) >> ada5: Command Queueing enabled >> ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) >> ada5: Previously was known as ad14 >> lapic1: Forcing LINT1 to edge trigger >> >> SMP: AP CPU #1 Launched! >> lapic4: Forcing LINT1 to edge trigger >> SMP: AP CPU #4 Launched! >> lapic11: Forcing LINT1 to edge trigger >> SMP: AP CPU #11 Launched! >> lapic7: Forcing LINT1 to edge trigger >> SMP: AP CPU #7 Launched! >> lapic5: Forcing LINT1 to edge trigger >> SMP: AP CPU #5 Launched! >> lapic9: Forcing LINT1 to edge trigger >> SMP: AP CPU #9 Launched! >> lapic2: Forcing LINT1 to edge trigger >> SMP: AP CPU #2 Launched! >> lapic3: Forcing LINT1 to edge trigger >> SMP: AP CPU #3 Launched! >> lapic10: Forcing LINT1 to edge trigger >> SMP: AP CPU #10 Launched! >> lapic6: Forcing LINT1 to edge trigger >> SMP: AP CPU #6 Launched! >> lapic8: Forcing LINT1 to edge trigger >> SMP: AP CPU #8 Launched! >> uhub3: 4 ports with 4 removable, self powered >> uhub1: 4 ports with 4 removable, self powered >> Root mount waiting for: usbus3 usbus1 >> uhub0: 2 ports with 2 removable, self powered >> uhub2: 2 ports with 2 removable, self powered >> Root mount waiting for: usbus3 usbus1 >> ugen1.2: at usbus1 >> uhub4: >> on usbus1 >> ugen3.2: at usbus3 >> uhub5: >> on usbus3 >> Root mount waiting for: usbus3 usbus1 >> uhub4: 6 ports with 6 removable, self powered >> uhub5: 8 ports with 8 removable, self powered >> ugen1.3: at usbus1 >> ukbd0: on >> usbus1 >> kbd2 at ukbd0 >> ugen3.3: at usbus3 >> Root mount waiting for: usbus3 >> ugen3.4: at usbus3 >> Trying to mount root from zfs:tron []... >> ums0: on >> usbus1 >> ums0: 5 buttons and [XYZ] coordinates ID=0 >> run0: <1.0> on usbus3 >> run0: MAC/BBP RT3070 (rev 0x0201), RF RT3020 (MIMO 1T1R), address >> 94:db:c9:e3:6e:c8 >> ubt0: >> on usbus3 >> WARNING: attempt to domain_add(bluetooth) after domainfinalize() >> WARNING: attempt to domain_add(netgraph) after domainfinalize() >> tap0: Ethernet address: 00:bd:8c:54:07:00 >> tap0: link state changed to UP >> bridge0: Ethernet address: 02:4c:89:ce:33:00 >> tap0: promiscuous mode enabled >> bridge0: link state changed to UP >> em1: link state changed to DOWN >> em1: promiscuous mode enabled >> em1: link state changed to UP >> tap0: link state changed to DOWN >> tap0: link state changed to UP >> tap0: link state changed to DOWN >> tap0: link state changed to UP >> >> >> >> >>> Yep, no problem. I saw the checkin and started a rebuild a few minutes >>> ago :-) >>> >>> Thanks, >>> >>> Dan >>> >>> On Sat, 30 Mar 2013, Neel Natu wrote: >>> >>> Hi Dan, >>>> >>>> On Fri, Mar 29, 2013 at 3:20 PM, Dan Mack wrote: >>>> >>>> >>>>> >>>>> Here you go Neel: >>>>> >>>>> root@olive:~ # bhyvectl --vm=coco --get-stats --cpu=0 >>>>> vcpu0 >>>>> vm exits due to external interrupt 45051 >>>>> number of times hlt was intercepted 2277 >>>>> number of NMIs delivered to vcpu 0 >>>>> vcpu total runtime 105001652482 >>>>> number of ticks vcpu was idle 919 >>>>> vcpu migration across host cpus 13 >>>>> number of times hlt was ignored 0 >>>>> total number of vm exits 9240925 >>>>> root@olive:~ # >>>>> root@olive:~ # >>>>> root@olive:~ # bhyvectl --vm=coco --get-stats --cpu=1 >>>>> vcpu1 >>>>> vm exits due to external interrupt 149431 >>>>> number of times hlt was intercepted 4222 >>>>> number of NMIs delivered to vcpu 0 >>>>> vcpu total runtime 106876622528 >>>>> number of ticks vcpu was idle 869 >>>>> vcpu migration across host cpus 20 >>>>> number of times hlt was ignored 0 >>>>> total number of vm exits 8065909 >>>>> >>>>> >>>>> That did not help a whole lot because we were not keeping track of >>>> all the >>>> reasons a vcpu could exit. >>>> >>>> I have fixed that in r248935: >>>> http://svnweb.freebsd.org/**base?view=revision&revision=**248935 >>>> >>>> Do you mind updating your vmm.ko with the change and getting the stats >>>> again? >>>> >>>> best >>>> Neel >>>> >>>> >>>> Dan >>>>> >>>>> >>>>> >>>>> On Fri, 29 Mar 2013, Neel Natu wrote: >>>>> >>>>> Hi Dan, >>>>> >>>>>> >>>>>> On Fri, Mar 29, 2013 at 12:38 PM, Dan Mack >>>>>> wrote: >>>>>> >>>>>> >>>>>> I ran the procsystime dtrace script on bhyve when it was chewing up >>>>>>> all >>>>>>> the CPU in vCPU=2 mode, and this is what I see for about 10s of >>>>>>> runtime: >>>>>>> >>>>>>> root@olive:/usr/share/dtrace/******toolkit # ./procsystime -n bhyve >>>>>>> -aT >>>>>>> >>>>>>> Tracing... Hit Ctrl-C to end... >>>>>>> dtrace: 158536 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 207447 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 189205 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 164341 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 246307 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 187640 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 214771 dynamic variable drops with non-empty dirty list >>>>>>> dtrace: 221265 dynamic variable drops with non-empty dirty list >>>>>>> ^C >>>>>>> dtrace: 243468 dynamic variable drops with non-empty dirty list >>>>>>> >>>>>>> Elapsed Times for processes bhyve, >>>>>>> >>>>>>> SYSCALL TIME (ns) >>>>>>> _umtx_op 18349 >>>>>>> writev 135712 >>>>>>> preadv 16175267 >>>>>>> pwritev 22924378 >>>>>>> ioctl 4353897920 >>>>>>> TOTAL: 4393151626 >>>>>>> >>>>>>> CPU Times for processes bhyve, >>>>>>> >>>>>>> SYSCALL TIME (ns) >>>>>>> _umtx_op 8815 >>>>>>> writev 103145 >>>>>>> pwritev 10647023 >>>>>>> preadv 15159976 >>>>>>> ioctl 3943399889 >>>>>>> TOTAL: 3969318848 >>>>>>> >>>>>>> Syscall Counts for processes bhyve, >>>>>>> >>>>>>> SYSCALL COUNT >>>>>>> _umtx_op 17 >>>>>>> writev 43 >>>>>>> pwritev 639 >>>>>>> preadv 1183 >>>>>>> ioctl 652782 >>>>>>> TOTAL: 654664 >>>>>>> >>>>>>> Not sure if that helps or not. Hotkernel shows the cpu in the >>>>>>> kernel`acpi_cpu_c1 call most of the time, normal ? >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> root@olive:/usr/share/dtrace/******toolkit # ./hotkernel >>>>>>> >>>>>>> Sampling... Hit Ctrl-C to end. >>>>>>> ^C >>>>>>> FUNCTION COUNT PCNT >>>>>>> zfs.ko`space_map_sync 1 0.0% >>>>>>> kernel`syscall_thread_exit 1 0.0% >>>>>>> kernel`should_yield 1 0.0% >>>>>>> zfs.ko`lzjb_compress 1 0.0% >>>>>>> zfs.ko`dbuf_rele 1 0.0% >>>>>>> zfs.ko`dsl_pool_tempreserve_******space 1 >>>>>>> 0.0% >>>>>>> >>>>>>> kernel`_bus_dmamap_load_buffer 1 0.0% >>>>>>> kernel`cpu_idleclock 1 0.0% >>>>>>> kernel`_rw_runlock_cookie 1 0.0% >>>>>>> kernel`pmap_enter 1 0.0% >>>>>>> kernel`__lockmgr_args 1 0.0% >>>>>>> zfs.ko`vdev_geom_io_start 1 0.0% >>>>>>> kernel`ahci_ch_intr 1 0.0% >>>>>>> kernel`__rw_rlock 1 0.0% >>>>>>> kernel`VOP_UNLOCK_APV 1 0.0% >>>>>>> zfs.ko`arc_buf_thaw 1 0.0% >>>>>>> kernel`atomic_set_int 1 0.0% >>>>>>> kernel`g_io_schedule_down 1 0.0% >>>>>>> kernel`__mtx_lock_flags 1 0.0% >>>>>>> kernel`_callout_stop_safe 1 0.0% >>>>>>> kernel`sched_idletd 1 0.0% >>>>>>> kernel`_sx_slock 1 0.0% >>>>>>> vmm.ko`lapic_set_intr 1 0.0% >>>>>>> kernel`cpusetobj_ffs 1 0.0% >>>>>>> zfs.ko`dbuf_hold 1 0.0% >>>>>>> vmm.ko`vlapic_intr_accepted 1 0.0% >>>>>>> zfs.ko`sa_attr_op 1 0.0% >>>>>>> zfs.ko`dmu_buf_hold_array_by_******dnode 1 >>>>>>> 0.0% >>>>>>> >>>>>>> zfs.ko`taskq_run_safe 1 0.0% >>>>>>> kernel`vm_reserv_free_page 1 0.0% >>>>>>> kernel`_sx_sunlock 1 0.0% >>>>>>> kernel`vn_rlimit_fsize 1 0.0% >>>>>>> zfs.ko`vdev_queue_io_to_issue 1 0.0% >>>>>>> kernel`vn_write 1 0.0% >>>>>>> kernel`lock_init 1 0.0% >>>>>>> vmm.ko`vmx_resume 1 0.0% >>>>>>> zfs.ko`zio_gang_tree_free 1 0.0% >>>>>>> zfs.ko`dbuf_rele_and_unlock 1 0.0% >>>>>>> kernel`mnt_vnode_next_active 1 0.0% >>>>>>> kernel`vm_page_remove 1 0.0% >>>>>>> zfs.ko`vdev_queue_deadline_******compare 1 >>>>>>> 0.0% >>>>>>> >>>>>>> kernel`zone_free_item 1 0.0% >>>>>>> vmm.ko`vmx_launch 1 0.0% >>>>>>> kernel`m_pullup 1 0.0% >>>>>>> kernel`pmap_change_wiring 1 0.0% >>>>>>> kernel`sched_userret 1 0.0% >>>>>>> zfs.ko`dbuf_write_done 1 0.0% >>>>>>> zfs.ko`zio_execute 1 0.0% >>>>>>> kernel`PHYS_TO_VM_PAGE 2 0.0% >>>>>>> zfs.ko`avl_rotation 2 0.0% >>>>>>> kernel`ehci_device_isoc_fs_******enter 2 >>>>>>> 0.0% >>>>>>> >>>>>>> vmm.ko`vm_exitinfo 2 0.0% >>>>>>> zfs.ko`zfs_freebsd_write 2 0.0% >>>>>>> vmm.ko`vcpu_stats 2 0.0% >>>>>>> kernel`atomic_add_long 2 0.0% >>>>>>> vmm.ko`0xffffffff81 2 0.0% >>>>>>> kernel`vm_reserv_alloc_page 2 0.0% >>>>>>> kernel`pagezero 2 0.0% >>>>>>> kernel`bzero 2 0.0% >>>>>>> zfs.ko`fletcher_2_native 2 0.0% >>>>>>> vmm.ko`vmx_getreg 2 0.0% >>>>>>> vmm.ko`vm_guest_msrs 3 0.0% >>>>>>> vmm.ko`vmx_setreg 3 0.0% >>>>>>> vmm.ko`vm_get_register 3 0.0% >>>>>>> kernel`sx_try_xlock_ 3 0.0% >>>>>>> kernel`vm_radix_lookup_le 3 0.0% >>>>>>> vmm.ko`lapic_pending_intr 4 0.0% >>>>>>> kernel`syscall_thread_enter 4 0.0% >>>>>>> kernel`cpu_set_syscall_retval 4 0.0% >>>>>>> kernel`__mnt_vnode_next_all 4 0.0% >>>>>>> kernel`vfs_stdsync 4 0.0% >>>>>>> vmm.ko`vmm_fetch_instruction 5 0.0% >>>>>>> vmm.ko`vlapic_op_mem_read 6 0.0% >>>>>>> kernel`_sx_xunlock 7 0.0% >>>>>>> vmm.ko`vmm_emulate_instruction 7 0.0% >>>>>>> vmm.ko`vlapic_op_mem_write 7 0.0% >>>>>>> kernel`__mtx_lock_spin_flags 8 0.0% >>>>>>> vmm.ko`lapic_timer_tick 8 0.0% >>>>>>> zfs.ko`list_next 8 0.0% >>>>>>> zfs.ko`l2arc_feed_thread 8 0.0% >>>>>>> kernel`userret 9 0.0% >>>>>>> vmm.ko`vm_gpa2hpa 9 0.0% >>>>>>> vmm.ko`vmm_decode_instruction 11 0.0% >>>>>>> vmm.ko`ept_vmmmap_get 11 0.0% >>>>>>> vmm.ko`vm_lapic 11 0.0% >>>>>>> kernel`spinlock_enter 14 0.0% >>>>>>> kernel`__mtx_lock_sleep 15 0.0% >>>>>>> vmm.ko`vcpu_set_state 16 0.0% >>>>>>> vmm.ko`vm_nmi_pending 16 0.0% >>>>>>> kernel`malloc_type_zone_******allocated 16 >>>>>>> 0.0% >>>>>>> >>>>>>> kernel`devfs_ioctl_f 16 0.0% >>>>>>> kernel`pmap_kextract 17 0.0% >>>>>>> kernel`sys_ioctl 18 0.0% >>>>>>> kernel`malloc 21 0.0% >>>>>>> kernel`cpu_fetch_syscall_args 24 0.0% >>>>>>> kernel`uma_zfree_arg 28 0.0% >>>>>>> vmm.ko`vlapic_pending_intr 29 0.0% >>>>>>> kernel`trap 30 0.0% >>>>>>> kernel`hpet_get_timecount 31 0.0% >>>>>>> dtrace.ko`dtrace_trap 35 0.0% >>>>>>> vmm.ko`vlapic_update_ppr 39 0.0% >>>>>>> vmm.ko`vmmdev_ioctl 40 0.0% >>>>>>> kernel`copyout 43 0.0% >>>>>>> kernel`critical_enter 44 0.0% >>>>>>> kernel`copyin 51 0.1% >>>>>>> vmm.ko`vmcs_read 53 0.1% >>>>>>> kernel`fpusave 61 0.1% >>>>>>> kernel`uma_zalloc_arg 62 0.1% >>>>>>> kernel`devvn_refthread 69 0.1% >>>>>>> kernel`atomic_load_acq_int 70 0.1% >>>>>>> kernel`fpurestore 70 0.1% >>>>>>> vmm.ko`vlapic_timer_tick 75 0.1% >>>>>>> kernel`amd64_syscall 77 0.1% >>>>>>> kernel`bcopy 91 0.1% >>>>>>> kernel`fpudna 103 0.1% >>>>>>> kernel`free 103 0.1% >>>>>>> kernel`critical_exit 105 0.1% >>>>>>> kernel`dev_relthread 108 0.1% >>>>>>> kernel`spinlock_exit 119 0.1% >>>>>>> kernel`kern_ioctl 138 0.1% >>>>>>> kernel`fpuexit 242 0.3% >>>>>>> kernel`0xffffffff80 306 0.3% >>>>>>> vmm.ko`restore_host_msrs 398 0.4% >>>>>>> vmm.ko`vm_run 428 0.4% >>>>>>> vmm.ko`restore_guest_msrs 433 0.5% >>>>>>> kernel`cpu_idle 906 1.0% >>>>>>> vmm.ko`vmx_run 9266 9.7% >>>>>>> kernel`acpi_cpu_c1 81134 85.2% >>>>>>> >>>>>>> Hope this helps, let me know if you have any ideas on what to try >>>>>>> next. >>>>>>> >>>>>>> >>>>>>> Could you try the following command to get vcpu stats? >>>>>>> >>>>>> sudo bhyvectl --vm=vm1 --get-stats --cpu=0 >>>>>> sudo bhyvectl --vm=vm1 --get-stats --cpu=1 >>>>>> >>>>>> best >>>>>> Neel >>>>>> >>>>>> >>>>>> Dan >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, 29 Mar 2013, Dan Mack wrote: >>>>>>> >>>>>>> On Thu, 28 Mar 2013, Peter Grehan wrote: >>>>>>> >>>>>>> >>>>>>>> Hi Dan, >>>>>>>> >>>>>>>> >>>>>>>>> Any ideas? >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Just to confirm: your host system is an i7 3930k, which looks >>>>>>>>> like a 6 >>>>>>>>> core system. Should be plenty beefy enough for a 2 vCPU guest. >>>>>>>>> >>>>>>>>> >>>>>>>>> That's what I thought, 6 cores, 12 threads, 64 GB RAM. >>>>>>>> >>>>>>>> The issue you are seeing looks a bit like what happens when >>>>>>>> systems are >>>>>>>> >>>>>>>> heavily oversubscribed and vCPUs can't get enough run time. >>>>>>>>> >>>>>>>>> >>>>>>>>> I was running in on an idle system; only bhyve was running. >>>>>>>> >>>>>>>> >>>>>>>> How much spare RAM is on the host when you started the vm ? Also, >>>>>>>> is >>>>>>>> >>>>>>>> there much running on the host when the guest is started ? >>>>>>>>> >>>>>>>>> >>>>>>>>> 64GB >>>>>>>> >>>>>>>> Here's some more information (sorry for the formatting): >>>>>>>> >>>>>>>> Case 1: single vCPU via: >>>>>>>> root 64112 1.3 0.0 4219232 25588 1 D+ 2:09PM 0:02.47 >>>>>>>> /usr/sbin/bhyve -c 1 -m 4096 -AI -H -P -g 0 -s 0:0,hostbridge -s >>>>>>>> 1:0,virtio-net,tap0 -s 2:0,virtio-blk,./diskdev -S 31,uart,stdio >>>>>>>> coco >>>>>>>> >>>>>>>> Top shows it idle (note low load, 4GB memory given to guest, still >>>>>>>> 53GB >>>>>>>> free RAM, nothing else clocking CPU cycles): >>>>>>>> >>>>>>>> root@olive:~ # top -bH >>>>>>>> last pid: 64148; load averages: 0.03, 0.22, 0.22 up 0+16:29:31 >>>>>>>> 14:11:51 >>>>>>>> 24 processes: 1 running, 23 sleeping >>>>>>>> >>>>>>>> Mem: 32M Active, 50M Inact, 8498M Wired, 504K Cache, 53G Free >>>>>>>> ARC: 778M Total, 146M MFU, 390M MRU, 16K Anon, 16M Header, 226M >>>>>>>> Other >>>>>>>> Swap: >>>>>>>> >>>>>>>> >>>>>>>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU >>>>>>>> COMMAND >>>>>>>> 64112 root 20 0 4120M 25980K vmidle 2 0:03 0.00% >>>>>>>> bhyve{coco vcpu 0} >>>>>>>> 925 root 20 0 14296K 2044K select 11 0:00 0.00% >>>>>>>> syslogd >>>>>>>> 2695 root 20 0 81452K 6952K select 2 0:00 0.00% sshd >>>>>>>> 2697 root 20 0 20224K 4632K ttyin 1 0:00 0.00% csh >>>>>>>> 64112 root 52 0 4120M 25980K kqread 9 0:00 0.00% >>>>>>>> bhyve{coco mevent} >>>>>>>> >>>>>>>> >>>>>>>> So, case #1 all is normal. I was able to build a kernel with very >>>>>>>> good >>>>>>>> performance. >>>>>>>> >>>>>>>> The boot sequence looks like the following: >>>>>>>> >>>>>>>> root@olive:/vms/coco # ./vmrun.sh coco >>>>>>>> Launching virtual machine "coco" ... >>>>>>>> Consoles: userboot >>>>>>>> >>>>>>>> FreeBSD/amd64 User boot, Revision 1.1 >>>>>>>> (root@olive.example.com, Thu Mar 28 21:20:49 CDT 2013) >>>>>>>> Loading /boot/defaults/loader.conf /boot/kernel/kernel text=0x5a2178 >>>>>>>> data=0x98088+0xef8b0 syms=[0x8+0xca8d8+0x8+******0x12bb08] >>>>>>>> >>>>>>>> / >>>>>>>> ______ ____ _____ _____ >>>>>>>> | ____| | _ \ / ____| __ \ >>>>>>>> | |___ _ __ ___ ___ | |_) | (___ | | | | >>>>>>>> | ___| '__/ _ \/ _ \| _ < \___ \| | | | >>>>>>>> | | | | | __/ __/| |_) |____) | |__| | >>>>>>>> | | | | | | || | | | >>>>>>>> |_| |_| \___|\___||____/|_____/|_____/ ``` >>>>>>>> ` >>>>>>>> s` >>>>>>>> `.....---.......--.``` >>>>>>>> -/ >>>>>>>> +------------Welcome to FreeBSD-----------+ +o .--` /y:` >>>>>>>> +. >>>>>>>> | | yo`:. :o >>>>>>>> `+- >>>>>>>> | 1. Boot Multi User [Enter] | y/ -/` >>>>>>>> -o/ >>>>>>>> | 2. Boot [S]ingle User | .- >>>>>>>> ::/sy+:. >>>>>>>> | 3. [Esc]ape to loader prompt | / >>>>>>>> `-- >>>>>>>> / >>>>>>>> | 4. Reboot | `: >>>>>>>> :` >>>>>>>> | | `: >>>>>>>> :` >>>>>>>> | Options: | / >>>>>>>> / >>>>>>>> | 5. Configure Boot [O]ptions... | .- >>>>>>>> -. >>>>>>>> | | -- -. >>>>>>>> | | `:` >>>>>>>> `:` >>>>>>>> | | .-- >>>>>>>> `--. >>>>>>>> | | .---.....----. >>>>>>>> +-----------------------------******------------+ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Booting... >>>>>>>> Copyright (c) 1992-2013 The FreeBSD Project. >>>>>>>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, >>>>>>>> 1994 >>>>>>>> The Regents of the University of California. All rights >>>>>>>> reserved. >>>>>>>> FreeBSD is a registered trademark of The FreeBSD Foundation. >>>>>>>> FreeBSD 10.0-CURRENT #0 r248859: Thu Mar 28 19:52:06 CDT 2013 >>>>>>>> root@cocopuff.example.com:/******sys/amd64/compile/BHYVE amd64 >>>>>>>> >>>>>>>> FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 >>>>>>>> CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz (3199.84-MHz K8-class >>>>>>>> CPU) >>>>>>>> Origin = "GenuineIntel" Id = 0x206d7 Family = 0x6 Model = 0x2d >>>>>>>> Stepping = 7 >>>>>>>> Features=0x8fa3ab7f>>>>>>> SEP,PGE,CMOV,PAT,PSE36,DTS,******MMX,FXSR,SSE,SSE2,SS,PBE> >>>>>>>> Features2=0x83bee217>>>>>>> **** >>>>>>>> CX16,xTPR,PDCM,PCID,DCA,SSE4.******1,SSE4.2,x2APIC,POPCNT,** >>>>>>>> TSCDLT,**AESNI,HV> >>>>>>>> AMD Features=0x2c100800 >>>>>>>> >>>>>>>> AMD Features2=0x1 >>>>>>>> TSC: P-state invariant >>>>>>>> real memory = 5368709120 (5120 MB) >>>>>>>> avail memory = 3957534720 (3774 MB) >>>>>>>> Event timer "LAPIC" quality 400 >>>>>>>> ACPI APIC Table: >>>>>>>> random device not loaded; using insecure entropy >>>>>>>> ioapic0: Changing APIC ID to 1 >>>>>>>> ioapic0 irqs 0-16 on motherboard >>>>>>>> kbd0 at kbdmux0 >>>>>>>> acpi0: on motherboard >>>>>>>> atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 >>>>>>>> Event timer "RTC" frequency 32768 Hz quality 0 >>>>>>>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >>>>>>>> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >>>>>>>> pcib0: port 0xcf8-0xcff on acpi0 >>>>>>>> pci0: on pcib0 >>>>>>>> pcib0: no PRT entry for 0.31.INTA >>>>>>>> virtio_pci0: port 0x2000-0x201f mem >>>>>>>> 0xc0000000-0xc0001fff at device 1.0 on pci0 >>>>>>>> vtnet0: on virtio_pci0 >>>>>>>> virtio_pci0: host features: 0x18020 >>>>>>>> virtio_pci0: negotiated features: 0x18020 >>>>>>>> >>>>>>>> vtnet0: Ethernet address: 00:a0:98:ac:0d:df >>>>>>>> virtio_pci1: port 0x2040-0x207f mem >>>>>>>> 0xc0002000-0xc0003fff at device 2.0 on pci0 >>>>>>>> vtblk0: on virtio_pci1 >>>>>>>> virtio_pci1: host features: 0x10000004 >>>>>>>> virtio_pci1: negotiated features: 0x10000004 >>>>>>>> >>>>>>>> vtblk0: 8192MB (16777216 512 byte sectors) >>>>>>>> uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 >>>>>>>> on >>>>>>>> pci0 >>>>>>>> uart2: console (9600,n,8,1) >>>>>>>> Timecounters tick every 10.000 msec >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> Timecounter "TSC-low" frequency 1599917752 Hz quality 1000 >>>>>>>> Trying to mount root from ufs:/dev/vtbd0p2 [rw]... >>>>>>>> Setting hostuuid: d4e0b8a0-97c6-11e2-b4c8-******00a098ac0ddf. >>>>>>>> >>>>>>>> Setting hostid: 0x24bfd025. >>>>>>>> Entropy harvesting: interrupts ethernet point_to_point kickstart. >>>>>>>> Starting file system checks: >>>>>>>> /dev/vtbd0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS >>>>>>>> /dev/vtbd0p2: clean, 610778 free (14298 frags, 74560 blocks, 0.7% >>>>>>>> fragmentation) >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> Mounting local file systems:. >>>>>>>> Writing entropy file:. >>>>>>>> Setting hostname: cocopuff.example.com. >>>>>>>> vtnet0: link state changed to UP >>>>>>>> Starting Network: lo0 vtnet0. >>>>>>>> lo0: flags=8049 metric 0 mtu >>>>>>>> 16384 >>>>>>>> options=600003 >>>>>>>> >>>>>>>> inet6 ::1 prefixlen 128 >>>>>>>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >>>>>>>> inet 127.0.0.1 netmask 0xff000000 >>>>>>>> nd6 options=21 >>>>>>>> vtnet0: flags=8943>>>>>>> MULTICAST> >>>>>>>> metric 0 mtu 1500 >>>>>>>> options=80028 >>>>>>>> >>>>>>>> ether 00:a0:98:ac:0d:df >>>>>>>> inet 172.22.222.230 netmask 0xffff0000 broadcast >>>>>>>> 172.22.255.255 >>>>>>>> inet6 fe80::2a0:98ff:feac:ddf%vtnet0 prefixlen 64 scopeid >>>>>>>> 0x1 >>>>>>>> nd6 options=29 >>>>>>>> >>>>>>>> media: Ethernet 1000baseT >>>>>>>> status: active >>>>>>>> Starting devd. >>>>>>>> add net default: gateway 172.22.22.252 >>>>>>>> add net ::ffff:0.0.0.0: gateway ::1 >>>>>>>> add net ::0.0.0.0: gateway ::1 >>>>>>>> add net fe80::: gateway ::1 >>>>>>>> add net ff02::: gateway ::1 >>>>>>>> ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib >>>>>>>> 32-bit compatibility ldconfig path: /usr/lib32 >>>>>>>> Creating and/or trimming log files. >>>>>>>> Starting syslogd. >>>>>>>> Clearing /tmp (X related). >>>>>>>> Updating motd:. >>>>>>>> Performing sanity check on sshd configuration. >>>>>>>> Starting sshd. >>>>>>>> Starting cron. >>>>>>>> Starting background file system checks in 60 seconds. >>>>>>>> >>>>>>>> Fri Mar 29 14:09:20 CDT 2013 >>>>>>>> >>>>>>>> FreeBSD/amd64 (cocopuff.example.com) (console) >>>>>>>> >>>>>>>> login: >>>>>>>> >>>>>>>> >>>>>>>> ====== Next, boot the same image with two (2) vCPU: >>>>>>>> >>>>>>>> Command run by vmrun.sh: >>>>>>>> >>>>>>>> root 64163 182.5 0.1 4219360 48968 1 R+ 2:16PM 1:10.42 >>>>>>>> /usr/sbin/bhyve -c 2 -m 4096 -AI -H -P -g 0 -s 0:0,hostbridge -s >>>>>>>> 1:0,virtio-net,tap0 -s 2:0,virtio-blk,./diskdev -S 31,uart,stdio >>>>>>>> coco >>>>>>>> >>>>>>>> Top shows it running out of control: >>>>>>>> >>>>>>>> root@olive:~ # top -bH >>>>>>>> last pid: 64191; load averages: 1.45, 0.52, 0.31 up 0+16:35:12 >>>>>>>> 14:17:32 >>>>>>>> 25 processes: 3 running, 22 sleeping >>>>>>>> >>>>>>>> Mem: 32M Active, 50M Inact, 9016M Wired, 504K Cache, 53G Free >>>>>>>> ARC: 1293M Total, 370M MFU, 671M MRU, 8208K Anon, 17M Header, 227M >>>>>>>> Other >>>>>>>> Swap: >>>>>>>> >>>>>>>> >>>>>>>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU >>>>>>>> COMMAND >>>>>>>> 64163 root 102 0 4120M 120M CPU2 2 1:01 100.00% >>>>>>>> bhyve{coco vcpu 1} >>>>>>>> 64163 root 102 0 4120M 120M CPU8 8 1:03 94.48% >>>>>>>> bhyve{coco vcpu 0} >>>>>>>> 15751 root 20 0 81452K 7092K select 5 0:00 0.00% sshd >>>>>>>> 807 root 20 0 13112K 4812K select 5 0:00 0.00% devd >>>>>>>> 925 root 20 0 14296K 2044K select 9 0:00 0.00% >>>>>>>> syslogd >>>>>>>> 1031 root 20 0 16380K 2400K nanslp 10 0:00 0.00% cron >>>>>>>> 2695 root 20 0 81452K 6952K select 2 0:00 0.00% sshd >>>>>>>> 2697 root 20 0 20224K 4632K ttyin 1 0:00 0.00% csh >>>>>>>> 64163 root 52 0 4120M 120M kqread 5 0:00 0.00% >>>>>>>> bhyve{coco mevent} >>>>>>>> 15753 root 20 0 20224K 4580K pause 1 0:00 0.00% csh >>>>>>>> >>>>>>>> Even though, it's just sitting at the login prompt: >>>>>>>> >>>>>>>> root@olive:/vms/coco # ./vmrun.sh coco >>>>>>>> Launching virtual machine "coco" ... >>>>>>>> Consoles: userboot >>>>>>>> >>>>>>>> FreeBSD/amd64 User boot, Revision 1.1 >>>>>>>> (root@olive.example.com, Thu Mar 28 21:20:49 CDT 2013) >>>>>>>> Loading /boot/defaults/loader.conf /boot/kernel/kernel text=0x5a2178 >>>>>>>> data=0x98088+0xef8b0 syms=[0x8+0xca8d8+0x8+******0x12bb08] >>>>>>>> >>>>>>>> / >>>>>>>> ______ ____ _____ _____ >>>>>>>> | ____| | _ \ / ____| __ \ >>>>>>>> | |___ _ __ ___ ___ | |_) | (___ | | | | >>>>>>>> | ___| '__/ _ \/ _ \| _ < \___ \| | | | >>>>>>>> | | | | | __/ __/| |_) |____) | |__| | >>>>>>>> | | | | | | || | | | >>>>>>>> |_| |_| \___|\___||____/|_____/|_____/ ``` >>>>>>>> ` >>>>>>>> s` >>>>>>>> `.....---.......--.``` >>>>>>>> -/ >>>>>>>> +------------Welcome to FreeBSD-----------+ +o .--` /y:` >>>>>>>> +. >>>>>>>> | | yo`:. :o >>>>>>>> `+- >>>>>>>> | 1. Boot Multi User [Enter] | y/ -/` >>>>>>>> -o/ >>>>>>>> | 2. Boot [S]ingle User | .- >>>>>>>> ::/sy+:. >>>>>>>> | 3. [Esc]ape to loader prompt | / >>>>>>>> `-- >>>>>>>> / >>>>>>>> | 4. Reboot | `: >>>>>>>> :` >>>>>>>> | | `: >>>>>>>> :` >>>>>>>> | Options: | / >>>>>>>> / >>>>>>>> | 5. Configure Boot [O]ptions... | .- >>>>>>>> -. >>>>>>>> | | -- -. >>>>>>>> | | `:` >>>>>>>> `:` >>>>>>>> | | .-- >>>>>>>> `--. >>>>>>>> | | .---.....----. >>>>>>>> +-----------------------------******------------+ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Booting... >>>>>>>> Copyright (c) 1992-2013 The FreeBSD Project. >>>>>>>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, >>>>>>>> 1994 >>>>>>>> The Regents of the University of California. All rights >>>>>>>> reserved. >>>>>>>> FreeBSD is a registered trademark of The FreeBSD Foundation. >>>>>>>> FreeBSD 10.0-CURRENT #0 r248859: Thu Mar 28 19:52:06 CDT 2013 >>>>>>>> root@cocopuff.example.com:/******sys/amd64/compile/BHYVE amd64 >>>>>>>> >>>>>>>> FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 >>>>>>>> CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz (3199.82-MHz K8-class >>>>>>>> CPU) >>>>>>>> Origin = "GenuineIntel" Id = 0x206d7 Family = 0x6 Model = 0x2d >>>>>>>> Stepping = 7 >>>>>>>> Features=0x8fa3ab7f>>>>>>> SEP,PGE,CMOV,PAT,PSE36,DTS,******MMX,FXSR,SSE,SSE2,SS,PBE> >>>>>>>> Features2=0x83bee217>>>>>>> **** >>>>>>>> CX16,xTPR,PDCM,PCID,DCA,SSE4.******1,SSE4.2,x2APIC,POPCNT,** >>>>>>>> TSCDLT,**AESNI,HV> >>>>>>>> AMD Features=0x2c100800 >>>>>>>> >>>>>>>> AMD Features2=0x1 >>>>>>>> TSC: P-state invariant >>>>>>>> real memory = 5368709120 (5120 MB) >>>>>>>> avail memory = 3957534720 (3774 MB) >>>>>>>> Event timer "LAPIC" quality 400 >>>>>>>> ACPI APIC Table: >>>>>>>> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs >>>>>>>> FreeBSD/SMP: 2 package(s) x 1 core(s) >>>>>>>> cpu0 (BSP): APIC ID: 0 >>>>>>>> cpu1 (AP): APIC ID: 1 >>>>>>>> random device not loaded; using insecure entropy >>>>>>>> ioapic0: Changing APIC ID to 2 >>>>>>>> ioapic0 irqs 0-16 on motherboard >>>>>>>> kbd0 at kbdmux0 >>>>>>>> acpi0: on motherboard >>>>>>>> atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 >>>>>>>> Event timer "RTC" frequency 32768 Hz quality 0 >>>>>>>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >>>>>>>> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >>>>>>>> pcib0: port 0xcf8-0xcff on acpi0 >>>>>>>> pci0: on pcib0 >>>>>>>> pcib0: no PRT entry for 0.31.INTA >>>>>>>> virtio_pci0: port 0x2000-0x201f mem >>>>>>>> 0xc0000000-0xc0001fff at device 1.0 on pci0 >>>>>>>> vtnet0: on virtio_pci0 >>>>>>>> virtio_pci0: host features: 0x18020 >>>>>>>> virtio_pci0: negotiated features: 0x18020 >>>>>>>> >>>>>>>> vtnet0: Ethernet address: 00:a0:98:ac:0d:df >>>>>>>> virtio_pci1: port 0x2040-0x207f mem >>>>>>>> 0xc0002000-0xc0003fff at device 2.0 on pci0 >>>>>>>> vtblk0: on virtio_pci1 >>>>>>>> virtio_pci1: host features: 0x10000004 >>>>>>>> virtio_pci1: negotiated features: 0x10000004 >>>>>>>> >>>>>>>> vtblk0: 8192MB (16777216 512 byte sectors) >>>>>>>> uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 >>>>>>>> on >>>>>>>> pci0 >>>>>>>> uart2: console (9600,n,8,1) >>>>>>>> Timecounters tick every 10.000 msec >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> SMP: AP CPU #1 Launched! >>>>>>>> Trying to mount root from ufs:/dev/vtbd0p2 [rw]... >>>>>>>> Setting hostuuid: d4e0b8a0-97c6-11e2-b4c8-******00a098ac0ddf. >>>>>>>> >>>>>>>> Setting hostid: 0x24bfd025. >>>>>>>> Entropy harvesting: interrupts ethernet point_to_point kickstart. >>>>>>>> Starting file system checks: >>>>>>>> /dev/vtbd0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS >>>>>>>> /dev/vtbd0p2: clean, 610776 free (14296 frags, 74560 blocks, 0.7% >>>>>>>> fragmentation) >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT >>>>>>>> Mounting local file systems:. >>>>>>>> Writing entropy file:. >>>>>>>> Setting hostname: cocopuff.example.com. >>>>>>>> vtnet0: link state changed to UP >>>>>>>> Starting Network: lo0 vtnet0. >>>>>>>> lo0: flags=8049 metric 0 mtu >>>>>>>> 16384 >>>>>>>> options=600003 >>>>>>>> >>>>>>>> inet6 ::1 prefixlen 128 >>>>>>>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >>>>>>>> inet 127.0.0.1 netmask 0xff000000 >>>>>>>> nd6 options=21 >>>>>>>> vtnet0: flags=8943>>>>>>> MULTICAST> >>>>>>>> metric 0 mtu 1500 >>>>>>>> options=80028 >>>>>>>> >>>>>>>> ether 00:a0:98:ac:0d:df >>>>>>>> inet 172.22.222.230 netmask 0xffff0000 broadcast >>>>>>>> 172.22.255.255 >>>>>>>> inet6 fe80::2a0:98ff:feac:ddf%vtnet0 prefixlen 64 scopeid >>>>>>>> 0x1 >>>>>>>> nd6 options=29 >>>>>>>> >>>>>>>> media: Ethernet 1000baseT >>>>>>>> status: active >>>>>>>> Starting devd. >>>>>>>> add net default: gateway 172.22.22.252 >>>>>>>> add net ::ffff:0.0.0.0: gateway ::1 >>>>>>>> add net ::0.0.0.0: gateway ::1 >>>>>>>> add net fe80::: gateway ::1 >>>>>>>> add net ff02::: gateway ::1 >>>>>>>> ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib >>>>>>>> 32-bit compatibility ldconfig path: /usr/lib32 >>>>>>>> Creating and/or trimming log files. >>>>>>>> Starting syslogd. >>>>>>>> syslogd: timed out waiting for child >>>>>>>> /etc/rc: WARNING: failed to start syslogd >>>>>>>> Clearing /tmp (X related). >>>>>>>> Updating motd:. >>>>>>>> Performing sanity check on sshd configuration. >>>>>>>> Starting sshd. >>>>>>>> Starting cron. >>>>>>>> Starting background file system checks in 60 seconds. >>>>>>>> >>>>>>>> Thu Apr 18 02:06:43 CDT 2013 >>>>>>>> >>>>>>>> FreeBSD/amd64 (cocopuff.example.com) (console) >>>>>>>> >>>>>>>> login: >>>>>>>> >>>>>>>> >>>>>>>> ========== end of boot sequence >>>>>>>> >>>>>>>> So, something is broken with SMP on the guest, I'd be happy to help >>>>>>>> just >>>>>>>> let me know. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> ______________________________******_________________ >>>>>>>> freebsd-virtualization@****freeb**sd.org < >>>>>>>> freebsd-virtualization@**freeb**sd.org < >>>>>>>> freebsd-virtualization@**freebsd.org >>>>>>>> >>mailing >>>>>>>> list >>>>>>>> http://lists.freebsd.org/******mailman/listinfo/freebsd-**** >>>>>>>> virtualization>>>>>>> listinfo/freebsd-****virtualization >>>>>>>> > >>>>>>>> >>>>>>> virtualization >>>>>>>> >>>>>>> freebsd-virtualization >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> To unsubscribe, send any mail to "freebsd-virtualization-** >>>>>>>> unsubscribe@freebsd.org >>>>>>> unsubscribe@freebsd.org>>>>>>> freebsd.org > >>>>>>>> >>>>>>>>> " >>>>>>>>> >>>>>>>> >>>>>>>> ______________________________******_________________ >>>>>>>> >>>>>>>> freebsd-virtualization@****freeb**sd.org < >>>>>>> freebsd-virtualization@**freeb**sd.org < >>>>>>> freebsd-virtualization@**freebsd.org >>>>>>> >>mailing >>>>>>> list >>>>>>> http://lists.freebsd.org/******mailman/listinfo/freebsd-****** >>>>>>> virtualization >>>>>>> >>>>>> virtualization >>>>>>> > >>>>>>> >>>>>> virtualization >>>>>>> >>>>>> freebsd-virtualization >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> To unsubscribe, send any mail to "freebsd-virtualization-** >>>>>>> unsubscribe@freebsd.org >>>>>> unsubscribe@freebsd.org >>>>>> unsubscribe@freebsd.org >>>>>>> > >>>>>>> >>>>>>>> " >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> ______________________________**_________________ >>> freebsd-virtualization@**freebsd.orgmailing list >>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**virtualization >>> To unsubscribe, send any mail to "freebsd-virtualization-** >>> unsubscribe@freebsd.org >>> " >>> >>> > From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 12 01:19:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0E159738 for ; Fri, 12 Apr 2013 01:19:00 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ia0-x236.google.com (mail-ia0-x236.google.com [IPv6:2607:f8b0:4001:c02::236]) by mx1.freebsd.org (Postfix) with ESMTP id D7AA376D for ; Fri, 12 Apr 2013 01:18:59 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id u20so1976909iag.27 for ; Thu, 11 Apr 2013 18:18:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=kf0ivpFqhIzO3xjbR3e/eYp+k4fup1cSPdfL/ML78Aw=; b=lNAADN20Glui1EMKF6fX/B8GWsxADOjr4zYu3Z4M59L/pOgNL9JgYQG9idHCYE2f8j pJsWvX1i33YKbA/7A7sgkJOJdOH6dGCQK4l8DVR5LE4Ys0hCakimZskgQYZFvueJRsM6 8azPwsTSRzjelN0nux7TKxeNy1FlM7Ol0Rra5b7LzAcsvIzDu6tUmPs5f+y5md0BDIZE A6feN3JTyE7GG0YlAM5nCG4gEN/ryR/4O+KglC5oSLhG1Os9CyK7TVaQ64Et7cRrDtMK zSb0ZmKupxJ61Bkp4cvEJRR4z3haxxE/BXBx/U6l/xJDaTy7hD7tyyd227u/CjEiFsbL cnZw== MIME-Version: 1.0 X-Received: by 10.50.213.97 with SMTP id nr1mr455386igc.36.1365729539559; Thu, 11 Apr 2013 18:18:59 -0700 (PDT) Received: by 10.43.9.138 with HTTP; Thu, 11 Apr 2013 18:18:59 -0700 (PDT) In-Reply-To: <201304060127.r361RRGv068417@elf.torek.net> References: <201304060127.r361RRGv068417@elf.torek.net> Date: Thu, 11 Apr 2013 18:18:59 -0700 Message-ID: Subject: Re: two more small improvements From: Neel Natu To: Chris Torek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2013 01:19:00 -0000 Hi Chris, Thanks for the patches. Committed as: http://svnweb.freebsd.org/base?view=revision&revision=249396 http://svnweb.freebsd.org/base?view=revision&revision=249351 best Neel On Fri, Apr 5, 2013 at 6:27 PM, Chris Torek wrote: > When I first went to run bhyve on one hardware box I had the > entire system crash. This turned out to be due to the BIOS > disabling VMX. A message came out on the console, but the scripts > continued anyway, and then, boom. I first looked at the wrong bit > of code for fixing this, but I *think* I improved it anyway. :-) > That's the first patch below. > > The second patch below (note: I'll put them somewhere on torek.net > if that's better than just including them here) fixes the crash. > The easiest way to reproduce it is to attempt to create a VM > within a bhyve VM ("bh1" is a running VM): > > [before patch -- note, some wraparound fixed] > > root@bh1:~ # kldload vmm > vmx_init: processor does not support VMX operation > module_register_init: MOD_LOAD (vmm, 0xffffffff81a133b0, 0) error 6 > root@bh1:~ # sysctl hw.vmm > hw.vmm.destroy: beavis > hw.vmm.create: beavis > hw.vmm.pages_allocated: 0 > root@bh1:~ # sysctl hw.vmm.create > hw.vmm.create: beavis > root@bh1:~ # sysctl hw.vmm.create= > hw.vmm.create: beavisvm exit[1] > reason VMX > rip 0xffffffff81a1932c > inst_length 6 > error 0 > exit_reason 50 > qualification 0xfffffffffffffff0 > vm exit[0] > reason VMX > rip 0xffffffff81a1932c > inst_length 6 > error 0 > exit_reason 50 > qualification 0xfffffffffffffff0 > > [after patch] > > root@bh1:~ # kldload vmm > vmx_init: processor does not support VMX operation > module_register_init: MOD_LOAD (vmm, 0xffffffff81a133d0, 0) error 6 > root@bh1:~ # sysctl hw.vmm > hw.vmm.destroy: beavis > hw.vmm.create: beavis > hw.vmm.pages_allocated: 0 > root@bh1:~ # sysctl hw.vmm.create > hw.vmm.create: beavis > root@bh1:~ # sysctl hw.vmm.create= > hw.vmm.create: beavis > sysctl: hw.vmm.create=: Device not configured > root@bh1:~ # > > Chris > > changeset: 1962:7e204f321854 > user: Chris Torek > date: Fri Apr 05 19:10:09 2013 -0600 > files: sys/amd64/vmm/intel/vmx.c > description: > improve test for VMX-mode > > We only need VMX extensions to be enabled, not locked. > Moreover, we should not care about any VMX-in-SMX settings. > > > diff --git a/sys/amd64/vmm/intel/vmx.c b/sys/amd64/vmm/intel/vmx.c > --- a/sys/amd64/vmm/intel/vmx.c > +++ b/sys/amd64/vmm/intel/vmx.c > @@ -436,14 +436,33 @@ > return (ENXIO); > } > > +/* > + * Bits in MSR_IA32_FEATURE_CONTROL register. > + * > + * XXX move these to machine/specialreg.h > + */ > +#define MSR_IA32_FEAT_CTL_LOCK 0x01 /* locks the > control reg */ > +#define MSR_IA32_FEAT_CTL_VMX_SMX_EN 0x02 /* enable VMXON in > SMX mode */ > +#define MSR_IA32_FEAT_CTL_VMX_EN 0x04 /* enable VMXON > (non-SMX) */ > /* > - * Verify that MSR_IA32_FEATURE_CONTROL lock and VMXON enable bits > - * are set (bits 0 and 2 respectively). > + * Verify that VMXON is allowed. > + * > + * According to Intel docs, we just need VMX_EN to be > + * set. If the LOCK bit is not set we can set or clear > + * VMX_EN ourselves. Once the LOCK bit is set no more > + * changes are possible without a processor reset. > + * > + * Existing BIOSes currently set-and-lock the feature, but > + * this code should work with BIOSes that don't lock it. > */ > feature_control = rdmsr(MSR_IA32_FEATURE_CONTROL); > - if ((feature_control & 0x5) != 0x5) { > - printf("vmx_init: VMX operation disabled by BIOS\n"); > - return (ENXIO); > + if ((feature_control & MSR_IA32_FEAT_CTL_VMX_EN) == 0) { > + if (feature_control & MSR_IA32_FEAT_CTL_LOCK) { > + printf("vmx_init: VMX operation disabled by > BIOS\n"); > + return (ENXIO); > + } > + feature_control |= MSR_IA32_FEAT_CTL_VMX_EN; > + wrmsr(MSR_IA32_FEATURE_CONTROL, feature_control); > } > > /* Check support for primary processor-based VM-execution controls > */ > > changeset: 1963:6cea2a2ed727 > tag: tip > user: Chris Torek > date: Fri Apr 05 19:14:05 2013 -0600 > files: sys/amd64/include/vmm.h sys/amd64/vmm/vmm.c > sys/amd64/vmm/vmm_dev.c > description: > prevent host OS crash when VMX is disabled > > If VMX is disabled in the BIOS, vmm_init correctly returns an > error, but this still leaves the module loaded and running, and a > later sysctl to create a virtual machine crashed the host. > Have vm_create() check, and return an error for this condition. > > > diff --git a/sys/amd64/include/vmm.h b/sys/amd64/include/vmm.h > --- a/sys/amd64/include/vmm.h > +++ b/sys/amd64/include/vmm.h > @@ -87,7 +87,7 @@ > extern struct vmm_ops vmm_ops_intel; > extern struct vmm_ops vmm_ops_amd; > > -struct vm *vm_create(const char *name); > +int vm_create(const char *name, struct vm **retval); > void vm_destroy(struct vm *vm); > const char *vm_name(struct vm *vm); > int vm_malloc(struct vm *vm, vm_paddr_t gpa, size_t len); > diff --git a/sys/amd64/vmm/vmm.c b/sys/amd64/vmm/vmm.c > --- a/sys/amd64/vmm/vmm.c > +++ b/sys/amd64/vmm/vmm.c > @@ -213,6 +213,15 @@ > vmmdev_init(); > iommu_init(); > error = vmm_init(); > + if (error) { > + /* > + * Returning an error here does not force > + * the module to be unloaded, so we have > + * to make sure that vm_create()s will not > + * proceed. > + */ > + ops = NULL; > + } > break; > case MOD_UNLOAD: > error = vmmdev_cleanup(); > @@ -249,8 +258,8 @@ > > SYSCTL_NODE(_hw, OID_AUTO, vmm, CTLFLAG_RW, NULL, NULL); > > -struct vm * > -vm_create(const char *name) > +int > +vm_create(const char *name, struct vm **retval) > { > int i; > struct vm *vm; > @@ -258,8 +267,11 @@ > > const int BSP = 0; > > + if (ops == NULL) > + return (ENXIO); > + > if (name == NULL || strlen(name) >= VM_MAX_NAMELEN) > - return (NULL); > + return (EINVAL); > > vm = malloc(sizeof(struct vm), M_VM, M_WAITOK | M_ZERO); > strcpy(vm->name, name); > @@ -274,7 +286,8 @@ > vm->iommu = iommu_create_domain(maxaddr); > vm_activate_cpu(vm, BSP); > > - return (vm); > + *retval = vm; > + return (0); > } > > static void > diff --git a/sys/amd64/vmm/vmm_dev.c b/sys/amd64/vmm/vmm_dev.c > --- a/sys/amd64/vmm/vmm_dev.c > +++ b/sys/amd64/vmm/vmm_dev.c > @@ -475,9 +475,9 @@ > if (sc != NULL) > return (EEXIST); > > - vm = vm_create(buf); > - if (vm == NULL) > - return (EINVAL); > + error = vm_create(buf, &vm); > + if (error) > + return (error); > > sc = malloc(sizeof(struct vmmdev_softc), M_VMMDEV, M_WAITOK | > M_ZERO); > sc->vm = vm; > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 12 20:12:28 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 547AF5A7 for ; Fri, 12 Apr 2013 20:12:28 +0000 (UTC) (envelope-from nicolas@critsend.com) Received: from sender26.critsend.com (sender26.critsend.com [109.236.85.22]) by mx1.freebsd.org (Postfix) with ESMTP id DF0C31B88 for ; Fri, 12 Apr 2013 20:12:27 +0000 (UTC) Received: from sender26.critsend.com (localhost [127.0.0.1]) by sender26.critsend.com (Postfix) with ESMTP id 9A80E11655F for ; Fri, 12 Apr 2013 19:37:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=viadeo.com; i=@viadeo.com; q=dns/txt; s=critsend2; t=1365795430; h=Content-Type : MIME-Version : To : From : Subject : Reply-To : Precedence : Date : Message-ID : List-Unsubscribe; bh=/SgkiEiU5K1MdSvq9avaYMtk+0L1+warpwqem3XyVE8=; b=lMW7loekOHFr+/k6TNlfu8ZOiRhm/2KhqdicZKLVOltz2/GXojfhEStip8Vke9RMcu7wXR71SbphEotB8ugyAapRkmPiM6SmPbnlttngWZYRthEOr5NJseVgzACUsurFDTQvzxRhpXQRe9SPr4VlYVsw7fFbJQ6+93LUwSHu6Bg= MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org From: Shahzad Ahmad Subject: Message from Shahzad Ahmad Precedence: bulk Date: Fri, 12 Apr 2013 19:37:10 +0000 Message-ID: <4+paav3tcbbkacaeca2ebusuuzizzrsmm5mtabirtt2hu2gtmls37s37qeoayoexwd2ce5xzis3wxfdsm7zpbbcqgmunec4r4llfko5nutbnmaakxgqa6bx2mz3in5ffytmbjenl43l3kcr5pslm6vfpbhpa======+629882@critsend.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Reply-To: no-reply@viadeo.com List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2013 20:12:28 -0000 viadeo Invitation Hello, I would like to invite you to join my network. Kind regards, Shahzad Ahmad Store keer, grand hoteltoronto canada Confirm that you know Shahzad Message from Shahzad Ahmad If you no longer want to receive invitations to join Viadeo: Unsubscribe From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 13 20:19:15 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id ECCA7D0F for ; Sat, 13 Apr 2013 20:19:15 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 7AD12127C for ; Sat, 13 Apr 2013 20:19:15 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id fo12so3336810lab.24 for ; Sat, 13 Apr 2013 13:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:date:x-google-sender-auth:message-id :subject:from:to:content-type; bh=ntA3fR15ZDj62Y1kO4RUMu38GQ/+wgRHWDPda25o40o=; b=Oriue5y7wa/vVYFUr3xE5FOeDMgekJdzniognq3oc/fp8pcbsQX9NjMxpvwclCmD8n DuYQdECNEk4/zfqO2euIW0HTNbGDDlgoRXwz8vU4vaMKYPWNo1KMiHoB6fwJni5r+kbT GIko2gbxHu550fcD1cxemcS/MKPI+5hPK3+k4hkzGdfYvBajPE72cfsDo2gjJpyZv4xq PU4sb9ZpE+TNJcyyeXEwncyBRGfssRJ9niOVAbGFu1beaob/QZAxYkdw+eqoNbphgVD+ lyCHjlEozTkKw+eKlq7aVqqyA93Ii43vBeidOjwE19k5FKcBQYRYiKzcZJ3N84xLextm kMHA== MIME-Version: 1.0 X-Received: by 10.112.76.39 with SMTP id h7mr7544576lbw.118.1365884353767; Sat, 13 Apr 2013 13:19:13 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.146.135 with HTTP; Sat, 13 Apr 2013 13:19:13 -0700 (PDT) Date: Sat, 13 Apr 2013 13:19:13 -0700 X-Google-Sender-Auth: w-bIhV09Ig7C1LBWAHxTVFr_Y0Q Message-ID: Subject: AMD and BHyve? From: Craig Rodrigues To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Apr 2013 20:19:16 -0000 Hi, I recently updated one of my home machines to FreeBSD-10/amd64. Here is what I have: FreeBSD 10.0-CURRENT #0 r249181: Sat Apr 6 03:07:32 UTC 2013 rodrigc@dibbler.crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 WARNING: WITNESS option enabled, expect reduced performance. CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ (2511.48-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x60fb1 Family = 0xf Model = 0x6b Stepping = 1 Features=0x178bfbff Features2=0x2001 AMD Features=0xea500800 AMD Features2=0x11f Will this CPU work with BHyve when the AMD support is fully committed? Right now when I try to load vmm.ko I get: kldload vmm.ko amd_iommu_init: not implemented amdv_init: not implemented amdv_cleanup: not implemented module_register_init: MOD_LOAD (vmm, 0xffffffff81c893b0, 0) error 6 Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 13 21:44:01 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5E9E965C for ; Sat, 13 Apr 2013 21:44:01 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id B5173155F for ; Sat, 13 Apr 2013 21:44:00 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id k13so3701809wgh.29 for ; Sat, 13 Apr 2013 14:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=/BiBSVTEqB8uR/e/KPNE+CrD+MOaXs8SsdsWb/XWynQ=; b=fKJKHItwuWG+DWLpgTsFXgtbLleLpiw14YSXuOy4QWsYu/EY60KUu1oYfqBf6mTtQq l0+4UhRkBkkBKiCUdMgohX4YH66mkMPBndnvYE9CUO84LsjYHVjhLO8iboU5M9TE5zxG GDRjI/xoer/leF6jo1YVC9M7Czk19QUke3uAFuBLCpau8yU2O4yTC0E5ZubOVbq1s8ZM Tz2Cp6wxvR1Umdakb2tBhjgYSeYS+NLodXWCq7I3eSVHpcYYhRdcCJ+kgSp9ZEPIcOvH HFvZBXVRNY8r1Xin02Az6YKcamYSFah9QgsFWHTvKOGTUE1OF+bbRXzVHURUBuHOea/3 RMAA== MIME-Version: 1.0 X-Received: by 10.194.220.37 with SMTP id pt5mr24285577wjc.16.1365889434332; Sat, 13 Apr 2013 14:43:54 -0700 (PDT) Received: by 10.194.89.9 with HTTP; Sat, 13 Apr 2013 14:43:54 -0700 (PDT) In-Reply-To: References: <0A562DF1-BC1A-4416-8471-B27BC5E964EF@gmail.com> Date: Sat, 13 Apr 2013 14:43:54 -0700 Message-ID: Subject: Re: FW: AMD and BHyve? From: Anish To: freebsd-virtualization@freebsd.org, Anish Gupta Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Apr 2013 21:44:01 -0000 Hi Craig, >AMD Features=0xea500800 AMD Features2=0x11f Yes, the AMD processor you have support AMD-V or SVM and I got BHyVe working on AMD processors with SVM Nested Page Table/RVI support, equivalent of Intel Extended Page Table. Oct 30 04:46:53 phenom kernel: CPU: AMD Phenom(tm) II X4 965 Processor (3411.44-MHz K8-class CPU) Oct 30 04:46:53 phenom kernel: Origin = "AuthenticAMD" Id = 0x100f43 Family = 10 Model = 4 Stepping = 3 Oct 30 04:46:53 phenom kernel: Features=0x178bfbff Oct 30 04:46:53 phenom kernel: Features2=0x802009 Oct 30 04:46:53 phenom kernel: AMD Features=0xee500800 Oct 30 04:46:53 phenom kernel: AMD Features2=0x37ff I am not sure if "AMD Athlon(tm) 64 X2 " support SVM NPT/RVI. Some of earlier version SVM processor didn't support RVI. Neel and Peter have created projects/bhyve_svm for submitting AMD patches for BHyVe. It doesn't have complete patches yet, only vmcb related files. Here is some relevant log: vmm.ko related: --------------------- ..... pci0:6:0:0: reprobing on driver added pci7: driver added amd_iommu_init: not implemented vmm_ipi_init: installing ipi handler to interrupt vcpus at vector 254 SVM rev: 0x1 NASID:0x40 SVM Features:0xf AMD guest booting recent version of current: ------------------------- GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb SMAP type=01 base=0000000000000000 len=00000000000a0000 SMAP type=01 base=0000000000100000 len=000000001ff00000 Table 'APIC' at 0xf0500 APIC: Found table at 0xf0500 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 1 ACPI ID 1: enabled SMP: Added CPU 1 (AP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #2 r248677M: Sun Mar 24 19:27:46 UTC 2013 root@phenom:/usr/obj/mnt/head/sys/GENERIC amd64 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 WARNING: WITNESS option enabled, expect reduced performance. Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff8183f000. Hypervisor: Origin = "BHyVBHyVBHyV" Calibrating TSC clock ... TSC clock: 3598480367 Hz CPU: AMD Phenom(tm) II X4 965 Processor (3598.48-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f43 Family = 0x10 Model = 0x4 Stepping = 3 Features=0x783ab7f Features2=0x80a02001 AMD Features=0xee505880<,SYSCALL,,,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!> AMD Features2=0x37ff TSC: P-state invariant L1 2MB data TLB: 0 entries, 128-way associative L1 2MB instruction TLB: 5 entries, 0-way associative L1 4KB data TLB: 182 entries, 215-way associative L1 4KB instruction TLB: 219 entries, 52-way associative L1 data cache: 0 kbytes, 128 bytes/line, 0 lines/tag, 0-way associative L1 instruction cache: 0 kbytes, 253 bytes/line, 3 lines/tag, 0-way associative L2 2MB data TLB: 128 entries, 2-way associative L2 2MB instruction TLB: 0 entries, 2-way associative L2 4KB data TLB: 512 entries, 4-way associative L2 4KB instruction TLB: 512 entries, 4-way associative L2 unified cache: 512 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative real memory = 536870912 (512 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x0000000001864000 - 0x000000001f29ffff, 497270784 bytes (121404 pages) avail memory = 472559616 (450 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: INTR: Adding local APIC 1 as a target FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 x86bios: IVT 0x000000-0x0004ff at 0xfffffe0000000000 x86bios: SSEG 0x001000-0x001fff at 0xffffff8000229000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000 APIC: CPU 0 has ACPI ID 0 APIC: CPU 1 has ACPI ID 1 random device not loaded; using insecure entropy ULE: setup cpu 0 ULE: setup cpu 1 ACPI: RSDP 0xf0400 00024 (v02 BHYVE ) ACPI: XSDT 0xf0480 00034 (v01 BHYVE BVXSDT 00000001 INTL 20130214) ACPI: APIC 0xf0500 00052 (v01 BHYVE BVMADT 00000001 INTL 20130214) ACPI: FACP 0xf0600 0010C (v05 BHYVE BVFACP 00000001 INTL 20130214) ACPI: DSDT 0xf0800 000F2 (v02 BHYVE BVDSDT 00000001 INTL 20130214) ACPI: FACS 0xf0780 00040 MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 ioapic0: Changing APIC ID to 2 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0: intpin 9 polarity: low ioapic0 irqs 0-16 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 wlan: <802.11 Link Layer> snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 nfslock: pseudo-device null: random: module_register_init: MOD_LOAD (vesa, 0xffffffff80c39920, 0) error 19 io: kbd: new array size 4 kbd1 at kbdmux0 mem: hpt27xx: RocketRAID 27xx controller driver v1.0 hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2 acpi0: on motherboard ACPI: All ACPI Tables successfully acquired ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 Event timer "RTC" frequency 32768 Hz quality 0 ACPI timer: 1/45 1/18 1/17 1/17 1/17 1/16 1/17 1/16 1/17 1/17 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: could not get PCI interrupt routing table for \_SB_.PCI0 - AE_NOT_FOUND pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x1275, dev=0x1275, revid=0x00 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1af4, dev=0x1000, revid=0x00 domain=0, bus=0, slot=1, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) MSI-X supports 3 messages in map 0x14 map[10]: type I/O Port, range 32, base 0x2000, size 5, enabled pcib0: allocated type 4 (0x2000-0x201f) for rid 10 of pci0:0:1:0 map[14]: type Memory, range 32, base 0xc0000000, size 13, enabled found-> vendor=0x1af4, dev=0x1001, revid=0x00 domain=0, bus=0, slot=2, func=0 class=01-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) MSI-X supports 2 messages in map 0x14 map[10]: type I/O Port, range 32, base 0x2040, size 6, enabled pcib0: allocated type 4 (0x2040-0x207f) for rid 10 of pci0:0:2:0 map[14]: type Memory, range 32, base 0xc0002000, size 13, enabled found-> vendor=0x131f, dev=0x2000, revid=0x00 domain=0, bus=0, slot=31, func=0 class=07-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=4 map[10]: type I/O Port, range 32, base 0x3f8, size 3, enabled pcib0: allocated type 4 (0x3f8-0x3ff) for rid 10 of pci0:0:31:0 pcib0: no PRT entry for 0.31.INTA virtio_pci0: port 0x2000-0x201f mem 0xc0000000-0xc0001fff at device 1.0 on pci0 vtnet0: on virtio_pci0 virtio_pci0: host features: 0x18020 virtio_pci0: negotiated features: 0x18020 virtio_pci0: virtqueue 1 (vtnet0 transmit) requested indirect descriptors but not negotiated vtnet0: bpf attached vtnet0: Ethernet address: 00:a0:98:a8:28:2f virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported) msi: routing MSI-X IRQ 256 to local APIC 0 vector 50 msi: routing MSI-X IRQ 257 to local APIC 0 vector 51 msi: routing MSI-X IRQ 258 to local APIC 0 vector 52 virtio_pci0: using IRQs 256-258 for MSI-X virtio_pci0: using per VQ MSIX interrupts virtio_pci1: port 0x2040-0x207f mem 0xc0002000-0xc0003fff at device 2.0 on pci0 vtblk0: on virtio_pci1 virtio_pci1: host features: 0x10000004 virtio_pci1: negotiated features: 0x10000004 virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) msi: routing MSI-X IRQ 259 to local APIC 0 vector 53 msi: routing MSI-X IRQ 260 to local APIC 0 vector 54 virtio_pci1: using IRQs 259-260 for MSI-X virtio_pci1: using per VQ MSIX interrupts vtblk0: 16384MB (33554432 512 byte sectors) uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 55 uart2: fast interrupt uart2: console (9600,n,8,1) acpi0: wakeup code va 0xffffff80002b4000 pa 0x4000 Device configuration finished. procfs registered lapic: Divisor 2, Frequency 1797863662 Hz Timecounters tick every 10.000 msec vlan: initialized, using hash tables with chaining tcp_init: net.inet.tcp.tcbhashsize auto tuned to 4096 lo0: bpf attached hpt27xx: no controller detected. hptrr: no controller detected. GEOM: new disk vtbd0 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x00000011 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 1 vector 48 msi: Assigning MSI-X IRQ 257 to local APIC 1 vector 49 msi: Assigning MSI-X IRQ 259 to local APIC 1 vector 50 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1799240183 Hz quality -100 WARNING: WITNESS option enabled, expect reduced performance. g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT Trying to mount root from ufs:/dev/gpt/gprootfs [rw]... WARNING: / was not properly dismounted start_init: trying /sbin/init debug.witness.watch: 1 -> 0 Setting hostuuid: 6b60e354-82db-11e2-8af6-00a0980cc175. Setting hostid: 0x085ad664. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/gpt/gprootfs: 27125 files, 241278 used, 3816876 free (3372 frags, 476688 blocks, 0.1% fragmentation) g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT Mounting local file systems:. debug.witness.watch: 0 -> 0 Writing entropy file:. Setting hostname: bhyve-svm. Starting Network: lo0 vtnet0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 vtnet0: flags=8902 metric 0 mtu 1500 options=80028 ether 00:a0:98:a8:28:2f nd6 options=29 media: Ethernet 1000baseT status: active debug.witness.watch: 0 -> 0 Starting devd. debug.witness.watch: 0 -> 0 debug.witness.watch: 0 -> 0 Starting Network: vtnet0. vtnet0: flags=8902 metric 0 mtu 1500 options=80028 ether 00:a0:98:a8:28:2f nd6 options=29 media: Ethernet 1000baseT status: active debug.witness.watch: 0 -> 0 debug.witness.watch: 0 -> 0 debug.witness.watch: 0 -> 0 debug.witness.watch: 0 -> 0 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 debug.witness.watch: 0 -> 0 Creating and/or trimming log files. Starting syslogd. realpath: /dev/dumpdev: No such file or directory /etc/rc: WARNING: Dump device does no exist. Savecore not run. ELF ldconfig path: /lib /usr/lib /usr/lib/compat 32-bit compatibility ldconfig path: /usr/lib32 Clearing /tmp (X related). Updating motd:. Performing sanity check on sshd configuration. Starting sshd. Starting cron. debug.witness.watch: 0 -> 0 Starting background file system checks in 60 seconds. Sat Apr 13 21:35:56 UTC 2013 Apr 13 21:35:56 bhyve-svm getty[749]: open /dev/ttyu0: No such file or directory Apr 13 21:35:56 bhyve-svm getty[ FreeBSD/amd64 (bhyve-svm) (ttyu2) login: ... root@bhyve-svm:~ # uname -a FreeBSD bhyve-svm 10.0-CURRENT FreeBSD 10.0-CURRENT #2 r248677M: Sun Mar 24 19:27:46 UTC 2013 root@phenom:/usr/obj/mnt/head/sys/GENERIC amd64 root@bhyve-svm:~ # *From:* Craig Rodrigues > *Date:* April 13, 2013, 1:19:13 PM PDT > *To:* freebsd-virtualization@freebsd.org > *Subject:* *AMD and BHyve?* > > Hi, > > I recently updated one of my home machines > to FreeBSD-10/amd64. Here is what I have: > > FreeBSD 10.0-CURRENT #0 r249181: Sat Apr 6 03:07:32 UTC 2013 > rodrigc@dibbler.crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC > amd64 > FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 > WARNING: WITNESS option enabled, expect reduced performance. > CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ (2511.48-MHz K8-class > CPU) > Origin = "AuthenticAMD" Id = 0x60fb1 Family = 0xf Model = 0x6b > Stepping = > 1 > > > Features=0x178bfbff MOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> > Features2=0x2001 > AMD Features=0xea500800 > AMD Features2=0x11f > > > > Will this CPU work with BHyve when the AMD support > is fully committed? Right now when I try to load vmm.ko > I get: > > kldload vmm.ko > amd_iommu_init: not implemented > amdv_init: not implemented > amdv_cleanup: not implemented > module_register_init: MOD_LOAD (vmm, 0xffffffff81c893b0, 0) error 6 > > Thanks. > > -- > Craig > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > >