From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 00:16:50 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1218B381; Sun, 23 Feb 2014 00:16:50 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BD0681B39; Sun, 23 Feb 2014 00:16:49 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id EA9231534C0; Sun, 23 Feb 2014 01:16:47 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xXnYJol88AFR; Sun, 23 Feb 2014 01:16:45 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id A98EA153448; Sun, 23 Feb 2014 01:16:45 +0100 (CET) Message-ID: <53093DF3.2090505@digiware.nl> Date: Sun, 23 Feb 2014 01:16:51 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Bhyve and booting a ZFS-on-root system References: <53090599.5000505@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 00:16:50 -0000 On 22-2-2014 22:43, Craig Rodrigues wrote: > > > > On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen > wrote: > > > CPU: AMD Phenom(tm) II X6 1075T Processor (3013.84-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0x100fa0 Family = 0x10 Model = 0xa > > > Also, as Aryeh has pointed out, > you cannot run BHyve on an AMD processor unless you check out the Aryeh was the one that barged into my thread asking for the most recent snapshot of code for AMD. Which Micheal did indeed answered with the URL. > code from a special branch. Michael Dexter has provided a snapshot > from the branch: > > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002227.html I am running the vmm AMD code.... > but that snapshot doesn't have the ZFS on Root change. So you are telling me that ZFS on Root is available in the intel-code or more up to date current?? --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 10:59:50 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADA9A31D; Sun, 23 Feb 2014 10:59:50 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 350B310F3; Sun, 23 Feb 2014 10:59:50 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 4BC431534C4; Sun, 23 Feb 2014 11:59:48 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FdoNnpkLe9wl; Sun, 23 Feb 2014 11:59:46 +0100 (CET) Received: from [192.168.10.120] (10G [192.168.10.120]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPS id 1F610153448; Sun, 23 Feb 2014 11:59:46 +0100 (CET) References: <53090599.5000505@digiware.nl> In-Reply-To: Mime-Version: 1.0 (1.0) Message-Id: <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> X-Mailer: iPad Mail (9B206) From: Willem Jan Withagen Subject: Re: Bhyve and booting a ZFS-on-root system Date: Sun, 23 Feb 2014 11:59:45 +0100 To: Craig Rodrigues Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 10:59:50 -0000 Op 22 feb. 2014 om 22:28 heeft Craig Rodrigues het vol= gende geschreven: > On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen wr= ote: > Hi, >=20 > Just for the fun of it, I tried my build zfs-system scripts in a bhyve-vm.= >=20 > I use the 10.0-RELEASE iso to get to a shell, config and interface and > download my script. Installing does work, and on a regular system we can > go and boot into a ZFS-on-Root system. >=20 > ZFS on Root inside a BHyve VM was not working until > today. If you you have a FreeBSD-CURRENT system and then update > to this revision: > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html >=20 > then you can try it out. I'm running the amdsrc tree, so just upgrading current will probably not wor= k. Peter suggested that he would upgrade the amdsrc-tree shortly. so I'll wait just for that. I'm also looking into running a Ubuntu vm. i want to test/use zoneminder, an= d all my FreeBSD attempts run into trouble. So I'm also still looking for grub-bhyve..... --WjW=20= From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 14:46:24 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B02CED78 for ; Sun, 23 Feb 2014 14:46:24 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6B5731308 for ; Sun, 23 Feb 2014 14:46:20 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0F0C41534ED for ; Sun, 23 Feb 2014 15:46:19 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JUXd7299qMVg; Sun, 23 Feb 2014 15:46:16 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 8CDE6153448 for ; Sun, 23 Feb 2014 15:46:16 +0100 (CET) Message-ID: <530A09B7.2030505@digiware.nl> Date: Sun, 23 Feb 2014 15:46:15 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "freebsd-virtualization@freebsd.org" Subject: bhyve FreeBSD 10.0 VM crashed Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 14:46:24 -0000 Hi, I've kicked off 3 2G vm's which do a continous while(1) { make -j 4 buidlworld } And one of those VM's just crashed with the message below. Observation: The VM is clearly not able to recover from this. And probably needs to be destroyed and restarted... Don't know it it got as far as vmrun.sh restarting it. Question: 1) I expect this to be due to 3 VM's running in parallel with quite some load. But is this something that needs to be reported to stable@? Or does this belong here on this list. 2) Killing the VM and restarting probably also kills the memory but not the swap? And thus would allow to extract a core? I'll see when I reboot. --WjW ==== spin lock 0xffffffff814fa030 (smp rendezvous) held by 0xfffff80079e8d490 (tid 100097) too long timeout stopping cpus panic: spin lock held too long cpuid = 3 KDB: stack backtrace: #0 0xffffffff808e7dd0 at kdb_backtrace+0x60 #1 0xffffffff808af8b5 at panic+0x155 #2 0xffffffff8089cb71 at _mtx_lock_spin_cookie+0x241 #3 0xffffffff80c7ec7a at smp_tlb_shootdown+0x8a #4 0xffffffff80c8066c at pmap_invalidate_range+0x2fc #5 0xffffffff80932f6d at allocbuf+0x55d #6 0xffffffff809317cf at brelse+0x14f #7 0xffffffff80934b18 at bufdone+0x78 #8 0xffffffff809346c2 at biodone+0xe2 #9 0xffffffff8081c561 at g_io_schedule_up+0x1b1 #10 0xffffffff8081ca5d at g_up_procbody+0x6d #11 0xffffffff8088198a at fork_exit+0x9a #12 0xffffffff80c758ce at fork_trampoline+0xe Uptime: 1h21m3s Dumping 183 out of 2025 MB:..9%..18%..27%..35%..44%..53%..62%..79%..88%..97% Dump complete Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... cpu_reset: Stopping other CPUs timeout stopping cpus cpu_reset: Restarting BSP cpu_reset: Failed to restart BSP ==== From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 14:57:59 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEA78EE5 for ; Sun, 23 Feb 2014 14:57:59 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 411AD13A9 for ; Sun, 23 Feb 2014 14:57:59 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id pv20so4493792lab.30 for ; Sun, 23 Feb 2014 06:57:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=NznDEL5fDJpxrmr5atb99/0tM1i3TqEWbLYpuoSv5No=; b=H0qk5AwppVrloll9ggs3a8VYjG4SlnRuiIXuB4VAwqK7QZZBQ8kahE8sQpEFdwRZcA thxMH7gYQsZFa6ypawUygyfW2AyEIZ0fqkhNv4IVbZd1pwhQjo2LmWCfgXCmbQEFREwM TMJGd0SzJVXVnD03v2a9pDVrhD6owxgV2T7WisfGyZoBXv0P/wVHgsH0ilAjxrcWx0Du gR9BK16s4AICjZQM2OOa439LlizGbXZqF/aKSAmcq7buqK+fc6989aAIzTGotw718Bcm HSoPBRWf0A4oTBuRE+sRxKUQiDHIaqhZBuMgZBE5xDOpF49rFjzXQCNy+PJTCMlvTnyy cFHg== MIME-Version: 1.0 X-Received: by 10.152.27.193 with SMTP id v1mr9705625lag.4.1393167477264; Sun, 23 Feb 2014 06:57:57 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sun, 23 Feb 2014 06:57:57 -0800 (PST) Received: by 10.112.30.211 with HTTP; Sun, 23 Feb 2014 06:57:57 -0800 (PST) In-Reply-To: <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> Date: Sun, 23 Feb 2014 06:57:57 -0800 X-Google-Sender-Auth: nJx2Bma83PiJJiseGYlpc3Ti48A Message-ID: Subject: Re: Bhyve and booting a ZFS-on-root system From: Craig Rodrigues To: Willem Jan Withagen Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 14:57:59 -0000 On Feb 23, 2014 2:59 AM, "Willem Jan Withagen" wrote: > > > > Op 22 feb. 2014 om 22:28 heeft Craig Rodrigues het volgende geschreven: > >> On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen wrote: >>> >>> Hi, >>> >>> Just for the fun of it, I tried my build zfs-system scripts in a bhyve-vm. >>> >>> I use the 10.0-RELEASE iso to get to a shell, config and interface and >>> download my script. Installing does work, and on a regular system we can >>> go and boot into a ZFS-on-Root system. >> >> >> ZFS on Root inside a BHyve VM was not working until >> today. If you you have a FreeBSD-CURRENT system and then update >> to this revision: >> http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html >> >> then you can try it out. > > > I'm running the amdsrc tree, so just upgrading current will probably not work. > Peter suggested that he would upgrade the amdsrc-tree shortly. > so I'll wait just for that. > > I'm also looking into running a Ubuntu vm. i want to test/use zoneminder, and all my FreeBSD attempts run into trouble. > So I'm also still looking for grub-bhyve..... grub2-bhyve is in ports: http://www.freshports.org/sysutils/grub2-bhyve/ If you do a web search, you'll find several HOWTO articles for using it. Recently, Rudy wrote a nice one: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002305.html -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 15:57:59 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 105AB788 for ; Sun, 23 Feb 2014 15:57:59 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7DB3218D2 for ; Sun, 23 Feb 2014 15:57:57 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id s1NFvg2L079293 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 23 Feb 2014 16:57:47 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id s1NFvf10079290; Sun, 23 Feb 2014 16:57:41 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Sun, 23 Feb 2014 16:57:41 +0100 From: Paul Schenkeveld To: FreeBSD Virtualization Subject: Bhyve segfaults when accessing the network Message-ID: <20140223155741.GA75513@psconsult.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 15:57:59 -0000 Hi, I tried to get bhyve to run on my home server but got consistent crashes as soon as I touch the network. Hardware: Supermicro X9SPV-F3610ME-O (embedded i5) with 16GB RAM Host kernels (+userland of the same version): 10.0 RC5 10.0 RELEASE 10 STABLE The kernel has been stripped (unneeded drivers and stuff like DEBUG, AUDIT, CAPABILITIES, MAC removed) Guest operating systems: 9.2 RELEASE 10.0 RELEASE Procedure: # ifconfig tap0 create up # ifconfig bridge0 create up # ifconfig bridge0 addm tap0 addm em0 # kldload vmm # mkdir /vm/test # cd /vm/test # ln -s /images/FreeBSD-10.0-RELEASE-amd64-disc1.iso release.iso # sh vmrun.sh -c 1 -m 512 -t tap0 At the point where DHCP sends the first query I get a segmentation fault and the guest stops. I can provoke the same segfault by not putting the DHCP line into rc.conf and trying to ifconfig vtnet0 or by adding tap0 to bridge0 in the host after the guest has stopped. Although it worked perfectly fine with bhyve a couple of months ago (with an older build of 10.0-CURRENT that I don't have here anymore), I suspected the hardware and tried the same on my Macbook Pro with Vmware Fusion with a 10.0 RELEASE guest. The stripped 10 STABLE guest segfaults in the same way, the other two both work. I bought the home server for building software for older releases (mostly FreeBSD 8 and 9), my Macbook with Fusion is too slow and often in use for other purposes. Is this behaviour seen by others? I can provide more details or core dumps if that helps locating/solving the problem. With kind regards, Paul Schenkeveld From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:24:35 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B483095B for ; Sun, 23 Feb 2014 16:24:35 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6C9E81D9B for ; Sun, 23 Feb 2014 16:24:35 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id B351C153AA7; Sun, 23 Feb 2014 17:24:33 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QQJjERQudxkh; Sun, 23 Feb 2014 17:24:31 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b] (unknown [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 9A259153AA6; Sun, 23 Feb 2014 17:24:31 +0100 (CET) Message-ID: <530A20BE.4050602@digiware.nl> Date: Sun, 23 Feb 2014 17:24:30 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:24:35 -0000 On 23-2-2014 15:57, Craig Rodrigues wrote: >> I'm also looking into running a Ubuntu vm. i want to test/use > zoneminder, and all my FreeBSD attempts run into trouble. >> So I'm also still looking for grub-bhyve..... > > grub2-bhyve is in ports: http://www.freshports.org/sysutils/grub2-bhyve/ Yup, took me some time to realize that on this experimental box portsnap was not running nightly... > If you do a web search, you'll find several HOWTO articles for using it. > Recently, Rudy wrote a nice one: > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002305.html I did have that, from his announcement, But trying to follow that, I get: --- freetest# bhyvectl --destroy --vm=ubuntu freetest# grub-bhyve -r cd0 -m ./device.map -M 2048 ubuntu GNU GRUB version 2.00 > [I select to install Ubuntu Server] freetest# bhyve -c 2 -m 2048M -H -P -A \ ? -s 0:0,hostbridge \ ? -s 1:0,lpc -s 2:0,virtio-net,tap3 \ ? -s 3,ahci-cd,ubuntu-13.10.iso -l com1,stdio \ ? -s 4,virtio-blk,/dev/zvol/zfsroot/ubuntu ubuntu vm exit rdmsr 0xc0010015, cpu 0 --- And I've seen discussions in other threads about reading/writing cpu registers where some bits should be 0, but trap when being set.... (Or something close to this...???) So it could again be due to the amdsrc tree differences? --WjW And again this is running on my AMD-board. /dev/zvol/zfsroot/ubuntu is a 10G zvol tap3 is created and up and ./ubuntu-13.10.iso is the ubuntu server ISO suggested by Rudy. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:28:47 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DABCC9BE for ; Sun, 23 Feb 2014 16:28:47 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AB21B1DB4 for ; Sun, 23 Feb 2014 16:28:47 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so5506629pab.20 for ; Sun, 23 Feb 2014 08:28:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hl4xWd8pQcS7TEjKXUVwgRuyky/lGamZGslGO2z6uyA=; b=fCfe0ieAXGiVIQd2Wg2QtJsmUdZWc/f3YHtfleeLW8aUaboCZojXqqRuhKXDVbElLJ uEBPHH17k9a4/58Fsh/mmhaTt64Ql/CZ2R15yHSP5sdRfen8l9WeSWDBc33mkEQJlizz 66OPZUd4YPJqbxQDkcvvvhz/IkyXlt8+hHAXTnwG7xXGuiuKCoQN9X5lMARo7QeiN+uh 4Eiy7xmjVPR3KR1ff3BA0ysuZ6nqTjr/c40nUfCVEFagg5+Vf1zROQtedWsLLglCe8vF pz6whH/Phy9FbDbtBsGbW4CmmSxRjxHZxyoYthH5sxB0ehypmPurj6gxxVsie9vMdYdP 0C+Q== MIME-Version: 1.0 X-Received: by 10.66.149.37 with SMTP id tx5mr8518435pab.81.1393172927253; Sun, 23 Feb 2014 08:28:47 -0800 (PST) Received: by 10.68.155.38 with HTTP; Sun, 23 Feb 2014 08:28:46 -0800 (PST) In-Reply-To: References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> Date: Sun, 23 Feb 2014 11:28:46 -0500 Message-ID: Subject: Re: Bhyve and booting a ZFS-on-root system From: Aryeh Friedman To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:28:47 -0000 On Sun, Feb 23, 2014 at 9:57 AM, Craig Rodrigues wrote: > On Feb 23, 2014 2:59 AM, "Willem Jan Withagen" wrote: > > > > > > > > Op 22 feb. 2014 om 22:28 heeft Craig Rodrigues het > volgende geschreven: > > > >> On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen > wrote: > >>> > >>> Hi, > >>> > >>> Just for the fun of it, I tried my build zfs-system scripts in a > bhyve-vm. > >>> > >>> I use the 10.0-RELEASE iso to get to a shell, config and interface and > >>> download my script. Installing does work, and on a regular system we > can > >>> go and boot into a ZFS-on-Root system. > >> > >> > >> ZFS on Root inside a BHyve VM was not working until > >> today. If you you have a FreeBSD-CURRENT system and then update > >> to this revision: > >> > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html > >> > >> then you can try it out. > > > > > > I'm running the amdsrc tree, so just upgrading current will probably not > work. > > Peter suggested that he would upgrade the amdsrc-tree shortly. > > so I'll wait just for that. > > > > I'm also looking into running a Ubuntu vm. i want to test/use zoneminder, > and all my FreeBSD attempts run into trouble. > > So I'm also still looking for grub-bhyve..... > > grub2-bhyve is in ports: http://www.freshports.org/sysutils/grub2-bhyve/ > > If you do a web search, you'll find several HOWTO articles for using it. > Recently, Rudy wrote a nice one: > > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002305.html > Also keep in mind most automation at this point is nothing but generating scripts designed for immediate use or user modifiction... (don't start with our current ones because they are being debugged)... if you want a scattered but very detailed discussion on what is required for these scripts see http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/001979.html -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:35:19 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CF4EA62 for ; Sun, 23 Feb 2014 16:35:19 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2E5491E4F for ; Sun, 23 Feb 2014 16:35:19 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so5506535pad.14 for ; Sun, 23 Feb 2014 08:35:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=U+7724hJkmQLJ2Vox0IRDU7OC/t0B9iLYbChj93DDjA=; b=hHZ03dJzJpiJjC4E+VG//iGb5zyA7SPGYvoV+L2bj9iWQ6KGK8qD8NNuCCvkdAvJjf k84KfzwBeNcazh+avBWepF5jikoux/MyM4jp6Geek+4UnQ+oDQLrK3IASKx3Ifak3nmG qOSyv22iybr4sksDuCZYrTKpxgsrdnOqlO8X0JJpmIbo3h/N0WRkTew2CXmWWlUBWtHs Mpwync/Amno7IE81kwSu3pYMHttyoHzhGB5JFomGaLJeZhDjzYPf4vH6Lvj0v2OlthMY FLRTw3YfaMNircgfM8+Q3domy0vlAJdyzFrb+Y6jKLEsTjUirhmSuwvlQA2ALA3PCED8 fngA== MIME-Version: 1.0 X-Received: by 10.69.20.139 with SMTP id hc11mr20050556pbd.63.1393173318905; Sun, 23 Feb 2014 08:35:18 -0800 (PST) Received: by 10.68.155.38 with HTTP; Sun, 23 Feb 2014 08:35:18 -0800 (PST) In-Reply-To: <530A20BE.4050602@digiware.nl> References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> Date: Sun, 23 Feb 2014 11:35:18 -0500 Message-ID: Subject: Re: Bhyve and Ubuntu booting From: Aryeh Friedman To: Willem Jan Withagen Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" , Craig Rodrigues X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:35:19 -0000 > And I've seen discussions in other threads about reading/writing cpu > registers where some bits should be 0, but trap when being set.... > (Or something close to this...???) > > So it could again be due to the amdsrc tree differences? > > --WjW > > And again this is running on my AMD-board. > /dev/zvol/zfsroot/ubuntu is a 10G zvol > tap3 is created and up > and ./ubuntu-13.10.iso is the ubuntu server ISO suggested by Rudy. > 12.04.3 runs perfectly on the amd patch with no modification. are you setting up the device map dir correctly? -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:40:32 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79E25B0A for ; Sun, 23 Feb 2014 16:40:32 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 36B481F70 for ; Sun, 23 Feb 2014 16:40:32 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 859D812373; Mon, 24 Feb 2014 02:40:30 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA63826 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 02:40:29 +1000 Message-ID: <530A2475.6090608@freebsd.org> Date: Sun, 23 Feb 2014 08:40:21 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> In-Reply-To: <530A20BE.4050602@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:40:32 -0000 > vm exit rdmsr 0xc0010015, cpu 0 > --- > > And I've seen discussions in other threads about reading/writing cpu > registers where some bits should be 0, but trap when being set.... > (Or something close to this...???) > > So it could again be due to the amdsrc tree differences? It's Linux accessing different MSRs on AMD as opposed to Intel. There is an option to ignore unknown MSRs with bhyve, but that probably shouldn't be used on AMD systems until we've worked out if the MSRs need to be emulated or not. Hopefully that will be in the next updated :) later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:51:37 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 613D8C92; Sun, 23 Feb 2014 16:51:37 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1D03E10AB; Sun, 23 Feb 2014 16:51:37 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 78E14153AA7; Sun, 23 Feb 2014 17:51:35 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vD24xIKTidAl; Sun, 23 Feb 2014 17:51:33 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b] (unknown [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 694B21534C0; Sun, 23 Feb 2014 17:51:32 +0100 (CET) Message-ID: <530A2713.6000602@digiware.nl> Date: Sun, 23 Feb 2014 17:51:31 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> In-Reply-To: <530A2475.6090608@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:51:37 -0000 On 23-2-2014 17:40, Peter Grehan wrote: >> vm exit rdmsr 0xc0010015, cpu 0 >> --- >> >> And I've seen discussions in other threads about reading/writing cpu >> registers where some bits should be 0, but trap when being set.... >> (Or something close to this...???) >> >> So it could again be due to the amdsrc tree differences? > > It's Linux accessing different MSRs on AMD as opposed to Intel. There > is an option to ignore unknown MSRs with bhyve, but that probably > shouldn't be used on AMD systems until we've worked out if the MSRs need > to be emulated or not. Hopefully that will be in the next updated :) It's a toy box here so if anything needs to be tested, worked out. Just let me know. And I'm willing to run the risk of lost work, because using the ignore option bites me.... So if you want me to see how we fare, .. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 16:57:23 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36BEED21 for ; Sun, 23 Feb 2014 16:57:23 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id E52C310D3 for ; Sun, 23 Feb 2014 16:57:22 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 197D312221; Mon, 24 Feb 2014 02:57:21 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA64078 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 02:57:20 +1000 Message-ID: <530A2870.9080306@freebsd.org> Date: Sun, 23 Feb 2014 08:57:20 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> In-Reply-To: <530A2713.6000602@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 16:57:23 -0000 > It's a toy box here so if anything needs to be tested, worked out. > Just let me know. And I'm willing to run the risk of lost work, because > using the ignore option bites me.... You can try the "-w" option. Aryeh has reported that it works for him. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:00:42 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CA0EDBA for ; Sun, 23 Feb 2014 17:00:42 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4C94810EE for ; Sun, 23 Feb 2014 17:00:42 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 74E2312245; Mon, 24 Feb 2014 03:00:41 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA64307 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 03:00:40 +1000 Message-ID: <530A2938.40401@freebsd.org> Date: Sun, 23 Feb 2014 09:00:40 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: bhyve FreeBSD 10.0 VM crashed References: <530A09B7.2030505@digiware.nl> In-Reply-To: <530A09B7.2030505@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:00:42 -0000 > I've kicked off 3 2G vm's which do a continous > while(1) { make -j 4 buidlworld } How many vCPUs on each VM, and on the host ? Hypervisors in general don't do too well when there is a lot of CPU oversubscription. bhyve may be a bit worse than the crowd with this since we've not had a lot of experience in scheduler tweaking etc to try and mitigate this. Destroying the VM with bhyvectl will reclaim all resources. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:01:49 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97D9EE08; Sun, 23 Feb 2014 17:01:49 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 441E5115D; Sun, 23 Feb 2014 17:01:48 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 72BB9153AA8; Sun, 23 Feb 2014 18:01:47 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t2-6M5XToDTg; Sun, 23 Feb 2014 18:01:45 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b] (unknown [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id B1BD71534C0; Sun, 23 Feb 2014 18:01:45 +0100 (CET) Message-ID: <530A2978.7070403@digiware.nl> Date: Sun, 23 Feb 2014 18:01:44 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> In-Reply-To: <530A2870.9080306@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:01:49 -0000 On 23-2-2014 17:57, Peter Grehan wrote: >> It's a toy box here so if anything needs to be tested, worked out. >> Just let me know. And I'm willing to run the risk of lost work, because >> using the ignore option bites me.... > > You can try the "-w" option. Aryeh has reported that it works for him. -w does not exist on my bhyve bhyve: illegal option -- w --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:09:12 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 928F3EDC for ; Sun, 23 Feb 2014 17:09:12 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4F78411A7 for ; Sun, 23 Feb 2014 17:09:11 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D3270122F5; Mon, 24 Feb 2014 03:09:10 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA64500 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 03:09:10 +1000 Message-ID: <530A2B35.6090409@freebsd.org> Date: Sun, 23 Feb 2014 09:09:09 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> In-Reply-To: <530A2978.7070403@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:09:12 -0000 > -w does not exist on my bhyve > bhyve: illegal option -- w Hmmm, just checked and you're right. Looks like a mis-merge - this change went into CURRENT in early December, and the most recent sync for the SVM branch was from Jan 14 :( Now I'm wondering what else hasn't made it in :( later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:17:11 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DC5EF83; Sun, 23 Feb 2014 17:17:11 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0130D123A; Sun, 23 Feb 2014 17:17:10 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0FEFC1534EF; Sun, 23 Feb 2014 18:17:09 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UL1qmnWcUz5A; Sun, 23 Feb 2014 18:17:07 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b] (unknown [IPv6:2001:4cb8:3:1:3ce8:7381:241a:6e9b]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 0D16C1534ED; Sun, 23 Feb 2014 18:17:07 +0100 (CET) Message-ID: <530A2D11.707@digiware.nl> Date: Sun, 23 Feb 2014 18:17:05 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> In-Reply-To: <530A2B35.6090409@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:17:11 -0000 On 23-2-2014 18:09, Peter Grehan wrote: >> -w does not exist on my bhyve >> bhyve: illegal option -- w > > Hmmm, just checked and you're right. Looks like a mis-merge - this > change went into CURRENT in early December, and the most recent sync for > the SVM branch was from Jan 14 :( > > Now I'm wondering what else hasn't made it in :( Pervious time I tried diff between current and the SVM branch, but that was just a bit too much for me to sort out. If I feel up to it tonight, I'll go and digg some more, but big change it still is too much diff te really grasp. Against what current version did you merge/build the SVM branch? I'll either check that out, or try my subversion skills to see if that reduces the diff. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:20:11 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C99DBFCC for ; Sun, 23 Feb 2014 17:20:11 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8A9F11241 for ; Sun, 23 Feb 2014 17:20:11 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 1723011DC1; Mon, 24 Feb 2014 03:20:10 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA64710 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 03:20:08 +1000 Message-ID: <530A2DC8.6000107@freebsd.org> Date: Sun, 23 Feb 2014 09:20:08 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Paul Schenkeveld Subject: Re: Bhyve segfaults when accessing the network References: <20140223155741.GA75513@psconsult.nl> In-Reply-To: <20140223155741.GA75513@psconsult.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:20:11 -0000 Hi Paul, > Hardware: Supermicro X9SPV-F3610ME-O (embedded i5) with 16GB RAM > > Host kernels (+userland of the same version): > 10.0 RC5 > 10.0 RELEASE > 10 STABLE Does the segfault happen on h/w with all of these ? (I think you mention below that it's only 10-stable that has the issue with Fusion). > The kernel has been stripped (unneeded drivers and stuff like DEBUG, > AUDIT, CAPABILITIES, MAC removed) Can you post the kernel config you are using ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 17:26:48 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CFA219B for ; Sun, 23 Feb 2014 17:26:48 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id DCDD612C4 for ; Sun, 23 Feb 2014 17:26:47 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 7BF7E12305; Mon, 24 Feb 2014 03:26:46 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA64813 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 03:26:45 +1000 Message-ID: <530A2F55.8090307@freebsd.org> Date: Sun, 23 Feb 2014 09:26:45 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> <530A2D11.707@digiware.nl> In-Reply-To: <530A2D11.707@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:26:48 -0000 > Against what current version did you merge/build the SVM branch? > I'll either check that out, or try my subversion skills to see if that > reduces the diff. My mistake - the sync was a r259205, which was from Dec 10 and predates the -w option (r259635, Dec 17). later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 19:03:50 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16FD9309 for ; Sun, 23 Feb 2014 19:03:50 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C0ADA1ABB for ; Sun, 23 Feb 2014 19:03:49 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0CBEF153AA6; Sun, 23 Feb 2014 20:03:48 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hyPa_4CJM4Yg; Sun, 23 Feb 2014 20:03:46 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:8033:fb68:f361:a8fd] (unknown [IPv6:2001:4cb8:3:1:8033:fb68:f361:a8fd]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 6E7A61534EF; Sun, 23 Feb 2014 20:03:46 +0100 (CET) Message-ID: <530A4610.9030607@digiware.nl> Date: Sun, 23 Feb 2014 20:03:44 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Aryeh Friedman Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" , Craig Rodrigues X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 19:03:50 -0000 On 23-2-2014 17:35, Aryeh Friedman wrote: > > And I've seen discussions in other threads about reading/writing cpu > registers where some bits should be 0, but trap when being set.... > (Or something close to this...???) > > So it could again be due to the amdsrc tree differences? > > --WjW > > And again this is running on my AMD-board. > /dev/zvol/zfsroot/ubuntu is a 10G zvol > tap3 is created and up > and ./ubuntu-13.10.iso is the ubuntu server ISO suggested by Rudy. > > > 12.04.3 runs perfectly on the amd patch with no modification. are you > setting up the device map dir correctly? I've verbosely copied the scripts that Rudy has online. (Apart from using other names, and will review them again) Right I'll download that ubuntu one as well. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 19:24:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8853B19 for ; Sun, 23 Feb 2014 19:24:25 +0000 (UTC) Received: from mail-pb0-f46.google.com (mail-pb0-f46.google.com [209.85.160.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A67011C51 for ; Sun, 23 Feb 2014 19:24:25 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id um1so5572061pbc.19 for ; Sun, 23 Feb 2014 11:24:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=xAeyG7SaI2wGPFieU10rnzgnXeF+IEWrGqOSCC7npq0=; b=Jb0kIOtm7YFDx2In3TPjjJ/unK4xbQzc9b48ogaisSJ/ancEUgFdev87tTo7NHGcFh 8XLuiFty1hJulWrrCtkUw+MltsKbKEx2MaGG5EI9WSsiKprtoIGz8UOb3EDnwYW7HTiU sKGfr4wliXljHqvVAiyIYy5wkK2HJlH/PPeAf4/fvR32mBG4qUEmQzqY+4QD8PwJjX4K AjtmRER6OTmZCsspZDLz39JQ2YLqdY0ltveIn4UWxn5n2nX0DNdYd3mhz7wdSm8zKSE6 FIelCOUcJ8y2mnYs4o7zmBkGwB15sLH8dtL63zh6A+wgV5mQOBcoYV3iJtrG7xfH6QCt /JEQ== X-Gm-Message-State: ALoCoQmZvYEgo6s/Yxxk+X3b+/oDcFM9NiRuobAlHZt9ZX22TsZWAjsCnhTnDpZYJuMvGNDPAK87 X-Received: by 10.68.132.233 with SMTP id ox9mr20776504pbb.138.1393183458970; Sun, 23 Feb 2014 11:24:18 -0800 (PST) Received: from Michaels-MacBook-Pro.local (c-98-246-202-204.hsd1.or.comcast.net. [98.246.202.204]) by mx.google.com with ESMTPSA id sy2sm42558834pbc.28.2014.02.23.11.24.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 23 Feb 2014 11:24:18 -0800 (PST) Message-ID: <530A4AE1.7080703@callfortesting.org> Date: Sun, 23 Feb 2014 11:24:17 -0800 From: Michael Dexter User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: Bhyve and booting a ZFS-on-root system References: <53090599.5000505@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 19:24:25 -0000 On 2/22/14 1:28 PM, Craig Rodrigues wrote: > ZFS on Root inside a BHyve VM was not working until > today. If you you have a FreeBSD-CURRENT system and then update > to this revision: > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html Only two items are needed for booting ZFS VM's: /boot/userboot.so and /usr/sbin/bhyveload I posted these binaries and the patch for testing and you may find them useful in the interim: http://bhyve.org/zfsboot.tar While I have not tried a ZFS VM on AMD, it will load a UFS2 one. Michael From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 20:33:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE6C4313; Sun, 23 Feb 2014 20:33:58 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C211012E1; Sun, 23 Feb 2014 20:33:57 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id s1NKXmcx076051 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 23 Feb 2014 21:33:53 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id s1NKXmYA076050; Sun, 23 Feb 2014 21:33:48 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Sun, 23 Feb 2014 21:33:48 +0100 From: Paul Schenkeveld To: freebsd-virtualization@freebsd.org Subject: Re: Bhyve segfaults when accessing the network Message-ID: <20140223203348.GA74438@psconsult.nl> References: <20140223155741.GA75513@psconsult.nl> <530A2DC8.6000107@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <530A2DC8.6000107@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 20:33:58 -0000 Hi Peter, Thanks for your quick response! On Sun, Feb 23, 2014 at 09:20:08AM -0800, Peter Grehan wrote: > Hi Paul, > > > Hardware: Supermicro X9SPV-F3610ME-O (embedded i5) with 16GB RAM > > > > Host kernels (+userland of the same version): > > 10.0 RC5 > > 10.0 RELEASE > > 10 STABLE > > Does the segfault happen on h/w with all of these ? (I think you > mention below that it's only 10-stable that has the issue with Fusion). The segfault occurs on all of these on real h/w and on Fusion only the guest faults (repeatedly) on a 10 STABLE host (my previous email was not very clear). I have only tested a 10.0 RELEASE guest on Fusion. > > The kernel has been stripped (unneeded drivers and stuff like DEBUG, > > AUDIT, CAPABILITIES, MAC removed) > > Can you post the kernel config you are using ? See below. > later, > > Peter. With kind regards, Paul Schenkeveld ------------------------------------------------------------------------ cpu HAMMER ident HOMESERVER #makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_OFFLOAD # TCP offload options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journaling options QUOTA # Enable disk quotas for UFS options MD_ROOT # MD is a potential root device options NFSCL # New Network Filesystem Client options NFSD # New Network Filesystem Server options NFSLOCKD # Network Lock Manager options NFS_ROOT # NFS usable as /, requires NFSCL options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options COMPAT_FREEBSD7 # Compatible with FreeBSD7 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. options KBD_INSTALL_CDEV # install a CDEV entry in /dev options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) #options AUDIT # Security event auditing #options CAPABILITY_MODE # Capsicum capability mode #options CAPABILITIES # Capsicum capabilities options PROCDESC # Support for process descriptors #options MAC # TrustedBSD MAC Framework options KDTRACE_FRAME # Ensure frames are compiled in options KDTRACE_HOOKS # Kernel DTrace hooks options DDB_CTF # Kernel ELF linker loads CTF data options INCLUDE_CONFIG_FILE # Include this file in kernel # Debugging support. Always need this: options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel # CPU frequency control device cpufreq # Bus support. device acpi device pci # Floppy drives #device fdc # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers options ATA_STATIC_ID # Static device numbering device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA # SCSI Controllers #device ahc # AHA2940 and onboard AIC7xxx devices #options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. #device ahd # AHA39320/29320 and onboard AIC79xx devices #options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. #device esp # AMD Am53C974 (Tekram DC-390(T)) #device hptiop # Highpoint RocketRaid 3xxx series #device isp # Qlogic family #device ispfw # Firmware for QLogic HBAs- normally a module #device mpt # LSI-Logic MPT-Fusion #device mps # LSI-Logic MPT-Fusion 2 #device ncr # NCR/Symbios Logic #device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') #device trm # Tekram DC395U/UW/F DC315U adapters #device adv # Advansys SCSI adapters #device adw # Advansys wide SCSI adapters #device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. #device bt # Buslogic/Mylex MultiMaster SCSI adapters #device isci # Intel C600 SAS controller # ATA/SCSI peripherals device scbus # SCSI bus (required for ATA/SCSI) #device ch # SCSI media changers device da # Direct Access (disks) #device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct ATA/SCSI access) #device ses # Enclosure Services (SES and SAF-TE) #device ctl # CAM Target Layer # RAID controllers interfaced to the SCSI subsystem #device amr # AMI MegaRAID #device arcmsr # Areca SATA II RAID #XXX it is not 64-bit clean, -scottl #device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID #device ciss # Compaq Smart RAID 5* #device dpt # DPT Smartcache III, IV - See NOTES for options #device hptmv # Highpoint RocketRAID 182x #device hptnr # Highpoint DC7280, R750 #device hptrr # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx #device hpt27xx # Highpoint RocketRAID 27xx #device iir # Intel Integrated RAID #device ips # IBM (Adaptec) ServeRAID #device mly # Mylex AcceleRAID/eXtremeRAID #device twa # 3ware 9000 series PATA/SATA RAID #device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller # RAID controllers #device aac # Adaptec FSA RAID #device aacp # SCSI passthrough for aac (requires CAM) #device aacraid # Adaptec by PMC RAID #device ida # Compaq Smart RAID #device mfi # LSI MegaRAID SAS #device mlx # Mylex DAC960 family #XXX pointer/int warnings #device pst # Promise Supertrak SX6000 #device twe # 3ware ATA RAID # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver options VESA # Add support for VESA BIOS Extensions (VBE) device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc options SC_PIXEL_MODE # add support for the raster text mode device agp # support several AGP chipsets # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support #device cbb # cardbus (yenta) bridge #device pccard # PC Card (16-bit) bus #device cardbus # CardBus (32-bit) bus # Serial (COM) ports device uart # Generic UART driver # Parallel port #device ppc #device ppbus # Parallel port bus (required) #device lpt # Printer #device ppi # Parallel port interface device #device vpo # Requires scbus and da #device puc # Multi I/O cards and multi-channel UARTs # PCI Ethernet NICs. #device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE #device de # DEC/Intel DC21x4x (``Tulip'') device em # Intel PRO/1000 Gigabit Ethernet Family #device igb # Intel PRO/1000 PCIE Server Gigabit Family #device ixgbe # Intel PRO/10GbE PCIE Ethernet Family #device le # AMD Am7900 LANCE and Am79C9xx PCnet #device ti # Alteon Networks Tigon I/II gigabit Ethernet #device txp # 3Com 3cR990 (``Typhoon'') #device vx # 3Com 3c590, 3c595 (``Vortex'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support #device ae # Attansic/Atheros L2 FastEthernet #device age # Attansic/Atheros L1 Gigabit Ethernet #device alc # Atheros AR8131/AR8132 Ethernet #device ale # Atheros AR8121/AR8113/AR8114 Ethernet #device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet #device bfe # Broadcom BCM440x 10/100 Ethernet #device bge # Broadcom BCM570xx Gigabit Ethernet #device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn #device dc # DEC/Intel 21143 and various workalikes #device et # Agere ET1310 10/100/Gigabit Ethernet #device fxp # Intel EtherExpress PRO/100B (82557, 82558) #device gem # Sun GEM/Sun ERI/Apple GMAC #device hme # Sun HME (Happy Meal Ethernet) #device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet #device lge # Level 1 LXT1001 gigabit Ethernet #device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet #device nfe # nVidia nForce MCP on-board Ethernet #device nge # NatSemi DP83820 gigabit Ethernet #device nve # nVidia nForce MCP on-board Ethernet Networking #device pcn # AMD Am79C97x PCI 10/100 (precedence over 'le') #device re # RealTek 8139C+/8169/8169S/8110S #device rl # RealTek 8129/8139 #device sf # Adaptec AIC-6915 (``Starfire'') #device sge # Silicon Integrated Systems SiS190/191 #device sis # Silicon Integrated Systems SiS 900/SiS 7016 #device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet #device ste # Sundance ST201 (D-Link DFE-550TX) #device stge # Sundance/Tamarack TC9021 gigabit Ethernet #device tl # Texas Instruments ThunderLAN #device tx # SMC EtherPower II (83c170 ``EPIC'') #device vge # VIA VT612x gigabit Ethernet #device vr # VIA Rhine, Rhine II #device wb # Winbond W89C840F #device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # ISA Ethernet NICs. pccard NICs included. #device cs # Crystal Semiconductor CS89x0 NIC # 'device ed' requires 'device miibus' #device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards #device ex # Intel EtherExpress Pro/10 and Pro/10+ #device ep # Etherlink III based cards #device fe # Fujitsu MB8696x based cards #device sn # SMC's 9000 series of Ethernet chips #device xe # Xircom pccard Ethernet # Wireless NIC cards #device wlan # 802.11 support #options IEEE80211_DEBUG # enable debug msgs #options IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's #options IEEE80211_SUPPORT_MESH # enable 802.11s draft support #device wlan_wep # 802.11 WEP support #device wlan_ccmp # 802.11 CCMP support #device wlan_tkip # 802.11 TKIP support #device wlan_amrr # AMRR transmit rate control algorithm #device an # Aironet 4500/4800 802.11 wireless NICs. #device ath # Atheros NICs #device ath_pci # Atheros pci/cardbus glue #device ath_hal # pci/cardbus chip support #options AH_SUPPORT_AR5416 # enable AR5416 tx/rx descriptors #options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation #options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later #device ath_rate_sample # SampleRate tx rate control for ath #device bwi # Broadcom BCM430x/BCM431x wireless NICs. #device bwn # Broadcom BCM43xx wireless NICs. #device ipw # Intel 2100 wireless NICs. #device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. #device iwn # Intel 4965/1000/5000/6000 wireless NICs. #device malo # Marvell Libertas wireless NICs. #device mwl # Marvell 88W8363 802.11n wireless NICs. #device ral # Ralink Technology RT2500 wireless NICs. #device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. #device wpi # Intel 3945ABG wireless NICs. # Pseudo devices. device loop # Network loopback device random # Entropy device device padlock_rng # VIA Padlock RNG device rdrand_rng # Intel Bull Mountain RNG device ether # Ethernet support device vlan # 802.1Q VLAN support device tun # Packet tunnel. device md # Memory "disks" device gif # IPv6 and IPv4 tunneling #device faith # IPv6-to-IPv4 relaying (translation) device firmware # firmware assist module # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter # USB support options USB_DEBUG # enable debug msgs device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device xhci # XHCI PCI->USB interface (USB 3.0) device usb # USB Bus (required) device ukbd # Keyboard device umass # Disks/Mass storage - Requires scbus and da # Sound support device sound # Generic sound driver (required) device snd_cmi # CMedia CMI8338/CMI8738 device snd_csa # Crystal Semiconductor CS461x/428x device snd_emu10kx # Creative SoundBlaster Live! and Audigy device snd_es137x # Ensoniq AudioPCI ES137x device snd_hda # Intel High Definition Audio device snd_ich # Intel, NVidia and other ICH AC'97 Audio device snd_via8233 # VIA VT8233x Audio # MMC/SD #device mmc # MMC/SD bus #device mmcsd # MMC/SD memory card #device sdhci # Generic PCI SD Host Controller # VirtIO support device virtio # Generic VirtIO bus (required) device virtio_pci # VirtIO PCI device device vtnet # VirtIO Ethernet device device virtio_blk # VirtIO Block device device virtio_scsi # VirtIO SCSI device device virtio_balloon # VirtIO Memory Balloon device # HyperV drivers #device hyperv # HyperV drivers # Xen HVM Guest Optimizations # NOTE: XENHVM depends on xenpci. They must be added or removed together. #options XENHVM # Xen HVM kernel infrastructure #device xenpci # Xen HVM Hypervisor services driver # VMware support #device vmx # VMware VMXNET3 Ethernet From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 21:04:46 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89424FE3; Sun, 23 Feb 2014 21:04:46 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3BFD615C4; Sun, 23 Feb 2014 21:04:46 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id CB961153AA9; Sun, 23 Feb 2014 22:04:44 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YgSBaF96fSrZ; Sun, 23 Feb 2014 22:04:43 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0] (unknown [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 2C348153AA7; Sun, 23 Feb 2014 22:04:43 +0100 (CET) Message-ID: <530A6269.3070806@digiware.nl> Date: Sun, 23 Feb 2014 22:04:41 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan , "freebsd-virtualization@freebsd.org" Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> <530A2D11.707@digiware.nl> <530A2F55.8090307@freebsd.org> In-Reply-To: <530A2F55.8090307@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 21:04:46 -0000 On 23-2-2014 18:26, Peter Grehan wrote: >> Against what current version did you merge/build the SVM branch? >> I'll either check that out, or try my subversion skills to see if that >> reduces the diff. > > My mistake - the sync was a r259205, which was from Dec 10 and > predates the -w option (r259635, Dec 17). I manually added this patch... And then I can get 13.10 to boot. First I get 20* the msr ignore message, but now I in the installer. So you might want to put that in your SVM tree to get more mileage on the code in Linux envs. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 21:12:21 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72F5F52B for ; Sun, 23 Feb 2014 21:12:21 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 2E4B5165F for ; Sun, 23 Feb 2014 21:12:20 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 8ABAF122E0; Mon, 24 Feb 2014 07:12:19 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-188-143-252.hsd1.ca.comcast.net [67.188.143.252]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSA69175 (AUTH peterg@ptree32.com.au); Mon, 24 Feb 2014 07:12:18 +1000 Message-ID: <530A642F.1000507@freebsd.org> Date: Sun, 23 Feb 2014 13:12:15 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> <530A2D11.707@digiware.nl> <530A2F55.8090307@freebsd.org> <530A6269.3070806@digiware.nl> In-Reply-To: <530A6269.3070806@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 21:12:21 -0000 > On 23-2-2014 18:26, Peter Grehan wrote: >>> Against what current version did you merge/build the SVM branch? >>> I'll either check that out, or try my subversion skills to see if that >>> reduces the diff. >> >> My mistake - the sync was a r259205, which was from Dec 10 and >> predates the -w option (r259635, Dec 17). > > I manually added this patch... > And then I can get 13.10 to boot. Great ! > First I get 20* the msr ignore message, but now I in the installer. Can you post the MSRs that were hit ? I'd be interested to see what they are. > So you might want to put that in your SVM tree to get more mileage on > the code in Linux envs. I'll see if I can sync up to that point without messing anything up. Thanks for trying it out. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 21:16:45 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C184B91E; Sun, 23 Feb 2014 21:16:45 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7394D1689; Sun, 23 Feb 2014 21:16:45 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id EE069153AA9; Sun, 23 Feb 2014 22:16:43 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h8Cc_E2Zqg7N; Sun, 23 Feb 2014 22:16:41 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0] (unknown [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id E7269153448; Sun, 23 Feb 2014 22:16:41 +0100 (CET) Message-ID: <530A6538.1070008@digiware.nl> Date: Sun, 23 Feb 2014 22:16:40 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> <530A2D11.707@digiware.nl> <530A2F55.8090307@freebsd.org> <530A6269.3070806@digiware.nl> <530A642F.1000507@freebsd.org> In-Reply-To: <530A642F.1000507@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 21:16:45 -0000 On 23-2-2014 22:12, Peter Grehan wrote: >> On 23-2-2014 18:26, Peter Grehan wrote: >>>> Against what current version did you merge/build the SVM branch? >>>> I'll either check that out, or try my subversion skills to see if that >>>> reduces the diff. >>> >>> My mistake - the sync was a r259205, which was from Dec 10 and >>> predates the -w option (r259635, Dec 17). >> >> I manually added this patch... >> And then I can get 13.10 to boot. > > Great ! > >> First I get 20* the msr ignore message, but now I in the installer. > > Can you post the MSRs that were hit ? I'd be interested to see what > they are. This is the preamble before we get into the CD-installation. The process is really really slow, and consumes about 150% op CPU. aka 1,5 CPU.... --WjW rdmsr to register 0xc0010015 on vcpu 0 rdmsr to register 0xc0010112 on vcpu 0 rdmsr to register 0xc0010048 on vcpu 0 wrmsr to register 0xc0010048(0x400) on vcpu 0 rdmsr to register 0xc001102a on vcpu 0 wrmsr to register 0xc001102a(0) on vcpu 0 rdmsr to register 0xc0010140 on vcpu 0 rdmsr to register 0xc0010140 on vcpu 0 rdmsr to register 0xc0010000 on vcpu 0 rdmsr to register 0xc0010001 on vcpu 0 rdmsr to register 0xc0010002 on vcpu 0 rdmsr to register 0xc0010003 on vcpu 0 rdmsr to register 0xc0010004 on vcpu 0 wrmsr to register 0xc0010004(0xffff) on vcpu 0 rdmsr to register 0xc0010004 on vcpu 0 rdmsr to register 0xc0010048 on vcpu 1 wrmsr to register 0xc0010048(0x400) on vcpu 1 rdmsr to register 0xc001102a on vcpu 1 wrmsr to register 0xc001102a(0) on vcpu 1 rdmsr to register 0xc0010140 on vcpu 1 rdmsr to register 0xc0010140 on vcpu 1 rdmsr to register 0xc001001f on vcpu 1 wrmsr to register 0xc001001f(0x400000000000) on vcpu 1 rdmsr to register 0xc001001f on vcpu 0 wrmsr to register 0xc001001f(0x400000000000) on vcpu 0 rdmsr to register 0xc001103a on vcpu 0 Starting system log daemon: syslogd, klogd. From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 23 23:35:43 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C41714F; Sun, 23 Feb 2014 23:35:43 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B8E151221; Sun, 23 Feb 2014 23:35:42 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id B52AD153AA8; Mon, 24 Feb 2014 00:35:39 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FHeKA8DksJAq; Mon, 24 Feb 2014 00:35:37 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0] (unknown [IPv6:2001:4cb8:3:1:35b9:35cb:94ce:e2c0]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 6E0BB1534EF; Mon, 24 Feb 2014 00:35:37 +0100 (CET) Message-ID: <530A85C7.5070107@digiware.nl> Date: Mon, 24 Feb 2014 00:35:35 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Bhyve and booting a ZFS-on-root system References: <53090599.5000505@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 23:35:43 -0000 On 22-2-2014 22:28, Craig Rodrigues wrote: > On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen > wrote: > > Hi, > > Just for the fun of it, I tried my build zfs-system scripts in a > bhyve-vm. > > I use the 10.0-RELEASE iso to get to a shell, config and interface and > download my script. Installing does work, and on a regular system we can > go and boot into a ZFS-on-Root system. > > > ZFS on Root inside a BHyve VM was not working until > today. If you you have a FreeBSD-CURRENT system and then update > to this revision: > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html Good pointer..... Added this patch to my tree, recompiled buildworld, and installed. And I'm up and running a Root on ZFS vm.... So that works too. Now If only i figured out why my linux install is running horibily slow..... I sort of have the feeling that CD/iso access is the crulpit. Every time new things are being loaded it is like molasses. Hopefully the vm without the CD does perform. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Mon Feb 24 00:56:14 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E8A8CBF for ; Mon, 24 Feb 2014 00:56:14 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E0CA61815 for ; Mon, 24 Feb 2014 00:56:13 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id mc6so1287818lab.8 for ; Sun, 23 Feb 2014 16:56:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6r3XKQqH1J17aykY4MX5OP9dQIuTl2OKUimJqysR58c=; b=yGytn1T5ulk5DyVwrUqdpi6IIs/hxdNwK6abFzvSDVNUyI6IcEwMcnUmMJEtVy7HIu +OgEUlBUakcfzAC62/xIqlPnsWdZgSEkfT/WnlnX84wVDMXu+By/rE6HTMwRL1GBn1r8 /zWGkj7dbdp6WvKRP21p6CeGc7IveojTFEv8/rOz0IAy/z4ZWqkGTf1FxB3BPI8uiNoD KZamMvaHBMMVjnqwJPis3WGInHeJAcDl7E6zbj3mTrC6LheXhH0Do7iAyB92Cs18dfWa PHNhlLwUG81/FoZHPWCevGm2yfewkhASFkYgGCPsxbtOX62R7jmmkBSA8I4lZjsX+393 5dBg== MIME-Version: 1.0 X-Received: by 10.112.154.202 with SMTP id vq10mr10006736lbb.3.1393203371037; Sun, 23 Feb 2014 16:56:11 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sun, 23 Feb 2014 16:56:10 -0800 (PST) Received: by 10.112.30.211 with HTTP; Sun, 23 Feb 2014 16:56:10 -0800 (PST) In-Reply-To: <530A85C7.5070107@digiware.nl> References: <53090599.5000505@digiware.nl> <530A85C7.5070107@digiware.nl> Date: Sun, 23 Feb 2014 16:56:10 -0800 X-Google-Sender-Auth: 73QHlNTRL77xkT2QF3ltCBXU3G4 Message-ID: Subject: Re: Bhyve and booting a ZFS-on-root system From: Craig Rodrigues To: Willem Jan Withagen Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 00:56:14 -0000 On Feb 23, 2014 3:35 PM, "Willem Jan Withagen" wrote: > > On 22-2-2014 22:28, Craig Rodrigues wrote: > > > > ZFS on Root inside a BHyve VM was not working until > > today. If you you have a FreeBSD-CURRENT system and then update > > to this revision: > > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html > > Good pointer..... > > Added this patch to my tree, recompiled buildworld, and installed. > And I'm up and running a Root on ZFS vm.... > > So that works too. > That's good news! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Feb 24 09:00:01 2014 Return-Path: Delivered-To: freebsd-virtualization@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABB152B6 for ; Mon, 24 Feb 2014 09:00:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7E49B1968 for ; Mon, 24 Feb 2014 09:00:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1O901rF087217 for ; Mon, 24 Feb 2014 09:00:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1O9010j087216; Mon, 24 Feb 2014 09:00:01 GMT (envelope-from gnats) Date: Mon, 24 Feb 2014 09:00:01 GMT Message-Id: <201402240900.s1O9010j087216@freefall.freebsd.org> To: freebsd-virtualization@FreeBSD.org Cc: From: dfilter@FreeBSD.ORG (dfilter service) Subject: Re: kern/183835: commit references a PR X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dfilter service List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 09:00:01 -0000 The following reply was made to PR kern/183835; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/183835: commit references a PR Date: Mon, 24 Feb 2014 08:50:19 +0000 (UTC) Author: rodrigc Date: Mon Feb 24 08:50:06 2014 New Revision: 262436 URL: http://svnweb.freebsd.org/changeset/base/262436 Log: MFC r262142: In ue_attach_post_task(), initialize curvnet to vnet0 before calling if_attach(). Before this patch, curvnet was NULL. When the VIMAGE kernel option is enabled, this eliminates kernel panics when USB ethernet devices are plugged in. PR: 183835 Submitted by: Hiroo Oono Modified: stable/10/sys/dev/usb/net/usb_ethernet.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/dev/usb/net/usb_ethernet.c ============================================================================== --- stable/10/sys/dev/usb/net/usb_ethernet.c Mon Feb 24 08:21:49 2014 (r262435) +++ stable/10/sys/dev/usb/net/usb_ethernet.c Mon Feb 24 08:50:06 2014 (r262436) @@ -207,6 +207,7 @@ ue_attach_post_task(struct usb_proc_msg sysctl_ctx_init(&ue->ue_sysctl_ctx); error = 0; + CURVNET_SET_QUIET(vnet0); ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { device_printf(ue->ue_dev, "could not allocate ifnet\n"); @@ -253,6 +254,8 @@ ue_attach_post_task(struct usb_proc_msg if (ifp->if_capabilities & IFCAP_VLAN_MTU) ifp->if_hdrlen = sizeof(struct ether_vlan_header); + CURVNET_RESTORE(); + snprintf(num, sizeof(num), "%u", ue->ue_unit); ue->ue_sysctl_oid = SYSCTL_ADD_NODE(&ue->ue_sysctl_ctx, &SYSCTL_NODE_CHILDREN(_net, ue), @@ -266,6 +269,7 @@ ue_attach_post_task(struct usb_proc_msg return; fail: + CURVNET_RESTORE(); free_unr(ueunit, ue->ue_unit); if (ue->ue_ifp != NULL) { if_free(ue->ue_ifp); _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Mon Feb 24 09:05:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B840A51F; Mon, 24 Feb 2014 09:05:00 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD0031A13; Mon, 24 Feb 2014 09:04:59 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf5so2225137lab.7 for ; Mon, 24 Feb 2014 01:04:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=KRkLcxHc+/+Arz6ir99e9Fby4VLjMu9etyg9WlPIevY=; b=zh6qL+Iop7VIoTXqo4uwTfoz1f/qKPVy1b5QEyXbl2mcv5wAXRORpjtVM0nRf2pebn ksvgWQYnVNIjLA3jRsZWQjJbIiEXW8Wo5PNaHCQG6tR2lt8bbMFs80xutHxaA/ATTnDR hU38/etvqEob6nVJKWs9EW9RtMQ4JaCbOCPpIWV8CRUsTTbuhXdBLRoBfeb/Jm0knHSf oRR2AQDTbTLHDE7ytKh073j7d7sAJtqrTraFZIORtWPGzxsYFh2kPVSGkAlBz14XpHNA Oc8ZIfpP/U8WdBoieGR2SUt34hADCdhwlSOmoPwkXnK3amgh7N0/HyqMIg0z9htWyIM5 ePgg== MIME-Version: 1.0 X-Received: by 10.112.38.4 with SMTP id c4mr6022535lbk.84.1393232697995; Mon, 24 Feb 2014 01:04:57 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 01:04:57 -0800 (PST) Date: Mon, 24 Feb 2014 01:04:57 -0800 X-Google-Sender-Auth: GJ-RLk0zanzAiFwB-3MMsyh7h-o Message-ID: Subject: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 09:05:00 -0000 Hi, I will be giving a talk discussing what I have done settings up Jenkins ( https://jenkins.freebsd.org ) in the FreeBSD cluster, using BHyve VM's: The presentation will be on March 13 in Mountain View, California, U.S.A.: http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ If you think you can come, please RSVP on the Meetup site, so that the organizers can get a big enough room. I am interested in collaborating with FreeBSD hackers who can contribute to the effort and help expand testing of FreeBSD!! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Feb 24 11:07:00 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1085FB70 for ; Mon, 24 Feb 2014 11:07:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EFFCF1637 for ; Mon, 24 Feb 2014 11:06:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1OB6xFH027737 for ; Mon, 24 Feb 2014 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1OB6xlG027735 for freebsd-virtualization@FreeBSD.org; Mon, 24 Feb 2014 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Feb 2014 11:06:59 GMT Message-Id: <201402241106.s1OB6xlG027735@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 11:07:00 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Feb 24 12:11:28 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C1D5ACE; Mon, 24 Feb 2014 12:11:28 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB1981D98; Mon, 24 Feb 2014 12:11:27 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 12DB11534EF; Mon, 24 Feb 2014 13:11:26 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SsstFpkS5oBo; Mon, 24 Feb 2014 13:11:24 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:a009:5318:9749:5155] (unknown [IPv6:2001:4cb8:3:1:a009:5318:9749:5155]) by smtp.digiware.nl (Postfix) with ESMTP id F25881534ED; Mon, 24 Feb 2014 13:11:23 +0100 (CET) Message-ID: <530B36ED.7090506@digiware.nl> Date: Mon, 24 Feb 2014 13:11:25 +0100 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan , "freebsd-virtualization@freebsd.org" Subject: Re: Bhyve and Ubuntu booting References: <53090599.5000505@digiware.nl> <20AAA4CA-38F9-4D07-BAAA-434F8BC20129@digiware.nl> <530A20BE.4050602@digiware.nl> <530A2475.6090608@freebsd.org> <530A2713.6000602@digiware.nl> <530A2870.9080306@freebsd.org> <530A2978.7070403@digiware.nl> <530A2B35.6090409@freebsd.org> <530A2D11.707@digiware.nl> <530A2F55.8090307@freebsd.org> <530A6269.3070806@digiware.nl> In-Reply-To: <530A6269.3070806@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 12:11:28 -0000 On 2014-02-23 22:04, Willem Jan Withagen wrote: > On 23-2-2014 18:26, Peter Grehan wrote: >>> Against what current version did you merge/build the SVM branch? >>> I'll either check that out, or try my subversion skills to see if that >>> reduces the diff. >> >> My mistake - the sync was a r259205, which was from Dec 10 and >> predates the -w option (r259635, Dec 17). > > I manually added this patch... > And then I can get 13.10 to boot. > > First I get 20* the msr ignore message, but now I in the installer. > > So you might want to put that in your SVM tree to get more mileage on > the code in Linux envs. std-disclaimer: Running on AMD processor with SVM branch I've started the ubuntu install again, since last night it went into the dark hours. And my laptop shutdown the connection while I used console. So no recovery to the prompts :( Now I'm trying 2 Ubuntu installs: 13.10 12.04 And both run equally slow. And generate msr access that would otherwise be trapped. (running with 2 vCPUs, and 2G mem). From top: 67705 root 5 52 0 2081M 429M CPU1 1 74:38 147.66% bhyve: ubuntu-13.10 67881 root 5 52 0 2077M 194M CPU3 3 67:20 137.16% bhyve: ubuntu-12.04 And the load is always up in the 150%.... So obviously these 2 installers are doing a form sort of busy waiting? Are other people seeing the same on AMD cpus or on Intel CPUs? Thanx, --WjW From owner-freebsd-virtualization@FreeBSD.ORG Tue Feb 25 00:23:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4935BC68; Tue, 25 Feb 2014 00:23:06 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3975818D7; Tue, 25 Feb 2014 00:23:05 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id y1so2702137lam.9 for ; Mon, 24 Feb 2014 16:23:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=zF5xwKyUhnJUWBCaZIEMD8EDDkoPcaR6rFXKy9DjS+4=; b=BgimT1CPj3GL55pRqNudZQCgohXSPx2/5g/oJ9pZRig3i8ZVj7CsIZKX6RzLdqFJ7y nYbTsyYCz8DCtnAjftAQM2f9+CZlFzRuiEm60Z63pajEkrmU6lQaADus1QFGkaJDGI7B xH6fy3hS/SuqNJ0KNtOisZLu1td6qlbzXI2Yx8sEBoCOaSQ9E0nMR2Ovw2x7wV2t+d3S 03JY2DpK1BlSD1pfJdD2FXJeoZP634E1R0vDPsaohvEF8emD884lq19KoNmn9C94lL/H /sR+iIapOydjx88V3VcnR6cCd5uhVr/xGp8vwLp+kU76WEMfPFmoJ3anITf29i00fN+B UwlA== MIME-Version: 1.0 X-Received: by 10.112.132.131 with SMTP id ou3mr13073216lbb.29.1393287783296; Mon, 24 Feb 2014 16:23:03 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 16:23:03 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 16:23:03 -0800 X-Google-Sender-Auth: nA4hBqd4BzoJYgVvU9PqAqRTaUY Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Dmitry Morozovsky , Emanuel Haupt X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 00:23:06 -0000 On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > Hi, > > I will be giving a talk discussing what I have done > settings up Jenkins ( https://jenkins.freebsd.org ) > in the FreeBSD cluster, using BHyve VM's: > > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ > > If you think you can come, please RSVP on the Meetup site, so that > the organizers can get a big enough room. > > I am interested in collaborating with FreeBSD hackers who can > contribute to the effort and help expand testing of FreeBSD!! > Hi, For those who cannot attend the meetup in Mountain View, California, U.S.A., I will post a link to the slides after the presentation. I am not sure if video/webcast will be available. I have given a similar presentation at the 2013 FreeBSD Vendor Summit describing the use of Jenkins in the FreeNAS project, so you can look at this to get an idea: http://www.ixsystems.com/whats-new/jenkins-bhyve-and-webdriver-continuous-integration-testing-on-freenas/ However, my presentation on March 13 will be focused on how I set up https://jenkins.freebsd.org If you cannot make the March 13 meetup, I will be giving a similar presentation on May 17 at BSDCan: http://www.bsdcan.org/2014/schedule/events/445.en.html and on May 15 there will be a working group where we will discuss how to expand Continuous Testing in the FreeBSD project: https://wiki.freebsd.org/201405DevSummit Hope to see folks at one of these events, so we can do some interesting collaboration! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Feb 25 01:20:39 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E50A780 for ; Tue, 25 Feb 2014 01:20:39 +0000 (UTC) Received: from mail-pb0-x233.google.com (mail-pb0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31EA41CCD for ; Tue, 25 Feb 2014 01:20:39 +0000 (UTC) Received: by mail-pb0-f51.google.com with SMTP id un15so7312513pbc.24 for ; Mon, 24 Feb 2014 17:20:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=G22BsxlPnP8Q/sCRc+TdqKAJZsxMmU8kwkvxT7HQ+CU=; b=igBUxLLmtBYf546CvxEgvmmj7xdievmmpYbsqeNOvmkYqq29W4Ynd0tB+cmkgFyKLM MVdtoy97DMb1L312Fmg4PgiqD3m7ICYo2YTcGk6MHVexuLvNfHdede6dw2JwrUzvC3KA /LPU5+cyydoF/UUVpybYJgplsQNYC+EDI3khorK0/Ei2WfWuTh4aAwUAjhLGEDwfTmYH Yx3ynWX2h7q8C85eamRxjoWXIFqWxfL5s8pmOVQJz8+e1dnxZqdFo76Zspc+gnHKZN+X kHtQyspzmZDgJBb/DYobobTT9aUbBsj6EBse6HRHjUxiMwtWZXQmbtLpLLQA2twQ3CIq XWbA== MIME-Version: 1.0 X-Received: by 10.66.149.37 with SMTP id tx5mr17144812pab.81.1393291237918; Mon, 24 Feb 2014 17:20:37 -0800 (PST) Received: by 10.68.155.38 with HTTP; Mon, 24 Feb 2014 17:20:37 -0800 (PST) Date: Mon, 24 Feb 2014 20:20:37 -0500 Message-ID: Subject: correctly putting all bhyve vm's on the same internal networtk From: Aryeh Friedman To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:20:39 -0000 Setting up a ineternal network on the bridge nic seems to now allow guests to communicate (see below for failed attempt transcript) root@lilith:/tmp # ifconfig bridge0 192.168.0.1 ... Install an instance and put it on 192.168.0.2 ... Host: root@lilith:/tmp # netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 10.0.10.1 UGS 0 0 re0 10.0.10.0/24 link#2 U 0 107642 re0 10.0.10.20 link#2 UHS 0 0 lo0 127.0.0.1 link#3 UH 0 4 lo0 192.168.0.0/24 link#4 U 0 316 bridge 192.168.0.1 link#4 UHS 0 68 lo0 ... ral0: flags=8802 metric 0 mtu 2290 ether 20:10:7a:58:b9:29 nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) status: no carrier re0: flags=8943 metric 0 mtu 1500 options=82099 ether e8:40:f2:97:38:d1 inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 02:66:19:ad:68:00 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap8 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap3 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: re0 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 20000 tap3: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:1a:4b:06:03 nd6 options=21 media: Ethernet autoselect status: no carrier tap8: flags=8943 metric 0 mtu 1500 options=80000 ether 00:bd:65:09:16:08 inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 nd6 options=21 media: Ethernet autoselect status: active Opened by PID 1362 root@lilith:/tmp # ifconfig ral0: flags=8802 metric 0 mtu 2290 ether 20:10:7a:58:b9:29 nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) status: no carrier re0: flags=8943 metric 0 mtu 1500 options=82099 ether e8:40:f2:97:38:d1 inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 02:66:19:ad:68:00 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap8 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap3 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: re0 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 20000 tap3: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:1a:4b:06:03 nd6 options=21 media: Ethernet autoselect status: no carrier tap8: flags=8943 metric 0 mtu 1500 options=80000 ether 00:bd:65:09:16:08 inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 nd6 options=21 media: Ethernet autoselect status: active Opened by PID 1362 root@lilith:/tmp # root@lilith:/tmp # root@lilith:/tmp # ifconfig ral0: flags=8802 metric 0 mtu 2290 ether 20:10:7a:58:b9:29 nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) status: no carrier re0: flags=8943 metric 0 mtu 1500 options=82099 ether e8:40:f2:97:38:d1 inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 02:66:19:ad:68:00 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap8 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap3 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: re0 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 20000 tap3: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:1a:4b:06:03 nd6 options=21 media: Ethernet autoselect status: no carrier tap8: flags=8943 metric 0 mtu 1500 options=80000 ether 00:bd:65:09:16:08 inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 nd6 options=21 media: Ethernet autoselect status: active Opened by PID 1362 ... root@lilith:/tmp # ping 128.114.27.16 PING 128.114.27.16 (128.114.27.16): 56 data bytes 64 bytes from 128.114.27.16: icmp_seq=0 ttl=45 time=164.864 ms 64 bytes from 128.114.27.16: icmp_seq=1 ttl=45 time=184.155 ms 64 bytes from 128.114.27.16: icmp_seq=2 ttl=45 time=222.135 ms ... $ ping 192.168.0.2 PING 192.168.0.2 (192.168.0.2): 56 data bytes 64 bytes from 192.168.0.2 : icmp_seq=0 ttl=64 time=0.225 ms 64 bytes from 192.168.0.2 : icmp_seq=1 ttl=64 time=3.790 ms ... login into guest ... $ ping 192.168.0.1 PING 192.168.0.1 (192.168.0.1): 56 data bytes 64 bytes from 192.168.0.1: icmp_seq=0 ttl=64 time=0.225 ms 64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=3.790 ms ^C --- 192.168.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.225/2.007/3.790/1.783 ms $ ping 10.0.10.20 PING 10.0.10.20 (10.0.10.20): 56 data bytes 64 bytes from 10.0.10.20: icmp_seq=0 ttl=64 time=0.222 ms 64 bytes from 10.0.10.20: icmp_seq=1 ttl=64 time=0.121 ms ^C --- 10.0.10.20 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.121/0.171/0.222/0.051 ms $ ping 10.0.10.1 PING 10.0.10.1 (10.0.10.1): 56 data bytes ^C --- 10.0.10.1 ping statistics --- -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Tue Feb 25 01:25:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E17896A for ; Tue, 25 Feb 2014 01:25:33 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 302721D66 for ; Tue, 25 Feb 2014 01:25:33 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id md12so3151067pbc.21 for ; Mon, 24 Feb 2014 17:25:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=qVMrLFgD7ouB0Wo+kU+thBBP8oIokmdiODhZofZXLiQ=; b=D2W3R5M9hfdmMtO59YoJ8ryr8CnePZ8wA9DdEnvZ+OeEbJOGV1c7joHkqiw6NFFgTt cpho//+o46spUJX1QhLaXMZ6I/zYtyckBbQzkRW92BRmxpdvmP/3PQAzvu6poIGbagt2 pmmb4trHtDx5c4YvYdxnYPYHlzUXkQJtH3UOIxsBkWpfMVgulzSUCI2M3IJiaovlvB4I 5QHvm9DE+Xfn4pEgnirQQgdJumTqrghRU0tf7eCpfhqw75ieN3ykF4hAogFnpM5bSi9R 5JLfHSj87ItcsXTjR6QIce5mrYkCGPce+7WY+LGj3sXMlsYuGxwNVCKoX7pnxw1CLmiL 3b/w== MIME-Version: 1.0 X-Received: by 10.68.139.73 with SMTP id qw9mr2967424pbb.121.1393291532765; Mon, 24 Feb 2014 17:25:32 -0800 (PST) Received: by 10.68.155.38 with HTTP; Mon, 24 Feb 2014 17:25:32 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 20:25:32 -0500 Message-ID: Subject: Re: correctly putting all bhyve vm's on the same internal networtk From: Aryeh Friedman To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:25:33 -0000 Forgot that net.inet.ip.forwarding=1 On Mon, Feb 24, 2014 at 8:20 PM, Aryeh Friedman wrote: > Setting up a ineternal network on the bridge nic seems to now allow guests > to communicate (see below for failed attempt transcript) > > root@lilith:/tmp # ifconfig bridge0 192.168.0.1 > ... > Install an instance and put it on 192.168.0.2 > ... > Host: > > root@lilith:/tmp # netstat -nr > Routing tables > > Internet: > Destination Gateway Flags Refs Use Netif Expire > default 10.0.10.1 UGS 0 0 re0 > 10.0.10.0/24 link#2 U 0 107642 re0 > 10.0.10.20 link#2 UHS 0 0 lo0 > 127.0.0.1 link#3 UH 0 4 lo0 > 192.168.0.0/24 link#4 U 0 316 bridge > 192.168.0.1 link#4 UHS 0 68 lo0 > ... > > ral0: flags=8802 metric 0 mtu 2290 > ether 20:10:7a:58:b9:29 > nd6 options=21 > media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) > status: no carrier > re0: flags=8943 metric 0 > mtu 1500 > > options=82099 > ether e8:40:f2:97:38:d1 > inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > bridge0: flags=8843 metric 0 mtu > 1500 > ether 02:66:19:ad:68:00 > inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 > nd6 options=1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap8 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap3 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > member: re0 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 20000 > tap3: flags=8902 metric 0 mtu 1500 > options=80000 > ether 00:bd:1a:4b:06:03 > nd6 options=21 > media: Ethernet autoselect > status: no carrier > tap8: flags=8943 metric 0 > mtu 1500 > options=80000 > ether 00:bd:65:09:16:08 > inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 > nd6 options=21 > media: Ethernet autoselect > status: active > Opened by PID 1362 > root@lilith:/tmp # ifconfig > ral0: flags=8802 metric 0 mtu 2290 > ether 20:10:7a:58:b9:29 > nd6 options=21 > media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) > status: no carrier > re0: flags=8943 metric 0 > mtu 1500 > > options=82099 > ether e8:40:f2:97:38:d1 > inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > bridge0: flags=8843 metric 0 mtu > 1500 > ether 02:66:19:ad:68:00 > inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 > nd6 options=1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap8 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap3 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > member: re0 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 20000 > tap3: flags=8902 metric 0 mtu 1500 > options=80000 > ether 00:bd:1a:4b:06:03 > nd6 options=21 > media: Ethernet autoselect > status: no carrier > tap8: flags=8943 metric 0 > mtu 1500 > options=80000 > ether 00:bd:65:09:16:08 > inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 > nd6 options=21 > media: Ethernet autoselect > status: active > Opened by PID 1362 > root@lilith:/tmp # > root@lilith:/tmp # > root@lilith:/tmp # ifconfig > ral0: flags=8802 metric 0 mtu 2290 > ether 20:10:7a:58:b9:29 > nd6 options=21 > media: IEEE 802.11 Wireless Ethernet autoselect (autoselect) > status: no carrier > re0: flags=8943 metric 0 > mtu 1500 > > options=82099 > ether e8:40:f2:97:38:d1 > inet 10.0.10.20 netmask 0xffffff00 broadcast 10.0.10.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > bridge0: flags=8843 metric 0 mtu > 1500 > ether 02:66:19:ad:68:00 > inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 > nd6 options=1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap8 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap3 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > member: re0 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 20000 > tap3: flags=8902 metric 0 mtu 1500 > options=80000 > ether 00:bd:1a:4b:06:03 > nd6 options=21 > media: Ethernet autoselect > status: no carrier > tap8: flags=8943 metric 0 > mtu 1500 > options=80000 > ether 00:bd:65:09:16:08 > inet6 fe80::2bd:65ff:fe09:1608%tap8 prefixlen 64 scopeid 0x6 > nd6 options=21 > media: Ethernet autoselect > status: active > Opened by PID 1362 > ... > root@lilith:/tmp # ping 128.114.27.16 > PING 128.114.27.16 (128.114.27.16): 56 data bytes > 64 bytes from 128.114.27.16: icmp_seq=0 ttl=45 time=164.864 ms > 64 bytes from 128.114.27.16: icmp_seq=1 ttl=45 time=184.155 ms > 64 bytes from 128.114.27.16: icmp_seq=2 ttl=45 time=222.135 ms > ... > $ ping 192.168.0.2 > PING 192.168.0.2 (192.168.0.2): 56 data bytes > 64 bytes from 192.168.0.2 : icmp_seq=0 ttl=64 > time=0.225 ms > 64 bytes from 192.168.0.2 : icmp_seq=1 ttl=64 > time=3.790 ms > ... > login into guest > ... > $ ping 192.168.0.1 > PING 192.168.0.1 (192.168.0.1): 56 data bytes > 64 bytes from 192.168.0.1: icmp_seq=0 ttl=64 time=0.225 ms > 64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=3.790 ms > ^C > --- 192.168.0.1 ping statistics --- > 2 packets transmitted, 2 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 0.225/2.007/3.790/1.783 ms > $ ping 10.0.10.20 > PING 10.0.10.20 (10.0.10.20): 56 data bytes > 64 bytes from 10.0.10.20: icmp_seq=0 ttl=64 time=0.222 ms > 64 bytes from 10.0.10.20: icmp_seq=1 ttl=64 time=0.121 ms > ^C > --- 10.0.10.20 ping statistics --- > 2 packets transmitted, 2 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 0.121/0.171/0.222/0.051 ms > $ ping 10.0.10.1 > PING 10.0.10.1 (10.0.10.1): 56 data bytes > ^C > --- 10.0.10.1 ping statistics --- > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Tue Feb 25 01:40:39 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38C1EBA3 for ; Tue, 25 Feb 2014 01:40:39 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EC6251E57 for ; Tue, 25 Feb 2014 01:40:38 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id x3so1600603qcv.32 for ; Mon, 24 Feb 2014 17:40:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=IdNVH66ANhe5GeVRFCs7Crv02dNg5VJ70vLjcX7cXys=; b=s4o0BIPCVqxCKLhOD9UQEYUmrv8scYzBOil9/4qCCGnL0yatDZBb5EHtxPNqoYxOQN Rluwvm30rzyqVVjmAOBJfXYh35Z/xsIXsnx781AmyjfdAkTTZaqs3wAIaFPINYqNsp0Z pTkjAz8k5/Gx5D4NSE/Atf4F2UxKiLnc0vRsckii+cZvsm/k3i5P7UH1bBnfUhL6j1+e s6C8NYKNuNOhhfzyUSTPJlkI0N7gn4PqGh83ivCzfm2cVhqJXJ9Qs5LajB9NUqIZeGmy T7ihmUxaazNsJQ1k5CwgwEWSUMZu3+a/r5M38n4RXHeYd2mNwi1huDAg2VpFJKs2y2WR mkYg== MIME-Version: 1.0 X-Received: by 10.224.113.204 with SMTP id b12mr34089975qaq.35.1393292438122; Mon, 24 Feb 2014 17:40:38 -0800 (PST) Received: by 10.140.87.71 with HTTP; Mon, 24 Feb 2014 17:40:38 -0800 (PST) In-Reply-To: <20140223155741.GA75513@psconsult.nl> References: <20140223155741.GA75513@psconsult.nl> Date: Mon, 24 Feb 2014 17:40:38 -0800 Message-ID: Subject: Re: Bhyve segfaults when accessing the network From: Neel Natu To: Paul Schenkeveld Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:40:39 -0000 Hi Paul, On Sun, Feb 23, 2014 at 7:57 AM, Paul Schenkeveld wrote: > Hi, > > I tried to get bhyve to run on my home server but got consistent > crashes as soon as I touch the network. > > Hardware: Supermicro X9SPV-F3610ME-O (embedded i5) with 16GB RAM > > Host kernels (+userland of the same version): > 10.0 RC5 > 10.0 RELEASE > 10 STABLE > > The kernel has been stripped (unneeded drivers and stuff like DEBUG, > AUDIT, CAPABILITIES, MAC removed) > > Guest operating systems: > 9.2 RELEASE > 10.0 RELEASE > > Procedure: > # ifconfig tap0 create up > # ifconfig bridge0 create up > # ifconfig bridge0 addm tap0 addm em0 > # kldload vmm > # mkdir /vm/test > # cd /vm/test > # ln -s /images/FreeBSD-10.0-RELEASE-amd64-disc1.iso release.iso > # sh vmrun.sh -c 1 -m 512 -t tap0 > > > /etc/ttys> > > > > At the point where DHCP sends the first query I get a segmentation fault > and the guest stops. > > I can provoke the same segfault by not putting the DHCP line into > rc.conf and trying to ifconfig vtnet0 or by adding tap0 to bridge0 in > the host after the guest has stopped. > > Although it worked perfectly fine with bhyve a couple of months ago > (with an older build of 10.0-CURRENT that I don't have here anymore), I > suspected the hardware and tried the same on my Macbook Pro with Vmware > Fusion with a 10.0 RELEASE guest. > > The stripped 10 STABLE guest segfaults in the same way, the other two > both work. > > I bought the home server for building software for older releases > (mostly FreeBSD 8 and 9), my Macbook with Fusion is too slow and often > in use for other purposes. > > Is this behaviour seen by others? > > I can provide more details or core dumps if that helps locating/solving > the problem. > I am not clear about what exactly is segfaulting: is it the bhyve process or the guest kernel? Do you have a screenshot and/or a backtrace of the segfaulting entity? best Neel > With kind regards, > > Paul Schenkeveld > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Tue Feb 25 09:54:06 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76703811 for ; Tue, 25 Feb 2014 09:54:06 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 02E77151B for ; Tue, 25 Feb 2014 09:54:03 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id s1P9roJB083516 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 25 Feb 2014 10:53:55 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id s1P9rnch083515; Tue, 25 Feb 2014 10:53:49 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Tue, 25 Feb 2014 10:53:49 +0100 From: Paul Schenkeveld To: Neel Natu Subject: Re: Bhyve segfaults when accessing the network Message-ID: <20140225095349.GA82923@psconsult.nl> References: <20140223155741.GA75513@psconsult.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 09:54:06 -0000 Hi Neel, On Mon, Feb 24, 2014 at 05:40:38PM -0800, Neel Natu wrote: > Hi Paul, > > On Sun, Feb 23, 2014 at 7:57 AM, Paul Schenkeveld wrote: > > Hi, > > > > I tried to get bhyve to run on my home server but got consistent > > crashes as soon as I touch the network. > > > > Hardware: Supermicro X9SPV-F3610ME-O (embedded i5) with 16GB RAM > > > > Host kernels (+userland of the same version): > > 10.0 RC5 > > 10.0 RELEASE > > 10 STABLE > > > > The kernel has been stripped (unneeded drivers and stuff like DEBUG, > > AUDIT, CAPABILITIES, MAC removed) > > > > Guest operating systems: > > 9.2 RELEASE > > 10.0 RELEASE > > > > Procedure: > > # ifconfig tap0 create up > > # ifconfig bridge0 create up > > # ifconfig bridge0 addm tap0 addm em0 > > # kldload vmm > > # mkdir /vm/test > > # cd /vm/test > > # ln -s /images/FreeBSD-10.0-RELEASE-amd64-disc1.iso release.iso > > # sh vmrun.sh -c 1 -m 512 -t tap0 > > > > > > > /etc/ttys> > > > > > > > > At the point where DHCP sends the first query I get a segmentation fault > > and the guest stops. > > > > I can provoke the same segfault by not putting the DHCP line into > > rc.conf and trying to ifconfig vtnet0 or by adding tap0 to bridge0 in > > the host after the guest has stopped. > > > > Although it worked perfectly fine with bhyve a couple of months ago > > (with an older build of 10.0-CURRENT that I don't have here anymore), I > > suspected the hardware and tried the same on my Macbook Pro with Vmware > > Fusion with a 10.0 RELEASE guest. > > > > The stripped 10 STABLE guest segfaults in the same way, the other two > > both work. > > > > I bought the home server for building software for older releases > > (mostly FreeBSD 8 and 9), my Macbook with Fusion is too slow and often > > in use for other purposes. > > > > Is this behaviour seen by others? > > > > I can provide more details or core dumps if that helps locating/solving > > the problem. > > > > I am not clear about what exactly is segfaulting: is it the bhyve > process or the guest kernel? Do you have a screenshot and/or a > backtrace of the segfaulting entity? It looks like it's the bhyve process. I'm travelling at the moment, if can get to the machine from the hotel I'll try and send you a screen shot tonight, otherwise when I'm back at the end of the week. Thank you. > best > Neel With kind regards, Paul Schenkeveld From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 27 14:14:17 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8756A425 for ; Thu, 27 Feb 2014 14:14:17 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 54F271D1A for ; Thu, 27 Feb 2014 14:14:17 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so2491269pde.10 for ; Thu, 27 Feb 2014 06:14:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=LHR0bwEMCkso9PHFj6NVTAVYx6cLeaTJJHgA7TUKOqs=; b=IL/QIh9CmHf7c7x8Im171TNTFOqC95DxSMnDmDumnNCG/7eDkyHZTHCf+FYwmmMGVZ 8k8j8hpeQto14dPGOBmcP28iy5UWuko1KgQuCfQZZ26U4k2iw7aCbBf8rebo1vDtWUGc B66fNKtqn/CHrmullZcgw98ywZLt/XTz9NYXw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=LHR0bwEMCkso9PHFj6NVTAVYx6cLeaTJJHgA7TUKOqs=; b=TzButWHLLpYJ4Y+cnVN3xNRy1uQl6I+G9D+2iEW33BzCsg5dhAnureCJV1q0agscBy g3A2UOIoRwkZBFxoRhRQ98DP45odKV6kKDmM89vcKQnFE5CNEWC0KHJsCaJJnytLL2/f hjjkeu0II/06Eh7ebSzWxER9lN02R4XJztqydPZsaAnyUaUaXDtI3Lu1dyaLxkM8zGqC rgSwfU2dH3jokyX15GWnkVfegC4xA6YYZggF2p2SwOadUTROo9j67AAdCbFN9cdQmEPt o+zInaZm0LJteou60DlCyCfjzIvDzzrwTgjGj8LcJatUy+2aMkGg81dEpdJFnSg9OFHu Ac5A== X-Gm-Message-State: ALoCoQlThHViGuyblB0DsMKUFlH41A2FulGPvhzvd2I/qBOPYSLBK70l9vGvZZxIJvW9xvMSc0CB X-Received: by 10.68.194.97 with SMTP id hv1mr13507416pbc.162.1393510456728; Thu, 27 Feb 2014 06:14:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.68.29.100 with HTTP; Thu, 27 Feb 2014 06:13:36 -0800 (PST) In-Reply-To: <52E63047.3040709@callfortesting.org> References: <52E63047.3040709@callfortesting.org> From: Takuya ASADA Date: Thu, 27 Feb 2014 16:13:36 +0200 Message-ID: Subject: Re: CFT: vmrc Virtual Machine rc script To: Michael Dexter Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 14:14:17 -0000 Hi Michael, I tried openbsd6, ubuntu7, centos9 by remove comment out on install.sh, and did provision command like this: $ sudo /usr/local/etc/rc.d/vm provision openbsd6 Then I get an error: vm: provisioning VM openbsd6 /usr/local/vm//openbsd6/openbsd6.conf: 21: Syntax error: "(" unexpected Same as another images. Where this error coming from? Note: I'm on FreeBSD-CURRENT, not 10.0. 2014-01-27 12:09 GMT+02:00 Michael Dexter : > > Hello all, > > For the one-week anniversary of bhyve's release, I wish to share the > next iteration of my virtual machine scripts that have evolved from a > 2005 jail rc/disk image framework to a FreeBSD 9.0 bhyve binary and VM > build script to a 10.* sequential provision script. It's primary purpose > is to facilitate the exposure of bugs in bhyve and related FreeBSD > components but you may find it useful for continuous VM use or for > integration into your own scripts. > > Because bhyve is only the latest multiplicity solution on FreeBSD, this > rc script includes basic jail and qemu support and could include chroot, > simh and GXemul support. Of these, QEMU is intended to help prepare OSs > like OpenBSD that require VirtIO and console adjustments prior to > booting under bhyve. > > Synopsis: > > Unpack vmrc.tar > Run 'sh install.sh' or perform the steps manually > Verify your host_nic and host_zpool in /usr/local/etc/vm.conf > /usr/local/etc/rc.d/vm (Show usage) > /usr/local/etc/rc.d/vm provision vm0 (Note: /usr/local/vm/vm0/vm0.conf) > (Fetches OS install media, formats disk, installs to disk) > /usr/local/etc/rc.d/vm start vm0 (Loads and boots vm0) > > Alternatively: > /usr/local/etc/rc.d/vm iso vm0 (Fetches and boots and ISO image) > > "ISO" mode will prepare a disk image to install the OS to and includes > fetch support for FreeBSD, Ubuntu and CentOS. > > Manual scripts exist in install/ to fetch FreeNAS, pfSense and OpenBSD. > > Status: > > Know issues and known desired features are at the end of > instructions.txt plus various "fix:" notes throughout the script. This > is not mature enough for packaging yet but does some things quite > reliably and is ready for feedback. Because development is ongoing, I > invite you to mail me about issues directly and I can post it to > something like GitHub if there is interest. > > Please read the instructions.txt to avoid FAQ's and I have done my best > to make the script itself readable. > > > Site: http://bhyve.org/vmrc/ > > Download: http://bhyve.org/vmrc/vmrc.tar (latest version) > > > Thank you Neel, Peter, John and Tycho for your hard work on bhyve. > > Thank you Devin for greatly improving this script and Allan at > ScaleEngine for testing it and providing resources for bhyve's overall > improvement. > > > Thank you for testing bhyve! > > Michael > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 27 14:25:51 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5420FBA5 for ; Thu, 27 Feb 2014 14:25:51 +0000 (UTC) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 103331E3E for ; Thu, 27 Feb 2014 14:25:50 +0000 (UTC) Received: by mail-qg0-f51.google.com with SMTP id q108so5883314qgd.10 for ; Thu, 27 Feb 2014 06:25:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=HOHn54SpoT55CJGxru3VANs3kSGysNm9oltUEDnNm2g=; b=U/rPRx6OHYUW/+lcp3orVj8pevmO+cGi4+Ev25EEWXMuCWhMml5OtPUjx4KMcN26D9 NxfdorykY3Quu9GKDrrtTFjFct2ZII6Nr7HxfalOf3Y09KSjSDM5UUv3GKK7lirOF8sX FTDo4Ht5htrP9biOhIJ3nPH06KLkm8lxO26SqNFdneh0E8k16Q7ev/YvJ/1XDXnJGNMn 09ZBfVvAXlEe4KRKWMmWYDMF6YexL15goei6bHsVeShzBXDYsKjqwo5I0F25pg9Uvgzk OE5UKJ1V4GSi91lfme6RWy+RIHRkHSVv4GcWT7RtKEpOQyATXghSeIGryK35Z/Z3aQhi 5UIQ== MIME-Version: 1.0 X-Received: by 10.140.36.107 with SMTP id o98mr7467677qgo.25.1393511150210; Thu, 27 Feb 2014 06:25:50 -0800 (PST) Received: by 10.96.70.74 with HTTP; Thu, 27 Feb 2014 06:25:50 -0800 (PST) Received: by 10.96.70.74 with HTTP; Thu, 27 Feb 2014 06:25:50 -0800 (PST) In-Reply-To: References: <52E63047.3040709@callfortesting.org> Date: Thu, 27 Feb 2014 09:25:50 -0500 Message-ID: Subject: Re: CFT: vmrc Virtual Machine rc script From: Matt Donovan To: Takuya ASADA Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 14:25:51 -0000 The error is coming from the openbsd.conf line 21 On Feb 27, 2014 8:14 AM, "Takuya ASADA" wrote: > > Hi Michael, > > I tried openbsd6, ubuntu7, centos9 by remove comment out on install.sh, and > did provision command like this: > $ sudo /usr/local/etc/rc.d/vm provision openbsd6 > > Then I get an error: > vm: provisioning VM openbsd6 > /usr/local/vm//openbsd6/openbsd6.conf: 21: Syntax error: "(" unexpected > > Same as another images. > > Where this error coming from? > > Note: I'm on FreeBSD-CURRENT, not 10.0. > > > 2014-01-27 12:09 GMT+02:00 Michael Dexter : > > > > > Hello all, > > > > For the one-week anniversary of bhyve's release, I wish to share the > > next iteration of my virtual machine scripts that have evolved from a > > 2005 jail rc/disk image framework to a FreeBSD 9.0 bhyve binary and VM > > build script to a 10.* sequential provision script. It's primary purpose > > is to facilitate the exposure of bugs in bhyve and related FreeBSD > > components but you may find it useful for continuous VM use or for > > integration into your own scripts. > > > > Because bhyve is only the latest multiplicity solution on FreeBSD, this > > rc script includes basic jail and qemu support and could include chroot, > > simh and GXemul support. Of these, QEMU is intended to help prepare OSs > > like OpenBSD that require VirtIO and console adjustments prior to > > booting under bhyve. > > > > Synopsis: > > > > Unpack vmrc.tar > > Run 'sh install.sh' or perform the steps manually > > Verify your host_nic and host_zpool in /usr/local/etc/vm.conf > > /usr/local/etc/rc.d/vm (Show usage) > > /usr/local/etc/rc.d/vm provision vm0 (Note: /usr/local/vm/vm0/vm0.conf) > > (Fetches OS install media, formats disk, installs to disk) > > /usr/local/etc/rc.d/vm start vm0 (Loads and boots vm0) > > > > Alternatively: > > /usr/local/etc/rc.d/vm iso vm0 (Fetches and boots and ISO image) > > > > "ISO" mode will prepare a disk image to install the OS to and includes > > fetch support for FreeBSD, Ubuntu and CentOS. > > > > Manual scripts exist in install/ to fetch FreeNAS, pfSense and OpenBSD. > > > > Status: > > > > Know issues and known desired features are at the end of > > instructions.txt plus various "fix:" notes throughout the script. This > > is not mature enough for packaging yet but does some things quite > > reliably and is ready for feedback. Because development is ongoing, I > > invite you to mail me about issues directly and I can post it to > > something like GitHub if there is interest. > > > > Please read the instructions.txt to avoid FAQ's and I have done my best > > to make the script itself readable. > > > > > > Site: http://bhyve.org/vmrc/ > > > > Download: http://bhyve.org/vmrc/vmrc.tar (latest version) > > > > > > Thank you Neel, Peter, John and Tycho for your hard work on bhyve. > > > > Thank you Devin for greatly improving this script and Allan at > > ScaleEngine for testing it and providing resources for bhyve's overall > > improvement. > > > > > > Thank you for testing bhyve! > > > > Michael > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " freebsd-virtualization-unsubscribe@freebsd.org" The error is due to a syntax errror in your openbs.conf. Can you post your configuration as it appears you have an extra ). From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 27 14:38:57 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 233AE700 for ; Thu, 27 Feb 2014 14:38:57 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E33A010CD for ; Thu, 27 Feb 2014 14:38:56 +0000 (UTC) Received: by mail-pd0-f176.google.com with SMTP id r10so2494057pdi.7 for ; Thu, 27 Feb 2014 06:38:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=S/vHGh/sH8ADxx75OpzCsMhQSxZ0RPG7smkiHDPt1kk=; b=ov3B51CvTBX0hy8zAx5Zas2ZvsUOicvKYgbSnvQZezCP+9SupDtiSAjYZCCnmvxOn+ z1/GOsLobTUoTSjze34KMSCLWYkL31lcJEzKea8qwELFd5S7rqlk4bOY0yEQOhRGTANE p5os/d6WKYphHpqnq4+q9XJnSr+pgA/E1lNKE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=S/vHGh/sH8ADxx75OpzCsMhQSxZ0RPG7smkiHDPt1kk=; b=hNdGM/du7+enBsJD1LZ1CVDqW+Pm+8Lfinw2u36b484Un0/4yDUe3E548+GogNNopQ 0v6E5Qr20P3uaT+ziCq09AAml7IPGoIPtou00k6rWoNLJA1eDNiTI+e8adb63kyGHN9S IcUNkGGgJXxMshqXgAE9n0n0mFTYfRaZt8JS6Lo+e8kjWohhWcM4iqvPGrndKaqVxzoY EPlxULedDCkH3B+8BW+5EEV+C9nt3IS6CuEVNu4zcjt7S2Kbxb+ih1e1JaQpFfk+K7Nj dKZm/KCiDQaXTLzZ3w7dCdw6d4CEIVgvYsOGa6QN4Oaw++JT09RuQPzS5hUHKfzVAcuW bU5w== X-Gm-Message-State: ALoCoQnwnmFIEspN8L+94RM4yDTIaSYBoCX/o16/UnRJOSZUjXZuJqEheDd4/Ru5xgEatYe7ooR6 X-Received: by 10.66.145.166 with SMTP id sv6mr15775411pab.31.1393511936472; Thu, 27 Feb 2014 06:38:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.68.29.100 with HTTP; Thu, 27 Feb 2014 06:38:16 -0800 (PST) In-Reply-To: References: <52E63047.3040709@callfortesting.org> From: Takuya ASADA Date: Thu, 27 Feb 2014 16:38:16 +0200 Message-ID: Subject: Re: CFT: vmrc Virtual Machine rc script To: Matt Donovan Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 14:38:57 -0000 2014-02-27 16:25 GMT+02:00 Matt Donovan : > The error is coming from the openbsd.conf line 21 > The error is due to a syntax errror in your openbs.conf. Can you post your > configuration as it appears you have an extra ). > It's true, but I even never changes it. Which means the configuration file is may broken by default. This line: grub_boot_cmd="echo -e \"kopenbsd -h com0 (hd0,openbsd1)/bsd\nboot\" | grub-bhyve -r hd0 -m "${host_vmroot}/${vm_name}/device.map" -M "$vm_ram" "$vm_name"" From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 27 14:59:01 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18E6A747 for ; Thu, 27 Feb 2014 14:59:01 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C9E6B1303 for ; Thu, 27 Feb 2014 14:59:00 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so4053192qae.31 for ; Thu, 27 Feb 2014 06:58:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=ZSLtwDlu1yn4eN99jkmeBBCChJubBfzvfDYWLlqFkIk=; b=OwGBWJFTTnjR1dbCynhpnXM2PiQBXChOKxPb0u5V7WTXjJxdSLMQVZuPWQEOS9uyzA cjm313LRZ+VnPkfR86ocNcJ8K7t92xpAGWfxoFZiyuozRQz4yDvNPR1C6ZvBhMm2vuq7 mJMgsVVhESw9UTPP8UmllvgRzTs9qmHFMXSLZZQfY4OXpnxXbUyZorc1AkqlRpADnzyS VPkxM/71S9tIuT6pyn2KJdN7cy/62oGNlmoW4XgoCIlaoSVdbrJIXZwOIIE2iHFKzPrs z6QyvZ1Yq8sIEluTWKjPyyKDZs+ogXhD2hptf+9Y/OqTimCFspqKMR07k+yKv7u6l/Bc HhIw== X-Gm-Message-State: ALoCoQmXcAmN+UmLJED4ZpFJFZodCh9pflurP9Kha201q/5ts28wSAwRJIFYYaV602CjQoraB82q MIME-Version: 1.0 X-Received: by 10.224.96.138 with SMTP id h10mr10623164qan.46.1393513134473; Thu, 27 Feb 2014 06:58:54 -0800 (PST) Received: by 10.140.80.45 with HTTP; Thu, 27 Feb 2014 06:58:54 -0800 (PST) X-Originating-IP: [2001:470:28:12b::3] Date: Thu, 27 Feb 2014 15:58:54 +0100 Message-ID: Subject: bhyve core dump... From: Andrea Brancatelli To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 14:59:01 -0000 Hello guys. I was installing mysql56 in a freebsd10 inside freebsd 10 and while doing the install the machine bombed out. In the console this is what I got: Assertion failed: (aior !=3D NULL), function ahci_handle_dma, file /usr/src/usr.sbin/bhyve/pci_ahci.c, line 445. Abort trap (core dumped) The VM was calculating the SHA256 of the distfile when it bombed out. Nothing else was going on inside the VM at that time. I even have the .core if it matters. Any idea? --=20 *Andrea BrancatelliSchema 31 S.r.l. - Socio UnicoResponsabile ITROMA - FIRENZE - PALERMO ITALYTel: +39. 06.98.358.472* *Cell: +39 331.2488468Fax: +39. 055.71.880.466Societ=C3=A0 del Gruppo SC31 ITALIA* From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 01:14:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E94F3B8 for ; Fri, 28 Feb 2014 01:14:41 +0000 (UTC) Received: from mail-qa0-f50.google.com (mail-qa0-f50.google.com [209.85.216.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2EA431B88 for ; Fri, 28 Feb 2014 01:14:40 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id cm18so38139qab.9 for ; Thu, 27 Feb 2014 17:14:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=ioSiWDeBAJRfAN+zcDYFA3sfSDP8V/yyFqiDQZWVhRA=; b=WByCQmCJS3q3m1nUD0b7JqqXN4NnIzzhi/pK1/s62aQk6aGMHHwp28xzBr9zm9xMX/ KefWvrZjmSak9a1RDVG0H8HLLjUHuexwEP5w90NcwdohY+V1cmc03GVk5wQPfFN6gjQS 1EmXFnrol0LGkwp3wrpv8gNjgoGcn7V2YGXOXD4bTe7AEnhur03iplZAUc2J9WZ5hjNl xiGUBjqKaDRuRRHBrGKBkIQAwklstImiM8ILxsJagY2CGqffXtkr+898wx4TlUjSmQTO E8ZEqDDybt3uKahUXgIixtUGitemtM1DbRdTamtqqHOAhEDA0nTkNrYPc1qYLYnw/ncS 7IGQ== X-Gm-Message-State: ALoCoQlhSEV83cex33h5Kvcooa89KGQPSZMeS3RHMR9IZvq5q9xoAXo/zizpGO6HSghu7tO0vLJS MIME-Version: 1.0 X-Received: by 10.140.19.212 with SMTP id 78mr76519qgh.84.1393549709187; Thu, 27 Feb 2014 17:08:29 -0800 (PST) Received: by 10.140.80.45 with HTTP; Thu, 27 Feb 2014 17:08:29 -0800 (PST) X-Originating-IP: [95.252.145.25] Date: Fri, 28 Feb 2014 02:08:29 +0100 Message-ID: Subject: bhyvectl From: Andrea Brancatelli To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 01:14:41 -0000 I just throw away at least one hour because I was running bhyvectl destroy --vm=something instead of bhyvectl --destroy --vm=something. Bhyvectl just silently ignored the wrong "destroy" instead of "--destroy". Maybe it's worth printing out something in such circumstances??? :-) Thanks... -- *Andrea Brancatelli* From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 01:40:02 2014 Return-Path: Delivered-To: freebsd-virtualization@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5BEA6FD8 for ; Fri, 28 Feb 2014 01:40:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2EB841D3E for ; Fri, 28 Feb 2014 01:40:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1S1e23O013705 for ; Fri, 28 Feb 2014 01:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1S1e1Ua013704; Fri, 28 Feb 2014 01:40:01 GMT (envelope-from gnats) Date: Fri, 28 Feb 2014 01:40:01 GMT Message-Id: <201402280140.s1S1e1Ua013704@freefall.freebsd.org> To: freebsd-virtualization@FreeBSD.org Cc: From: dfilter@FreeBSD.ORG (dfilter service) Subject: Re: kern/183835: commit references a PR X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dfilter service List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 01:40:02 -0000 The following reply was made to PR kern/183835; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/183835: commit references a PR Date: Fri, 28 Feb 2014 01:35:31 +0000 (UTC) Author: rodrigc Date: Fri Feb 28 01:35:24 2014 New Revision: 262594 URL: http://svnweb.freebsd.org/changeset/base/262594 Log: MFC r262142: In ue_attach_post_task(), initialize curvnet to vnet0 before calling if_attach(). Before this patch, curvnet was NULL. When the VIMAGE kernel option is enabled, this eliminates kernel panics when USB ethernet devices are plugged in. PR: 183835 Submitted by: Hiroo Oono Modified: stable/9/sys/dev/usb/net/usb_ethernet.c Directory Properties: stable/9/ (props changed) Modified: stable/9/sys/dev/usb/net/usb_ethernet.c ============================================================================== --- stable/9/sys/dev/usb/net/usb_ethernet.c Fri Feb 28 01:33:03 2014 (r262593) +++ stable/9/sys/dev/usb/net/usb_ethernet.c Fri Feb 28 01:35:24 2014 (r262594) @@ -207,6 +207,7 @@ ue_attach_post_task(struct usb_proc_msg sysctl_ctx_init(&ue->ue_sysctl_ctx); error = 0; + CURVNET_SET_QUIET(vnet0); ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { device_printf(ue->ue_dev, "could not allocate ifnet\n"); @@ -254,6 +255,8 @@ ue_attach_post_task(struct usb_proc_msg if (ifp->if_capabilities & IFCAP_VLAN_MTU) ifp->if_hdrlen = sizeof(struct ether_vlan_header); + CURVNET_RESTORE(); + snprintf(num, sizeof(num), "%u", ue->ue_unit); ue->ue_sysctl_oid = SYSCTL_ADD_NODE(&ue->ue_sysctl_ctx, &SYSCTL_NODE_CHILDREN(_net, ue), @@ -267,6 +270,7 @@ ue_attach_post_task(struct usb_proc_msg return; fail: + CURVNET_RESTORE(); free_unr(ueunit, ue->ue_unit); if (ue->ue_ifp != NULL) { if_free(ue->ue_ifp); _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 07:02:11 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D12163F7 for ; Fri, 28 Feb 2014 07:02:11 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B750817CF for ; Fri, 28 Feb 2014 07:02:11 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s1S724Xl007175 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 27 Feb 2014 23:02:05 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <5310346B.4000001@monkeybrains.net> Date: Thu, 27 Feb 2014 23:02:03 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: lazy bhyve doesn't boot until I conenct to console Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 07:02:11 -0000 I'm seeing something odd... I am running this command: bhyve -c 4 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s 2,virtio-net,tap1 -s 3,virtio-blk,/dev/zvol/data/vm1 -l com1,/dev/nmdm1A vm1 but I the VM is being lazy... The second I run 'tip vm1' I see ubuntu boot and then it comes up. bhyve doesn't boot until I connect to the nmdm device. FYI, I have this in /etc/remote: vm1:dv=/dev/nmdm1B:br#9600:pa=none: Maybe it is my start up script, which is getting a bit bloated. I failed at a simple start up script. Comments welcome: https://www.monkeybrains.net/support/bhyve.rc Rudy From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 07:15:46 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4358DA87 for ; Fri, 28 Feb 2014 07:15:46 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 04F4118B7 for ; Fri, 28 Feb 2014 07:15:45 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D2D58121CD; Fri, 28 Feb 2014 17:08:49 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSE35158 (AUTH peterg@ptree32.com.au); Fri, 28 Feb 2014 17:08:49 +1000 Message-ID: <531035FD.4030704@freebsd.org> Date: Thu, 27 Feb 2014 23:08:45 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: "Rudy (bulk)" Subject: Re: lazy bhyve doesn't boot until I conenct to console References: <5310346B.4000001@monkeybrains.net> In-Reply-To: <5310346B.4000001@monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 07:15:46 -0000 Hi Rudy, > I'm seeing something odd... I am running this command: > > bhyve -c 4 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s > 2,virtio-net,tap1 -s 3,virtio-blk,/dev/zvol/data/vm1 -l com1,/dev/nmdm1A > vm1 > but I the VM is being lazy... A number of people have reported this, and looking at it, I think it's a bug in bhyve. nmdm attempts to simulate modem control (not quite correctly - run back-back cu's to see some of the behaviour), but also bhyve doesn't open the device non-blocking. This results in the open() syscall waiting for simulated carrier-detect to arrive, which doesn't happen until the other end of the nmdm is opened. A fix should be showing up shortly. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 07:45:20 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F55A886; Fri, 28 Feb 2014 07:45:20 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D4F01B16; Fri, 28 Feb 2014 07:45:19 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s1S7jJMx010436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 27 Feb 2014 23:45:19 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <53103E8E.80709@monkeybrains.net> Date: Thu, 27 Feb 2014 23:45:18 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Peter Grehan Subject: [SOLVED] Re: lazy bhyve doesn't boot until I conenct to console References: <5310346B.4000001@monkeybrains.net> <531035FD.4030704@freebsd.org> In-Reply-To: <531035FD.4030704@freebsd.org> X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 07:45:20 -0000 On 2/27/14, 11:08 PM, Peter Grehan wrote: > Hi Rudy, > > >> I'm seeing something odd... I am running this command: >> >> bhyve -c 4 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s >> 2,virtio-net,tap1 -s 3,virtio-blk,/dev/zvol/data/vm1 -l com1,/dev/nmdm1A >> vm1 >> but I the VM is being lazy... > > A number of people have reported this, and looking at it, I think > it's a bug in bhyve. > > nmdm attempts to simulate modem control (not quite correctly - run > back-back cu's to see some of the behaviour), but also bhyve doesn't > open the device non-blocking. This results in the open() syscall > waiting for simulated carrier-detect to arrive, which doesn't happen > until the other end of the nmdm is opened. > I have a fix: echo into the nmdm after booting VM. Added this line to bhyve.rc echo >> ${bhyve_nmdmBdev} Rudy From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 08:00:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A71BBE47 for ; Fri, 28 Feb 2014 08:00:15 +0000 (UTC) Received: from mail-pa0-f42.google.com (mail-pa0-f42.google.com [209.85.220.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 77CC31C13 for ; Fri, 28 Feb 2014 08:00:15 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id kl14so435807pab.29 for ; Fri, 28 Feb 2014 00:00:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=BwfbKfpV4GcUiTy7dmZdKgeWyAiNmwJWICshuY6E78U=; b=NYSOBDnybWm3BLRvQppW6S6jTQGeyBReDn4Czhjrf9Y2UAODzPN2XV98rwIEYnp7Wq 7NMLtJxNZ/aaj5TfGHy6VRp9MYcMfxfTW+PPe70LNO8i3b0SaB5ylBVmMD1ACnblp2gs UwEI5wJbPe+XW8abfzbibGVJ015Zmzw4CNSx/CZeGbb/qbZOivsLg45EKg1r7P5l3Yrt PLOLF23lrt9M3O78+wUBH6dbdaZGpVRyDrVDO2an8KYrPBc5MbEWXjEY+AzCmouUajsM 9RLPgT4PfibAutly0PDpa9xntFdznYff+k0/Fi+xeg6wRubCrxvaVDw7ohsP1plZCvJp GR/g== X-Gm-Message-State: ALoCoQnKirNeC7psUswNE4XyZ2ALpFoQEQTzaMODlE6CFwdzdS0oNpzkCLA00tzaewYrwnjspxcx X-Received: by 10.68.203.102 with SMTP id kp6mr2020972pbc.14.1393574409764; Fri, 28 Feb 2014 00:00:09 -0800 (PST) Received: from Michaels-MacBook-Pro.local (c-98-246-202-204.hsd1.or.comcast.net. [98.246.202.204]) by mx.google.com with ESMTPSA id xs1sm7683293pac.7.2014.02.28.00.00.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 28 Feb 2014 00:00:09 -0800 (PST) Message-ID: <53104207.4000602@callfortesting.org> Date: Fri, 28 Feb 2014 00:00:07 -0800 From: Michael Dexter User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Takuya ASADA Subject: Re: CFT: vmrc Virtual Machine rc script References: <52E63047.3040709@callfortesting.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 08:00:15 -0000 On 2/27/14 6:13 AM, Takuya ASADA wrote: > Then I get an error: > vm: provisioning VM openbsd6 > /usr/local/vm//openbsd6/openbsd6.conf: 21: Syntax error: "(" unexpected Note that because the OpenBSD VM is using a pre-made disk image, it will not support "provision". You simply want "start". > Same as another images. > > Where this error coming from? >From a typo on line 6: :( vm_console=""nmdm" rather than: vm_console="nmdm" This is now fixed and thank you for testing it! Michael From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 28 08:55:13 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79A9148F for ; Fri, 28 Feb 2014 08:55:13 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 600B611AE for ; Fri, 28 Feb 2014 08:55:13 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s1S8tCr0017529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 28 Feb 2014 00:55:12 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <53104EF0.9080902@monkeybrains.net> Date: Fri, 28 Feb 2014 00:55:12 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: NEEDS TESTING - rc script for bhyve Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 08:55:13 -0000 OK, I did test it a bit. Really, I wanted something short to automate booting, but I ran into some issues that ballooned up the script. Things I put into the script: 0 - does no provisioning, I only want a start/stop script 1 - adds vmtab for configuring VMs 2 - supports grub loader 3 - uses nmdm 4 - runs VM with a heatbeat and automatically restarts unless the 'stop' is run 5 - 'stop' sends a graceful shutdown and the heartbeat loop doesn't restart as the pidfile is nuked 6 - catches stalled bhyveloader and kills it https://www.monkeybrains.net/support/bhyve.rc Rudy From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 1 00:45:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7FAFD92 for ; Sat, 1 Mar 2014 00:45:33 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A67071B68 for ; Sat, 1 Mar 2014 00:45:33 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id j107so4263275qga.8 for ; Fri, 28 Feb 2014 16:45:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=g7MBW/RJfm6QYPU/WsYfRQLYk3ScApaAc8a4URWAoUE=; b=c6haX9U7mM8FVs69DM2KVl6SuMAD40dZm+lCYpiWbft6/OUT5/BCRnEf83YQKYYh0O 6JepUWqoyW0I2kFtEIoO7+kP/kND+AwleSRGQ/0y/Fx6yLhDnoxDKjMO15WUQi7ofHZd yG32v8KfmAriUAEMU4bgKjweYG4+9qrKqKriYcL8FnqRCLlEY2o1H7yebYnO6ARUL5Zt c0347sRe71/shY96K/twFqVaySfAvkfDki/Ky4tTwkUoI2NUtbTOD17xcgo+v0+bShyZ emZxUKZgybGocLWaod7PLtpWOXlbhkzBrUdkvWu2tDkde78pY6MYERY2Oi/L0846z47p Iqvg== MIME-Version: 1.0 X-Received: by 10.224.132.65 with SMTP id a1mr7819755qat.69.1393634732760; Fri, 28 Feb 2014 16:45:32 -0800 (PST) Received: by 10.140.87.71 with HTTP; Fri, 28 Feb 2014 16:45:32 -0800 (PST) In-Reply-To: References: Date: Fri, 28 Feb 2014 16:45:32 -0800 Message-ID: Subject: Re: bhyve core dump... From: Neel Natu To: Andrea Brancatelli Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2014 00:45:34 -0000 Hi Andrea, On Thu, Feb 27, 2014 at 6:58 AM, Andrea Brancatelli wrote: > Hello guys. > > I was installing mysql56 in a freebsd10 inside freebsd 10 and while doing > the install the machine bombed out. In the console this is what I got: > > Assertion failed: (aior !=3D NULL), function ahci_handle_dma, file > /usr/src/usr.sbin/bhyve/pci_ahci.c, line 445. > Abort trap (core dumped) > > The VM was calculating the SHA256 of the distfile when it bombed out. > Nothing else was going on inside the VM at that time. > > I even have the .core if it matters. > The core file would greatly help if you can make it available. best Neel > Any idea? > > -- > > > > > *Andrea BrancatelliSchema 31 S.r.l. - Socio UnicoResponsabile ITROMA - > FIRENZE - PALERMO ITALYTel: +39. 06.98.358.472* > > *Cell: +39 331.2488468Fax: +39. 055.71.880.466Societ=E0 del Gruppo SC31 > ITALIA* > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 1 07:11:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C3A8D6D for ; Sat, 1 Mar 2014 07:11:15 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 08A401096 for ; Sat, 1 Mar 2014 07:11:14 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id md12so1700656pbc.35 for ; Fri, 28 Feb 2014 23:11:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=q3YDKhYfw0AUA0sA9tqs/UgZewYtxM3h/UoesLFwyUU=; b=FlP3IbKTjNq+dplOL76yrKD1C3yDaDx3GlSQdg+k82tS9JKMalyQsdPo3E87D/g8/P ApXwQIqLnOvElqKRBaP7A8vGpjw40n1ocU1IPwTESiKvn8D7mnTnSvsIP9VNOgn3iboP /L5O1kuB2QdB/pjLNQ9EVersJtdnQ4pSTM9iE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=q3YDKhYfw0AUA0sA9tqs/UgZewYtxM3h/UoesLFwyUU=; b=TFkgZzqwonKciNiuzpIfortzROX8awfkxthwvx+PQn68H4cvbwgvl+JVtvfC9D1D71 hSaD15aGRrUEOJ+VcM4STXiyNFgMFnJDpbxhsaGNJoaN9j9UD4E9AX6g9MLQLff5rb2S LbLcKzohSzMQc63B3rP/kchJTXBBrFzD8vM+Mft4nlXwhymtUXxuW0e1xIlTxW7dt+Zg SaR0hFP2v6vnOBSvtEB1A55VvVCwmPMdpvjVZ/UDj50jCS0TI7OhlTVIT0ygjHzq2mnl 2HKKdqDqUT5nGssU9BxAgSt91YbXnBKIgbhnyi65rBQquqrokSoryEoZ+bnx/gomGAP1 /VZg== X-Gm-Message-State: ALoCoQnF5ikygoxdOofByPbGR90TYCtzM86Onyr+gpVr3upSEkTrKWoVzQ4aC8Mysvc9shnxCar8 X-Received: by 10.68.135.137 with SMTP id ps9mr22303pbb.160.1393657874533; Fri, 28 Feb 2014 23:11:14 -0800 (PST) MIME-Version: 1.0 Received: by 10.68.29.100 with HTTP; Fri, 28 Feb 2014 23:10:34 -0800 (PST) In-Reply-To: <53104207.4000602@callfortesting.org> References: <52E63047.3040709@callfortesting.org> <53104207.4000602@callfortesting.org> From: Takuya ASADA Date: Sat, 1 Mar 2014 09:10:34 +0200 Message-ID: Subject: Re: CFT: vmrc Virtual Machine rc script To: Michael Dexter Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2014 07:11:15 -0000 2014-02-28 10:00 GMT+02:00 Michael Dexter : > On 2/27/14 6:13 AM, Takuya ASADA wrote: > > Then I get an error: > > vm: provisioning VM openbsd6 > > /usr/local/vm//openbsd6/openbsd6.conf: 21: Syntax error: "(" unexpected > > Note that because the OpenBSD VM is using a pre-made disk image, it will > not support "provision". You simply want "start". > > > Same as another images. > > > > Where this error coming from? > > From a typo on line 6: :( > > vm_console=""nmdm" rather than: > vm_console="nmdm" > > Oh I see. > This is now fixed and thank you for testing it! > I'll try it again :-) From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 2 10:56:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE4E5E9A for ; Sun, 2 Mar 2014 10:56:41 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 89EE211F0 for ; Sun, 2 Mar 2014 10:56:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 892CD989304 for ; Sun, 2 Mar 2014 11:49:49 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 99Tnl7-NA-AK; Sun, 2 Mar 2014 11:49:48 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id A4F2A989302; Sun, 2 Mar 2014 11:49:48 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it A4F2A989302 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393757388; bh=zB5BXtX/hc5SZk5wtrSxcXbA7spxoR+GLNoJ/ALmxG0=; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type; b=BLZengEhBmdgyjy1BS4yEl5a1BjEAum3yjGcjGllDqGJsr0usTRweC/erjnmKe5zN 2QHknxhOedh+BkF/yoNYkvnOZWpqsC7fYWARX57brlgi/t+JpBrEzOLmq7PF5U3SVU tDp+7BEQVWj9NcrJ0wDzD+VvgNoka0Qb8qbpcHSE= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id wVRSiSkSTLvW; Sun, 2 Mar 2014 11:49:48 +0100 (CET) Received: from webmail.cost.it (webmail.cost.it [192.168.222.20]) by webmail.cost.it (Postfix) with ESMTP id 4997A989303; Sun, 2 Mar 2014 11:49:48 +0100 (CET) Date: Sun, 2 Mar 2014 11:49:48 +0100 (CET) From: Giovanni Mattera To: freebsd-virtualization@freebsd.org Message-ID: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> In-Reply-To: <1478682364.5202322.1393757298583.JavaMail.zimbra@cost.it> Subject: freebsd 10.0 not work carp protocol on Hyper-v MIME-Version: 1.0 X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF27 (Win)/8.0.6_GA_5922) Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: w1iYmb8i77+CCVNy/FC4kgfjKueHkA== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Maurizio Marini X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 10:56:42 -0000 good morning I installed freebsd 10.0 on Hyper -V Windows Server 2012 R2. I would like to enable the CARP protocol but is still in a state of init . The error in dmessage is " ifa_del_loopback_route : deletion failed: 48" . How do you configure CARP on Hyper -V ? I've used both the card " Legacy Network " and " Network" but the result is the same. We are available to pay for this. Thank you. Giovanni Mattera ------------------------------------------------- CoST - Computers Services and Technologies s.r.l. Via Giuseppe Longhi 13 20137 Milano MI tel: +39 02454461 fax: +39 0245446333 cell:+393351217481 http://www.cost.it Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 2 14:13:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B457A92 for ; Sun, 2 Mar 2014 14:13:42 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8E04E1481 for ; Sun, 2 Mar 2014 14:13:42 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id p10so2668835pdj.12 for ; Sun, 02 Mar 2014 06:13:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=kS9IZoTjeEHMEzQUlApV4np/QrL5Zkimqe6uMKoGdFk=; b=VsdcjIjqj7R4U2aDxFSRBlhozd3vIt1oGcrIpUQP2xZn0ydiDWFD3kjIcA2QD5kp8u y58XKlCuT2RmPRJF0Z+Bm/B/KQZ9UKGzOSsocyAXoHzdcxFSoAzXDDk6ZGBrRSugm7Ch cB02hTTMHN2KVWo3AMkmBbZuUHREN05HpeOw4qpjJEAs/HJVOiqbdmpcA5dHpLV+WruZ WuaibOriE5K7TEaFZpX+yVC5jfHoImWKwZWlq60hhj4IWjnRZxX9fPC+tNqfngWiH0g9 ZuM1nGMN13mfoBQSxKRkJsjCIdGoM2/ShQJlg2iG81DZ3t+RL3Ubnm9meQfOWpOpDMyu o5cw== MIME-Version: 1.0 X-Received: by 10.66.148.134 with SMTP id ts6mr2801334pab.113.1393769621793; Sun, 02 Mar 2014 06:13:41 -0800 (PST) Received: by 10.68.155.38 with HTTP; Sun, 2 Mar 2014 06:13:41 -0800 (PST) Date: Sun, 2 Mar 2014 09:13:41 -0500 Message-ID: Subject: petitecloud 0.2.6 released From: Aryeh Friedman To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 14:13:42 -0000 Whats new: * API (with command line) * Hooks for micro pligins http://www.petitecloud.org -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 3 10:17:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7062C2D; Mon, 3 Mar 2014 10:17:33 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8960B2C7; Mon, 3 Mar 2014 10:17:33 +0000 (UTC) Received: from Julian-MBP3.local ([12.157.112.67]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s23AHJZ1073549 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 3 Mar 2014 02:17:22 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <531456AA.50203@freebsd.org> Date: Mon, 03 Mar 2014 02:17:14 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Giovanni Mattera , freebsd-virtualization@freebsd.org Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> In-Reply-To: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Maurizio Marini , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 10:17:33 -0000 On 3/2/14, 2:49 AM, Giovanni Mattera wrote: > good morning > I installed freebsd 10.0 on Hyper -V Windows Server 2012 R2. > I would like to enable the CARP protocol but is still in a state of init . > The error in dmessage is " ifa_del_loopback_route : deletion failed: 48" . > How do you configure CARP on Hyper -V ? > I've used both the card " Legacy Network " and " Network" but the result is the same. > We are available to pay for this. have you tried this kernel with real hardware? > Thank you. > > Giovanni Mattera > ------------------------------------------------- > CoST - Computers Services and Technologies s.r.l. > Via Giuseppe Longhi 13 > 20137 Milano MI > tel: +39 02454461 > fax: +39 0245446333 > cell:+393351217481 > http://www.cost.it > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. > La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. > Grazie. > > This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. > Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. > Thank you > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 3 11:06:55 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0CF2EF4B for ; Mon, 3 Mar 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ED9E895C for ; Mon, 3 Mar 2014 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s23B6sjb008705 for ; Mon, 3 Mar 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s23B6sQB008703 for freebsd-virtualization@FreeBSD.org; Mon, 3 Mar 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Mar 2014 11:06:54 GMT Message-Id: <201403031106.s23B6sQB008703@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 3 11:29:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C04D4E05; Mon, 3 Mar 2014 11:29:05 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C47AC60; Mon, 3 Mar 2014 11:29:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 9DC3C9896A2; Mon, 3 Mar 2014 12:28:04 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 0sddf2AAXezu; Mon, 3 Mar 2014 12:28:03 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 2304F987879; Mon, 3 Mar 2014 12:28:03 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 2304F987879 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393846083; bh=V12ljh/XR75Nx9iwlKYQ4X1xlfEMYHL8it1EIAmDDHk=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=A6/LQlSsLGIN9aaevicpZfb3xS1athK2vLfHSBq/AY9rKlGOAA5271dUMizyKNh2b u9IywCKy2XIyFo9x8V55KDtwpO1xOGJ6kAeK3UbcHLNaRAQn1sX1dATZM2VHTxau4t +/jtFYNHsJ58laXV/5wnSgilYGN1iJKmbBWMqyhs= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oAIr9h4_evDI; Mon, 3 Mar 2014 12:28:02 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id 9D69B986590; Mon, 3 Mar 2014 12:28:01 +0100 (CET) Date: Mon, 3 Mar 2014 12:28:50 +0100 From: Maurizio Marini To: Julian Elischer Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> In-Reply-To: <531456AA.50203@freebsd.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/LfyNV3t=vub5UTNXSfWYW=F"; protocol="application/pkcs7-signature" Cc: FreeBSD Net , Manuel Martini , freebsd-virtualization@freebsd.org, Giovanni Mattera X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 11:29:05 -0000 --Sig_/LfyNV3t=vub5UTNXSfWYW=F Content-Type: multipart/mixed; boundary="MP_/0Q1XfTp_K/vlErc8C/IdV4+" --MP_/0Q1XfTp_K/vlErc8C/IdV4+ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Mon, 03 Mar 2014 02:17:14 -0800 Julian Elischer wrote: > On 3/2/14, 2:49 AM, Giovanni Mattera wrote: > > good morning > > I installed freebsd 10.0 on Hyper -V Windows Server 2012 R2. > > I would like to enable the CARP protocol but is still in a state of ini= t . > > The error in dmessage is " ifa_del_loopback_route : deletion failed: 48= " . > > How do you configure CARP on Hyper -V ? > > I've used both the card " Legacy Network " and " Network" but the resul= t is > > the same. We are available to pay for this. >=20 > have you tried this kernel with real hardware? >=20 Hello Julian CARP with this kernel in other environment does work fine. The issue is that carp interface never goes MASTER, it stay in INIT state w= /out going into MASTER state. The error we see on dmesg is: ifa_del_loopback_route : deletion failed: 48. There is no block on MAC-ADDRESSES on Hyper-v, no firewall, nothing. We have tried with de interface (lelacy) and hn interface: same issue. this is the hn interface: hn0: on vmbus0 this is the ifconfig output: hn0: flags=3D8943 metric 0 = mtu 1500 options=3D18 ether 00:15:5d:2c:11:21 inet6 fe80::215:5dff:fe2c:1121%hn0 prefixlen 64 scopeid 0x3=20 inet 192.168.222.201 netmask 0xffffff00 broadcast 192.168.222.255=20 inet 192.168.222.200 netmask 0xffffff00 broadcast 192.168.222.255 v= hid 1=20 nd6 options=3D29 carp: INIT vhid 1 advbase 1 advskew 0 In attach I provide you full dmesg output I can provide you an ssh access to this freebsd virtual machine, no problem= at all. Googling around we find many people facing the same issue (CARP on Hyper-v), none was able to provide a solution in forums or mailing lists. Looking forward to hear from you. Best Regards --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --MP_/0Q1XfTp_K/vlErc8C/IdV4+ Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename=carp-dmesg.txt Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 module vmbus already present! module storvsc already present! module hn already present! module atapci_dis already present! CPU: Intel(R) Xeon(R) CPU 5140 @ 2.33GHz (1262.62-MHz K8-class = CPU) Origin =3D "GenuineIntel" Id =3D 0x6f6 Family =3D 0x6 Model =3D 0xf S= tepping =3D 6 Features=3D0xf83fbff Features2=3D0x80002201 AMD Features=3D0x20100800 AMD Features2=3D0x1 real memory =3D 2147483648 (2048 MB) avail memory =3D 2052059136 (1956 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: ioapic0: Changing APIC ID to 0 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 random: initialized vmbus0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, f7f00000 (3) failed cpu0: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177= ,0x376,0xffa0-0xffaf at device 7.1 on pci0 ata0: at channel 0 on atapci0 ata1: at channel 1 on atapci0 pci0: at device 7.3 (no driver attached) vgapci0: mem 0xf8000000-0xfbffffff irq 11 at devic= e 8.0 on pci0 vgapci0: Boot video device de0: port 0xec00-0xec7f mem 0xfebff000-0xfeb= fffff irq 11 at device 10.0 on pci0 de0: 21140A [10-100Mb/s] pass 2.0 de0: Ethernet address: 00:15:5d:2c:11:1a atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on= acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 orm0: at iomem 0xc0000-0xcbfff,0xcc000-0xcc7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 Timecounters tick every 20.000 msec storvsc0 on vmbus0 hyperv-utils0 on vmbus0 hyperv-utils0: Hyper-V Service attaching: Hyper-V Heartbeat Service hyperv-kvp0 on vmbus0 hyperv-kvp0: Hyper-V Service attaching: Hyper-V KVP Service hyperv-utils1 on vmbus0 hyperv-utils1: Hyper-V Service attaching: Hyper-V Shutdown Service hv_kvp_negotiate_version hyperv-utils2 on vmbus0 hyperv-utils2: Hyper-V Service attaching: Hyper-V Time Synch Service storvsc1 on vmbus0 Netvsc probe... DONE=20 hn0: on vmbus0 Netvsc initializing... hn0: Ethernet address: 00:15:5d:2c:11:21 Netvsc probe... DONE=20 hn1: on vmbus0 Netvsc initializing... Already initialized! hn1: Ethernet address: 00:15:5d:2c:11:22 random: unblocking device. da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: Fixed Direct Access SCSI-4 device=20 da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) cd0 at ata1 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-5 device=20 cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... Already initialized! Timecounter "TSC" frequency 1262623248 Hz quality 800 Trying to mount root from ufs:/dev/da0p2 [rw]... ifa_del_loopback_route: deletion failed: 48 carp: demoted by -240 to 0 (vhid removed) hn0: promiscuous mode disabled Closing Device ... Closing Device ... hn0: promiscuous mode enabled carp: demoted by 240 to 240 (interface down) --MP_/0Q1XfTp_K/vlErc8C/IdV4+-- --Sig_/LfyNV3t=vub5UTNXSfWYW=F Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzAzMTEyODUx WjAjBgkqhkiG9w0BCQQxFgQUpd07idz2Dx6kI4x8HTjHAaFR0PUwKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAFDbrQzT0hoZQrjAUeDqmZGm/2bMwMr7qAVDAK52rfBENrGWRznddP9wutyc0 v52qsjBD4CJW705EZfqQexpyNdMVI/tn+JBOD0xuQvATam7II2vkEkjvtP35TMse X4JpRTx9SMlRMo06Pg8Oj6ixzLsaheQgyq91cDp+3/6zvPqfmcyWllRL7AwidPuA U4sOCZlveuPs53VdIgI+matpJrI8IvuG5HpVFZGbMIhpS4bgSI4EnLBETt7ITksY re6TFfA7Oq+ZVv+LedDWq72wXzKgpjSwQ2AztI7pHaoVb/Wa9+DRRU+FEVlCgB3i Z1oMn/kK3aVJCuY+mTKm2A3QIgAAAAAAAA== --Sig_/LfyNV3t=vub5UTNXSfWYW=F-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 05:39:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01F7DAAB for ; Tue, 4 Mar 2014 05:39:32 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id B87D4174 for ; Tue, 4 Mar 2014 05:39:31 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id C726B12188; Tue, 4 Mar 2014 15:39:30 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSG34448 (AUTH peterg@ptree32.com.au); Tue, 4 Mar 2014 15:39:30 +1000 Message-ID: <53156710.1050503@freebsd.org> Date: Mon, 03 Mar 2014 21:39:28 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Andrea Brancatelli , "freebsd-virtualization@freebsd.org" Subject: Re: bhyve core dump... References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 05:39:32 -0000 Hi Andrea, > I was installing mysql56 in a freebsd10 inside freebsd 10 and while doing > the install the machine bombed out. In the console this is what I got: > > Assertion failed: (aior != NULL), function ahci_handle_dma, file > /usr/src/usr.sbin/bhyve/pci_ahci.c, line 445. > Abort trap (core dumped) > > The VM was calculating the SHA256 of the distfile when it bombed out. > Nothing else was going on inside the VM at that time. > > I even have the .core if it matters. This is the issue that was fixed in HEAD with http://svnweb.freebsd.org/base?view=revision&revision=260999 Not yet in stable: should be there soon. The workaround is to use virtio block as the disk emulation. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 05:42:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25905B15 for ; Tue, 4 Mar 2014 05:42:04 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id DC5DA1F5 for ; Tue, 4 Mar 2014 05:42:03 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 50F6C12189; Tue, 4 Mar 2014 15:42:02 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSG34631 (AUTH peterg@ptree32.com.au); Tue, 4 Mar 2014 15:42:01 +1000 Message-ID: <531567A7.6070000@freebsd.org> Date: Mon, 03 Mar 2014 21:41:59 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Andrea Brancatelli , "freebsd-virtualization@freebsd.org" Subject: Re: bhyvectl References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 05:42:04 -0000 Hi Andrea, > I just throw away at least one hour because I was running bhyvectl destroy > --vm=something instead of bhyvectl --destroy --vm=something. Bhyvectl just > silently ignored the wrong "destroy" instead of "--destroy". > > Maybe it's worth printing out something in such circumstances??? :-) The simple fix, which should show up soon, is to exit in bhyveload if the VM has already been created (i.e. is must be deleted before re-running bhyveload). This will also fix the case of not running the delete. And, bhyvectl should be fixed to flag an error for this case. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 10:37:49 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1A1D3DF; Tue, 4 Mar 2014 10:37:49 +0000 (UTC) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B26ABF32; Tue, 4 Mar 2014 10:37:49 +0000 (UTC) Received: by mail-ig0-f176.google.com with SMTP id uy17so10133509igb.3 for ; Tue, 04 Mar 2014 02:37:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=b8xpwE3B4C2eXMRobQK8TiDUxP2Cg3K2eAdDuq3wHFs=; b=fv0Q6jji9ipOf0NbU3Z0UTTlKVMrSJTVMe4xm7O3LCbSc/fy2AfXL2JjB96fvNW3CI 0bBmLcrzrunhgDsCS770DvMcxuyHxLN1sgUQjuLZJje0J3GFb/yXE2509895W1rD1Az5 VgS2pB7GGaaO48r8mdjsz1O4z87yDXKHehNJAFCMCY2fUr4pbvXqSug0idDtdXK6o53h vv9O3BitF/TYtQMxKMwM/J8MHwooqNvlVx9J48mBq+pHpqv/PNA7jMhvLjA6Lekjq4va SCGqCprO8oov3hdIQQcLO5PEOQT+lqz3WDfrvEhxAnOlJltMuFH7O9fz8XUPnYgB0PHd /Rfw== X-Received: by 10.43.181.69 with SMTP id ph5mr389862icc.78.1393929469159; Tue, 04 Mar 2014 02:37:49 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.192.67 with HTTP; Tue, 4 Mar 2014 02:37:28 -0800 (PST) In-Reply-To: <5268B533.80408@FreeBSD.org> References: <794fb75db92a4df0991a147919727277@BL2PR03MB210.namprd03.prod.outlook.com> <5268B533.80408@FreeBSD.org> From: Oleg Sidorkin Date: Tue, 4 Mar 2014 14:37:28 +0400 Message-ID: Subject: Re: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project To: Alexander Motin Content-Type: multipart/mixed; boundary=001a11c3b40cda057204f3c57e9b Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 10:37:50 -0000 --001a11c3b40cda057204f3c57e9b Content-Type: text/plain; charset=ISO-8859-1 Hi again. Disabling scan_for_luns leaves drives undetected. Calling xpt_rescan for each lun works for me. With the attached patch system boots and detects all configured drives. But also this patch introduces a race between drives detection and boot process, so sometimes system tries to mount undetected drive. I'm going to fix this by calling xpt_hold_boot() before xpt_rescan() and calling xpt_release_boot() in callback. Thanks On Thu, Oct 24, 2013 at 9:50 AM, Alexander Motin wrote: > Hi. > > I took some look and think problems are in scan_for_luns() routine: > - After the locking changes scanning normally uses different locks, not the > SIM one. That probably caused panic. > - But I think that scanning is simply not needed there -- FreeBSD CAM scans > every new bus automatically on registration (Even for late registered buses > it is done I think at least since FreeBSD 8). I think everything should just > work if you remove scan_for_luns() at all. > - If you still wish to force scan (due to having information about changed > list of devices, etc), then you can make CAM do all the magic for you by > calling xpt_rescan(). > > > On 24.10.2013 08:34, Oleg Sidorkin wrote: >> >> Hello again. >> >> Camlock patches are now committed and -CURRENT on Hyper-V now panics >> with almost the same stacktrace: >> >> FreeBSD 11.0-CURRENT #16 r257016: Wed Oct 23 21:08:44 UTC 2013 >> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 >> FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 >> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.57-MHz K8-class CPU) >> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >> Stepping = 7 >> >> ...... >> >> ZFS filesystem version: 5 >> ZFS storage pool version: features support (5000) >> Timecounters tick every 10.000 msec >> storvsc0 on vmbus0 >> kernel trap 12 with interrupts disabled >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 0; apic id = 00 >> fault virtual address = 0x20 >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff804f58cc >> stack pointer = 0x28:0xfffffe011dd5f5d0 >> frame pointer = 0x28:0xfffffe011dd5f600 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = resume, IOPL = 0 >> current process = 0 (hv_control_1 taskq) >> [ thread pid 0 tid 100047 ] >> Stopped at turnstile_broadcast+0x8c: movq >> 0x20(%rbx,%rax,1),%rdx >> db> bt >> Tracing pid 0 tid 100047 td 0xfffff8000331e000 >> turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfffffe011dd5f600 >> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011dd5f630 >> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011dd5f640 >> _sleep() at _sleep+0x18e/frame 0xfffffe011dd5f6c0 >> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011dd5f7f0 >> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011dd5f890 >> device_attach() at device_attach+0x3a2/frame 0xfffffe011dd5f8f0 >> hv_vmbus_child_device_register() at >> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011dd5f990 >> vmbus_channel_process_offer() at >> vmbus_channel_process_offer+0x133/frame 0xfffffe011dd5f9d0 >> work_item_callback() at work_item_callback+0x26/frame 0xfffffe011dd5f9f0 >> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >> 0xfffffe011dd5fa40 >> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >> 0xfffffe011dd5fa70 >> fork_exit() at fork_exit+0x9a/frame 0xfffffe011dd5fab0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011dd5fab0 >> --- trap 0, rip = 0, rsp = 0xfffffe011dd5fb70, rbp = 0 --- >> >> >> Thanks >> >> On Tue, Sep 24, 2013 at 3:04 AM, Abhishek Gupta (LIS) >> wrote: >>> >>> Hi Oleg, >>> >>> Please give us some time. I shall look at it. Thanks for reporting. >>> >>> Regards, >>> Abhishek >>> >>> -----Original Message----- >>> From: owner-freebsd-virtualization@freebsd.org >>> [mailto:owner-freebsd-virtualization@freebsd.org] On Behalf Of Oleg Sidorkin >>> Sent: Monday, September 23, 2013 7:21 AM >>> To: freebsd-virtualization@freebsd.org >>> Cc: Alexander Motin >>> Subject: [Hyper-V][camlock] storvsc driver panics during boot with >>> patches from camlock project >>> >>> Hello. >>> >>> I'm running the latest current (amd64) under Hyper-V with hyper-v >>> services enabled. >>> If camlock patches are applied >>> (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), >>> I'm hitting the following kernel panic during boot: >>> >>> FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 >>> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang >>> version 3.3 (tags/RELEASE_33/final 183502) 20130610 >>> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class CPU) >>> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >>> Stepping = >>> 7 >>> .... >>> Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 ZFS NOTICE: >>> Prefetch is disabled by default if less than 4GB of RAM is present; >>> to enable, add "vfs.zfs.prefetch_disable=0" to >>> /boot/loader.conf. >>> ZFS filesystem version: 5 >>> ZFS storage pool version: features support (5000) Timecounters tick every >>> 10.000 msec >>> storvsc0 on vmbus0 >>> Netvsc initializing... SMP: AP CPU #3 Launched! >>> SMP: AP CPU #2 Launched! >>> SMP: AP CPU #1 Launched! >>> kernel trap 12 with interrupts disabled >>> >>> >>> Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 >>> fault virtual address = 0x20 >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff804f444c >>> stack pointer = 0x28:0xfffffe011df38610 >>> frame pointer = 0x28:0xfffffe011df38640 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = resume, IOPL = 0 >>> current process = 0 (hv_control_1 taskq) >>> [ thread pid 0 tid 100046 ] >>> Stopped at turnstile_broadcast+0x8c: movq >>> 0x20(%rbx,%rax,1),%rdx >>> db> bt >>> Tracing pid 0 tid 100046 td 0xfffff80001f20490 >>> turnstile_broadcast() at turnstile_broadcast+0x8c/frame >>> 0xfffffe011df38640 >>> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011df38670 >>> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011df38680 >>> _sleep() at _sleep+0x18e/frame 0xfffffe011df38700 >>> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011df38800 >>> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011df388a0 >>> device_attach() at device_attach+0x396/frame 0xfffffe011df388f0 >>> hv_vmbus_child_device_register() at >>> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011df38990 >>> vmbus_channel_process_offer() at >>> vmbus_channel_process_offer+0x133/frame 0xfffffe011df389d0 >>> work_item_callback() at work_item_callback+0x26/frame 0xfffffe011df389f0 >>> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >>> 0xfffffe011df38a40 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >>> 0xfffffe011df38a70 >>> fork_exit() at fork_exit+0x9a/frame 0xfffffe011df38ab0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011df38ab0 >>> --- trap 0, rip = 0, rsp = 0xfffffe011df38b70, rbp = 0 --- >>> db> >>> >>> >>> This patch is not commited yet (CFT thread with changes description is >>> here: >>> http://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043333.html), >>> but it is going to be commited till the end of the year. >>> >>> As far as I understand, the invocation chain is >>> storvsc_attach->scan_for_luns->cam_periph_runccb >>> >>> Thanks >>> -- >>> Oleg Sidorkin >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to >>> "freebsd-virtualization-unsubscribe@freebsd.org" >> >> >> >> > > > -- > Alexander Motin -- Oleg Sidorkin --001a11c3b40cda057204f3c57e9b Content-Type: application/octet-stream; name="storvsc.patch" Content-Disposition: attachment; filename="storvsc.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hsd1ebsq0 SW5kZXg6IHN5cy9kZXYvaHlwZXJ2L3N0b3J2c2MvaHZfc3RvcnZzY19kcnZfZnJlZWJzZC5jCj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0KLS0tIHN5cy9kZXYvaHlwZXJ2L3N0b3J2c2MvaHZfc3RvcnZzY19kcnZfZnJlZWJz ZC5jCShyZXZpc2lvbiAyNjE5NjIpCisrKyBzeXMvZGV2L2h5cGVydi9zdG9ydnNjL2h2X3N0b3J2 c2NfZHJ2X2ZyZWVic2QuYwkod29ya2luZyBjb3B5KQpAQCAtNjY5LDIyICs2NjksNiBAQAogfQog CiAvKioKLSAqIEBicmllZiBjYWxsYmFjayBmdW5jdGlvbiBmb3IgY29tcGxldGluZyBhIHNpbmds ZSBMVU4gc2NhbgotICoKLSAqIFRoaXMgZnVuY3Rpb24gaXMgcmVzcG9uc2libGUgZm9yIHdha2lu ZyB1cCB0aGUgZXhlY3V0ZXIgb2YKLSAqIHRoZSBzY2FuIExVTiBDQ0IgYWN0aW9uIChjYW1fcGVy aXBoX3J1bmNjYi4pICBjYW1fcGVyaXBoX2NjYndhaXQKLSAqIHNsZWVwcyBvbiB0aGUgbXV0ZXgg YmVpbmcgc2lnbmFsZWQuCi0gKgotICogQHBhcmFtIHBlcmlwaCBhIHBvaW50ZXIgdG8gYSBDQU0g cGVyaXBoZXJhbAotICogQHBhcmFtIGRvbmVfY2NiIHBvaW50ZXIgdG8gQ0FNIGNvbnRyb2wgYmxv Y2sKLSAqLwotc3RhdGljIHZvaWQKLXN0b3J2c2NfeHB0ZG9uZShzdHJ1Y3QgY2FtX3BlcmlwaCAq cGVyaXBoLCB1bmlvbiBjY2IgKmRvbmVfY2NiKQotewotCXdha2V1cCgmZG9uZV9jY2ItPmNjYl9o LmNiZmNucCk7Ci19Ci0KLS8qKgogICogQGJyaWVmIHNjYW4gZm9yIGF0dGFjaGVkIGxvZ2ljYWwg dW5pdCBudW1iZXJzIChMVU5zKQogICoKICAqIEluIEh5cGVyLVYgdGhlcmUgaXMgbm8gYmFja2Vu ZCBjaGFuZ2VkIGRldmljZSBvcGVyYXRpb24gd2hpY2gKQEAgLTY5OCw1MiArNjgyLDI1IEBACiBz dGF0aWMgdm9pZAogc2Nhbl9mb3JfbHVucyhzdHJ1Y3Qgc3RvcnZzY19zb2Z0YyAqc2MpCiB7Ci0J dW5pb24gY2NiICpyZXF1ZXN0X2NjYjsKLQlzdHJ1Y3QgY2FtX3BhdGggKnBhdGggPSBzYy0+aHNf cGF0aDsKLQlzdHJ1Y3QgY2FtX3BhdGggKm15X3BhdGggPSBOVUxMOwotCWNhbV9zdGF0dXMgc3Rh dHVzOworCXVuaW9uIGNjYiAqY2NiOwogCWludCBsdW5fbmIgPSAwOwotCWludCBlcnJvcjsKIAot CXJlcXVlc3RfY2NiID0gbWFsbG9jKHNpemVvZih1bmlvbiBjY2IpLCBNX0NBTVhQVCwgTV9XQUlU T0spOwotCW15X3BhdGggPSBtYWxsb2Moc2l6ZW9mKCpteV9wYXRoKSwgTV9DQU1YUFQsIE1fV0FJ VE9LKTsKLQogCW10eF9sb2NrKCZzYy0+aHNfbG9jayk7CiAJZG8gewogCQkvKgogCQkgKiBTY2Fu IHRoZSBuZXh0IExVTi4gUmV1c2UgcGF0aCBhbmQgY2NiIHN0cnVjdHMuCiAJCSAqLwotCQliemVy byhteV9wYXRoLCBzaXplb2YoKm15X3BhdGgpKTsKLQkJYnplcm8ocmVxdWVzdF9jY2IsIHNpemVv ZigqcmVxdWVzdF9jY2IpKTsKLQkJc3RhdHVzID0geHB0X2NvbXBpbGVfcGF0aChteV9wYXRoLAot CQkJCSAgeHB0X3BlcmlwaCwKLQkJCQkgIHBhdGgtPmJ1cy0+cGF0aF9pZCwKLQkJCQkgIDAsCi0J CQkJICBsdW5fbmIpOwotCi0JCWlmIChzdGF0dXMgIT0gQ0FNX1JFUV9DTVApIHsKLQkJCW10eF91 bmxvY2soJnNjLT5oc19sb2NrKTsKLQkgICAgICAgCQl4cHRfcHJpbnQocGF0aCwgInNjYW5fZm9y X2x1bllZWTogY2FuJ3QgY29tcGlsZSIKLQkJCQkJICIgcGF0aCwgMHglcCBjYW4ndCBjb250aW51 ZVxuIiwKLQkJCQkJIHNjLT5oc19wYXRoKTsKLQkJCWZyZWUocmVxdWVzdF9jY2IsIE1fQ0FNWFBU KTsKLQkJCWZyZWUobXlfcGF0aCwgTV9DQU1YUFQpOwotCQkJcmV0dXJuOwotCQl9Ci0KLQkJeHB0 X3NldHVwX2NjYigmcmVxdWVzdF9jY2ItPmNjYl9oLCBteV9wYXRoLCA1KTsKLQkJcmVxdWVzdF9j Y2ItPmNjYl9oLmZ1bmNfY29kZSA9IFhQVF9TQ0FOX0xVTjsKLQkJcmVxdWVzdF9jY2ItPmNjYl9o LmNiZmNucCAgICA9IHN0b3J2c2NfeHB0ZG9uZTsKLQkJcmVxdWVzdF9jY2ItPmNyY24uZmxhZ3MJ ICAgICA9IENBTV9GTEFHX05PTkU7Ci0KLQkJZXJyb3IgPSBjYW1fcGVyaXBoX3J1bmNjYihyZXF1 ZXN0X2NjYiwgTlVMTCwgCi0JCQkJCQlDQU1fRkxBR19OT05FLCAwLCBOVUxMKTsKLQkJS0FTU0VS VChlcnJvciA9PSAwLCAoImNhbV9wZXJpcGhfcnVuY2NiIGZhaWxlZCAlZFxuIiwgZXJyb3IpKTsK LQkJeHB0X3JlbGVhc2VfcGF0aChteV9wYXRoKTsKKyAgICAgICAgICAgICAgICBpZiAoKGNjYiA9 IHhwdF9hbGxvY19jY2IoKSkgPT0gTlVMTCkKKyAgICAgICAgICAgICAgICAgICAgICAgIHJldHVy bjsKKyAgICAgICAgICAgICAgICBpZiAoeHB0X2NyZWF0ZV9wYXRoKCZjY2ItPmNjYl9oLnBhdGgs IHhwdF9wZXJpcGgsCisgICAgICAgICAgICAgICAgICAgIHNjLT5oc19wYXRoLT5idXMtPnBhdGhf aWQsCisgICAgICAgICAgICAgICAgICAgIDAsIGx1bl9uYikgIT0gQ0FNX1JFUV9DTVApIHsKKyAg ICAgICAgICAgICAgICAgICAgICAgIHhwdF9mcmVlX2NjYihjY2IpOworICAgICAgICAgICAgICAg ICAgICAgICAgcmV0dXJuOworICAgICAgICAgICAgICAgIH0KKyAgICAgICAgICAgICAgICB4cHRf cmVzY2FuKGNjYik7CiAJfSB3aGlsZSAoICsrbHVuX25iIDwgc2MtPmhzX2Rydl9wcm9wcy0+ZHJ2 X21heF9sdW5zX3Blcl90YXJnZXQpOwogCW10eF91bmxvY2soJnNjLT5oc19sb2NrKTsKLQlmcmVl KHJlcXVlc3RfY2NiLCBNX0NBTVhQVCk7Ci0JZnJlZShteV9wYXRoLCBNX0NBTVhQVCk7CiB9CiAK IC8qKgo= --001a11c3b40cda057204f3c57e9b-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 11:06:31 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CFDBA45 for ; Tue, 4 Mar 2014 11:06:31 +0000 (UTC) Received: from mail-ee0-x22d.google.com (mail-ee0-x22d.google.com [IPv6:2a00:1450:4013:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CCDB0229 for ; Tue, 4 Mar 2014 11:06:30 +0000 (UTC) Received: by mail-ee0-f45.google.com with SMTP id d17so4621211eek.18 for ; Tue, 04 Mar 2014 03:06:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=0YAoDnhCERd05fOpEFcctcaiHGKNSyAjPOELIYx5f7A=; b=Z1CRWnvmJDN4g0ZWGUIrNXZjy34epiHE33EhlbU/F5JdcHG2rMI4xMdlhRSZv9pdwh Cvc9ZqdUrbsh4PiXD4fnms8zRo3UIqeHjp3FeFjA6VcbS0234x/LqkifJusR6RszdeLQ uI2DTBB+ymsx4/+NW0u8ob0Sy3sZQS0bDPqsaonMLuRb3UrrDr5EjkGH3H8Y5DN2VgfI G9KJI+zEuHZUwvK5pc0TIIAFUtK3RqDODdBN1YZpj5dT9sCT1yvxSK5hGh3r5FLRBKRH kFt7Nk/P6N/Ju+qFY5bBLrRQekv/Tv+8+ezLx2xO36xM1IQi9pcNr9w5rt75631mjFzV oEHg== X-Received: by 10.15.21.2 with SMTP id c2mr45645936eeu.77.1393931189181; Tue, 04 Mar 2014 03:06:29 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id q44sm61564656eez.1.2014.03.04.03.06.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 04 Mar 2014 03:06:28 -0800 (PST) Sender: Alexander Motin Message-ID: <5315B3B1.5080909@FreeBSD.org> Date: Tue, 04 Mar 2014 13:06:25 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Oleg Sidorkin Subject: Re: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project References: <794fb75db92a4df0991a147919727277@BL2PR03MB210.namprd03.prod.outlook.com> <5268B533.80408@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:06:31 -0000 Oleg, It was found that real problem is not there. You do not need manual scanning. You should remove it. All you really need is to properly report bus to CAM: @@ -1147,7 +1062,7 @@ cpi->hba_eng_cnt = 0; cpi->max_target = STORVSC_MAX_TARGETS; cpi->max_lun = sc->hs_drv_props->drv_max_luns_per_target; - cpi->initiator_id = 0; + cpi->initiator_id = cpi->max_lun + 1; cpi->bus_id = cam_sim_bus(sim); cpi->base_transfer_speed = 300000; cpi->transport = XPORT_SAS; Default scanner skips initiator from the process, so initiator_id = 0 excluded from scan the only really used target. On 04.03.2014 12:37, Oleg Sidorkin wrote: > Hi again. > > Disabling scan_for_luns leaves drives undetected. > > Calling xpt_rescan for each lun works for me. With the attached patch > system boots and detects all configured drives. > But also this patch introduces a race between drives detection and > boot process, so sometimes system tries to mount undetected drive. > I'm going to fix this by calling xpt_hold_boot() before xpt_rescan() > and calling xpt_release_boot() in callback. > > Thanks > > On Thu, Oct 24, 2013 at 9:50 AM, Alexander Motin wrote: >> Hi. >> >> I took some look and think problems are in scan_for_luns() routine: >> - After the locking changes scanning normally uses different locks, not the >> SIM one. That probably caused panic. >> - But I think that scanning is simply not needed there -- FreeBSD CAM scans >> every new bus automatically on registration (Even for late registered buses >> it is done I think at least since FreeBSD 8). I think everything should just >> work if you remove scan_for_luns() at all. >> - If you still wish to force scan (due to having information about changed >> list of devices, etc), then you can make CAM do all the magic for you by >> calling xpt_rescan(). >> >> >> On 24.10.2013 08:34, Oleg Sidorkin wrote: >>> >>> Hello again. >>> >>> Camlock patches are now committed and -CURRENT on Hyper-V now panics >>> with almost the same stacktrace: >>> >>> FreeBSD 11.0-CURRENT #16 r257016: Wed Oct 23 21:08:44 UTC 2013 >>> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 >>> FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 >>> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.57-MHz K8-class CPU) >>> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >>> Stepping = 7 >>> >>> ...... >>> >>> ZFS filesystem version: 5 >>> ZFS storage pool version: features support (5000) >>> Timecounters tick every 10.000 msec >>> storvsc0 on vmbus0 >>> kernel trap 12 with interrupts disabled >>> >>> >>> Fatal trap 12: page fault while in kernel mode >>> cpuid = 0; apic id = 00 >>> fault virtual address = 0x20 >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff804f58cc >>> stack pointer = 0x28:0xfffffe011dd5f5d0 >>> frame pointer = 0x28:0xfffffe011dd5f600 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = resume, IOPL = 0 >>> current process = 0 (hv_control_1 taskq) >>> [ thread pid 0 tid 100047 ] >>> Stopped at turnstile_broadcast+0x8c: movq >>> 0x20(%rbx,%rax,1),%rdx >>> db> bt >>> Tracing pid 0 tid 100047 td 0xfffff8000331e000 >>> turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfffffe011dd5f600 >>> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011dd5f630 >>> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011dd5f640 >>> _sleep() at _sleep+0x18e/frame 0xfffffe011dd5f6c0 >>> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011dd5f7f0 >>> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011dd5f890 >>> device_attach() at device_attach+0x3a2/frame 0xfffffe011dd5f8f0 >>> hv_vmbus_child_device_register() at >>> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011dd5f990 >>> vmbus_channel_process_offer() at >>> vmbus_channel_process_offer+0x133/frame 0xfffffe011dd5f9d0 >>> work_item_callback() at work_item_callback+0x26/frame 0xfffffe011dd5f9f0 >>> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >>> 0xfffffe011dd5fa40 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >>> 0xfffffe011dd5fa70 >>> fork_exit() at fork_exit+0x9a/frame 0xfffffe011dd5fab0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011dd5fab0 >>> --- trap 0, rip = 0, rsp = 0xfffffe011dd5fb70, rbp = 0 --- >>> >>> >>> Thanks >>> >>> On Tue, Sep 24, 2013 at 3:04 AM, Abhishek Gupta (LIS) >>> wrote: >>>> >>>> Hi Oleg, >>>> >>>> Please give us some time. I shall look at it. Thanks for reporting. >>>> >>>> Regards, >>>> Abhishek >>>> >>>> -----Original Message----- >>>> From: owner-freebsd-virtualization@freebsd.org >>>> [mailto:owner-freebsd-virtualization@freebsd.org] On Behalf Of Oleg Sidorkin >>>> Sent: Monday, September 23, 2013 7:21 AM >>>> To: freebsd-virtualization@freebsd.org >>>> Cc: Alexander Motin >>>> Subject: [Hyper-V][camlock] storvsc driver panics during boot with >>>> patches from camlock project >>>> >>>> Hello. >>>> >>>> I'm running the latest current (amd64) under Hyper-V with hyper-v >>>> services enabled. >>>> If camlock patches are applied >>>> (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), >>>> I'm hitting the following kernel panic during boot: >>>> >>>> FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 >>>> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang >>>> version 3.3 (tags/RELEASE_33/final 183502) 20130610 >>>> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class CPU) >>>> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >>>> Stepping = >>>> 7 >>>> .... >>>> Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 ZFS NOTICE: >>>> Prefetch is disabled by default if less than 4GB of RAM is present; >>>> to enable, add "vfs.zfs.prefetch_disable=0" to >>>> /boot/loader.conf. >>>> ZFS filesystem version: 5 >>>> ZFS storage pool version: features support (5000) Timecounters tick every >>>> 10.000 msec >>>> storvsc0 on vmbus0 >>>> Netvsc initializing... SMP: AP CPU #3 Launched! >>>> SMP: AP CPU #2 Launched! >>>> SMP: AP CPU #1 Launched! >>>> kernel trap 12 with interrupts disabled >>>> >>>> >>>> Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 >>>> fault virtual address = 0x20 >>>> fault code = supervisor read data, page not present >>>> instruction pointer = 0x20:0xffffffff804f444c >>>> stack pointer = 0x28:0xfffffe011df38610 >>>> frame pointer = 0x28:0xfffffe011df38640 >>>> code segment = base 0x0, limit 0xfffff, type 0x1b >>>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>>> processor eflags = resume, IOPL = 0 >>>> current process = 0 (hv_control_1 taskq) >>>> [ thread pid 0 tid 100046 ] >>>> Stopped at turnstile_broadcast+0x8c: movq >>>> 0x20(%rbx,%rax,1),%rdx >>>> db> bt >>>> Tracing pid 0 tid 100046 td 0xfffff80001f20490 >>>> turnstile_broadcast() at turnstile_broadcast+0x8c/frame >>>> 0xfffffe011df38640 >>>> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011df38670 >>>> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011df38680 >>>> _sleep() at _sleep+0x18e/frame 0xfffffe011df38700 >>>> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011df38800 >>>> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011df388a0 >>>> device_attach() at device_attach+0x396/frame 0xfffffe011df388f0 >>>> hv_vmbus_child_device_register() at >>>> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011df38990 >>>> vmbus_channel_process_offer() at >>>> vmbus_channel_process_offer+0x133/frame 0xfffffe011df389d0 >>>> work_item_callback() at work_item_callback+0x26/frame 0xfffffe011df389f0 >>>> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >>>> 0xfffffe011df38a40 >>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >>>> 0xfffffe011df38a70 >>>> fork_exit() at fork_exit+0x9a/frame 0xfffffe011df38ab0 >>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011df38ab0 >>>> --- trap 0, rip = 0, rsp = 0xfffffe011df38b70, rbp = 0 --- >>>> db> >>>> >>>> >>>> This patch is not commited yet (CFT thread with changes description is >>>> here: >>>> http://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043333.html), >>>> but it is going to be commited till the end of the year. >>>> >>>> As far as I understand, the invocation chain is >>>> storvsc_attach->scan_for_luns->cam_periph_runccb >>>> >>>> Thanks >>>> -- >>>> Oleg Sidorkin >>>> _______________________________________________ >>>> freebsd-virtualization@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>>> To unsubscribe, send any mail to >>>> "freebsd-virtualization-unsubscribe@freebsd.org" >>> >>> >>> >>> >> >> >> -- >> Alexander Motin > > > -- Alexander Motin From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 15:09:17 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D13FFC9; Tue, 4 Mar 2014 15:09:17 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9F8A1FF6; Tue, 4 Mar 2014 15:09:17 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 29F1E21695; Tue, 4 Mar 2014 10:09:00 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute4.internal (MEProxy); Tue, 04 Mar 2014 10:09:01 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=8v/T5C0R0+ZtcwwGdCDWP6Jiuu0=; b=rpm MRBS/OfijibPVGLGWL5x6KvWYr6WeowwVCN224lG7yQvH5fekkSNOEnlDeHMoCZV X14FbzyjQG+OfXcbQ/t8Lg65x4Vubk74swZr8wgar0zY5zvibH1Sn5fsPvGxjK43 yuHF8cGriTXNPJL6/bWwVcZDRbbhGluCQM9sqYuM= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 124E4108B51; Tue, 4 Mar 2014 10:09:00 -0500 (EST) Message-Id: <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> X-Sasl-Enc: 54ObPpjiRiWTfPe+TVb7NF+qvRZF/OajidQ4sd4tl5tc 1393945740 From: Mark Felder To: Maurizio Marini , Julian Elischer MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-4527a23f In-Reply-To: <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Date: Tue, 04 Mar 2014 09:09:00 -0600 Cc: Giovanni Mattera , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 15:09:17 -0000 Did you try to enable spoofing of MAC Addresses in HyperV? http://vbry21.wordpress.com/2012/07/03/fixed-in-a-tick-solving-mulicast-issues-in-hyper-v-vms/ From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 15:35:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7480B969 for ; Tue, 4 Mar 2014 15:35:28 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4432FB for ; Tue, 4 Mar 2014 15:35:28 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so5283147pde.38 for ; Tue, 04 Mar 2014 07:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=lMkCyjSsaUvx9moAMB70cyubbUmO5hT1+OhBGq0rssw=; b=BH+x/ryPoEoc3TppCxRLoHfkmUYAfDkBKA4aj+1hYyoAzBE6+QCR8CUZlXyahQObid nht4CdDtYDfz5uq9jUq9c4tFBReXkCdQczspPZEkRP3d7h8wCVA4ASNDTcZbX/Vc2bPZ ZZ4FcoFQC+QTLCzDhsAxZEIX6F0zNsB/KC/CkmAac8ZVz3CUNIXtick3uf1avJUrPXxT F1FpDHgnlxDM5dICAlgG9+ZOJwtNkfHjTbGUibUTBZwrjUHSoZbLI5TEC83ecey/PxnQ /LnP8NnOvURb7yAGpn1+hcMekebgeyurJMXDKjn6vcxIWlIvNyVtlWhc0FT5rbS4ay+U k6Cw== MIME-Version: 1.0 X-Received: by 10.68.102.34 with SMTP id fl2mr352598pbb.2.1393947326947; Tue, 04 Mar 2014 07:35:26 -0800 (PST) Received: by 10.68.155.38 with HTTP; Tue, 4 Mar 2014 07:35:26 -0800 (PST) Date: Tue, 4 Mar 2014 10:35:26 -0500 Message-ID: Subject: Call for Submissions: 1st Creative use of Cloud Computing Contest From: Aryeh Friedman To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 15:35:28 -0000 http://www.petitecloud.org/contest.jsp -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 16:13:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22430D8C; Tue, 4 Mar 2014 16:13:05 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8A52488F; Tue, 4 Mar 2014 16:13:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 5AD74989E6C; Tue, 4 Mar 2014 17:11:57 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Kr5vdaqTAoye; Tue, 4 Mar 2014 17:11:55 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id DFFCB989E68; Tue, 4 Mar 2014 17:11:54 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it DFFCB989E68 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393949514; bh=zfEXUJhfA5gx8XDB87VaRig6MHnUdRHF48vxuQvckdM=; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type; b=GnnyPG37sGZukHPxa75Fvrv7eLDOGOz81Kesgt6Ru3bL/KZzOaGp8J589Ap2/gWoe fosTO/pJdHvrZTpd0RU6gPlP+dCmSMKZbshRU0qFE20+T3lmhrBa1ynYBFicloi65J d5PhGokUI9Y1WXldfrCwiPdalOGQ+gkhzkykIlgI= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id x37uxMDDlGGL; Tue, 4 Mar 2014 17:11:54 +0100 (CET) Received: from webmail.cost.it (webmail.cost.it [192.168.222.20]) by webmail.cost.it (Postfix) with ESMTP id 028B8989E58; Tue, 4 Mar 2014 17:11:53 +0100 (CET) Date: Tue, 4 Mar 2014 17:11:53 +0100 (CET) From: Giovanni Mattera To: Mark Felder Message-ID: <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> In-Reply-To: <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v MIME-Version: 1.0 X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF27 (Win)/8.0.6_GA_5922) Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: F9bKfPljswh1IpqjV+LhMNOi3TaKig== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Maurizio Marini , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:13:05 -0000 Yes i test it but not work=20 The CARP protocolol remain in state INIT.=20 ----- Messaggio originale ----- Da: "Mark Felder" =20 A: "Maurizio Marini" , "Julian Elischer" =20 Cc: "FreeBSD Net" , "Manuel Martini" , freebsd-virtualization@freebsd.org, "Giovanni Mattera" =20 Inviato: Marted=C3=AC, 4 marzo 2014 16:09:00=20 Oggetto: Re: freebsd 10.0 not work carp protocol on Hyper-v=20 Did you try to enable spoofing of MAC Addresses in HyperV?=20 http://vbry21.wordpress.com/2012/07/03/fixed-in-a-tick-solving-mulicast-iss= ues-in-hyper-v-vms/=20 From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 16:26:03 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60EA2507; Tue, 4 Mar 2014 16:26:03 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B3559AB; Tue, 4 Mar 2014 16:26:02 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6532C21597; Tue, 4 Mar 2014 11:25:44 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 04 Mar 2014 11:25:45 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id :references:to; s=smtpout; bh=5Oh9nrsiHKOpDdnZ/1EFZtDoikM=; b=Yc b5F2I/u0uYSv9EIGLbfMkjTz2qjwMpk2RR/F2Y4V/VmeDmRJVLaGVBONNbDxd0Mr mHOvN6oX64HFPya8qDDs6B/DTRl8eu5kfTDjIDsKw/QcQ5lino96MybeMfxFOYMk 0bnBTbO157FwYGLClxPfhklcjRCReMsNnpprvKIjE= X-Sasl-enc: chlEcHGw2AvwNNOsQqWdMoc7GbZUiKN52cWEkPlmrbxF 1393950344 Received: from [172.16.1.145] (unknown [68.117.126.78]) by mail.messagingengine.com (Postfix) with ESMTPA id 8B5FD68037C; Tue, 4 Mar 2014 11:25:43 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v From: Mark Felder In-Reply-To: <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> Date: Tue, 4 Mar 2014 10:25:42 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> To: Giovanni Mattera X-Mailer: Apple Mail (2.1874) Cc: Maurizio Marini , freebsd-virtualization@freebsd.org, Manuel Martini , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:26:03 -0000 On Mar 4, 2014, at 10:11, Giovanni Mattera = wrote: > Yes i test it but not work > The CARP protocolol remain in state INIT. >=20 That's unfortunate. If you do a packet dump do you even see the other = node's advertisements? CARP is really at the mercy of these half-baked virtual switches; if = they refuse to pass the traffic there's not much that can be done from = the virtualized guest's perspective. I don't know the procedure for = opening a support case with Microsoft but if that's an option it would = be nice to know what their response is. I did run into this a while back on XenServer and wondered if it would = be possible to work around it by doing a GRE tunnel between the two = nodes and doing CARP over that, tying it all in with scripts that are = called by devd events. I have not had time to experiment and see if you = can do multicast over GRE tunnels though and it's probably a very bad = idea.= From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 16:42:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7620B58 for ; Tue, 4 Mar 2014 16:42:32 +0000 (UTC) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 098EFB44 for ; Tue, 4 Mar 2014 16:42:30 +0000 (UTC) Received: (qmail 22477 invoked by uid 89); 4 Mar 2014 16:42:29 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@88.217.180.167) by mail.grem.de with ESMTPA; 4 Mar 2014 16:42:29 -0000 Date: Tue, 4 Mar 2014 17:42:29 +0100 From: Michael Gmelin To: Aryeh Friedman Subject: Re: Call for Submissions: 1st Creative use of Cloud Computing Contest Message-ID: <20140304174229.6fa46508@bsd64.grem.de> In-Reply-To: References: X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.18; amd64-portbld-freebsd9.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 16:42:32 -0000 On Tue, 4 Mar 2014 10:35:26 -0500 Aryeh Friedman wrote: > http://www.petitecloud.org/contest.jsp > How did this end up in my inbox? -- Michael Gmelin From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 17:43:11 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5DE499A; Tue, 4 Mar 2014 17:43:11 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id 36EAD15B; Tue, 4 Mar 2014 17:43:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id A6509A03FDF; Tue, 4 Mar 2014 18:42:11 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 5tGbe4tKGbVX; Tue, 4 Mar 2014 18:42:10 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 7C133A03FD9; Tue, 4 Mar 2014 18:42:10 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 7C133A03FD9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393954930; bh=OiWj5LVWFH3Dq4ti55oio3OziMGH9Pjrk0VSJKK6SKo=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=Wu3+v+ErUuJENSWGF/Cdhp2hfEkevsiIU4qcBmDZOeVgdFqPbFZHh4+llqf88EVKB fsfhCepE4mZZp2Uc82sY8+ll6Aogn4WSU6tuvtGS59ZlivPPIZvpAEA9PM5eJ6xWiO n1x8Zl9UU0APJJaw3lZUpCdys817oPGObQuLoBkU= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id CD9_EYqv7O1X; Tue, 4 Mar 2014 18:42:10 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id 19345A03FCD; Tue, 4 Mar 2014 18:42:08 +0100 (CET) Date: Tue, 4 Mar 2014 18:43:03 +0100 From: Maurizio Marini To: Mark Felder Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> In-Reply-To: <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/mE=qoOl=qgn_g2GbQPFHogw"; protocol="application/pkcs7-signature" Cc: FreeBSD Net , freebsd-virtualization@freebsd.org, Manuel Martini , Giovanni Mattera X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 17:43:11 -0000 --Sig_/mE=qoOl=qgn_g2GbQPFHogw Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 10:25:42 -0600 Mark Felder wrote: =20 > That's unfortunate. If you do a packet dump do you even see the other nod= e's > advertisements? >=20 > CARP is really at the mercy of these half-baked virtual switc Dear sir: https://forum.pfsense.org/index.php?topic=3D44529.0 "But shouldn't the CARP state go to "master" anyway, even if it couldn't fi= nd a live partner due to network/Hyper-V problems?" this is the question, forget network/Hyper-V problems, CARP should go MASTER anyway --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --Sig_/mE=qoOl=qgn_g2GbQPFHogw Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTc0MzAz WjAjBgkqhkiG9w0BCQQxFgQUDAYo/UhToOPNzhTw4TAnmm1lIU0wKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAMaz3jZxd2VOAVqN8YVkJJsZFUQCBtJENANQP1DZYisgIvlJkF02MWgy9DJ3a 14Vf8hTABGRxrZqPA9pRGNp2q2JK9GWTuVRjIiSW3yo7aR/RfZuNfUi9C2bLxJMd K8iNtoZQ9RgIXWLqFWMUnoIr44Ga58aLjHwAB0ueZNoD33QIGS9YslyWlZEhqTwK +Y9b5l3csPk1nTJLz83yRNAW2yYPAWCZiS03I0Zn+URm5QLg30/jcdVZgCdUEAIp wMvrFvM91fjDvSphV7RGwLanCG3n6JEqPrP5/04ajsqrXfR+8UAdebwwk5lyUjP3 Jeq84lpUnVjUaz13Pu83iFlXCAAAAAAAAA== --Sig_/mE=qoOl=qgn_g2GbQPFHogw-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 18:01:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBDF5C5; Tue, 4 Mar 2014 18:01:05 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0209.outbound.protection.outlook.com [207.46.163.209]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 030E32B4; Tue, 4 Mar 2014 18:01:04 +0000 (UTC) Received: from BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) by BL2PR03MB593.namprd03.prod.outlook.com (10.255.109.36) with Microsoft SMTP Server (TLS) id 15.0.888.9; Tue, 4 Mar 2014 18:00:56 +0000 Received: from BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.130]) by BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.167]) with mapi id 15.00.0888.003; Tue, 4 Mar 2014 18:00:56 +0000 From: "Abhishek Gupta (LIS)" To: Maurizio Marini , Mark Felder Subject: RE: freebsd 10.0 not work carp protocol on Hyper-v Thread-Topic: freebsd 10.0 not work carp protocol on Hyper-v Thread-Index: w1iYmb8i77+CCVNy/FC4kgfjKueHkO08X/8AgAAUAgCAAc/YAIAAEZKAgAAD3ACAABWdgIAABLBG Date: Tue, 4 Mar 2014 18:00:55 +0000 Message-ID: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org>, <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> In-Reply-To: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [71.227.189.220] x-forefront-prvs: 01401330D1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(377454003)(199002)(51704005)(189002)(24454002)(52314003)(164054003)(56776001)(69226001)(47736001)(54316002)(79102001)(95416001)(76786001)(86362001)(51856001)(94946001)(63696002)(80022001)(83322001)(19580395003)(92566001)(80976001)(46102001)(86612001)(95666003)(77096001)(94316002)(77982001)(85852003)(50986001)(47976001)(4396001)(76796001)(53806001)(49866001)(83072002)(59766001)(66066001)(76576001)(54356001)(56816005)(15975445006)(76482001)(19580405001)(93516002)(93136001)(33646001)(65816001)(47446002)(85306002)(81542001)(87936001)(2656002)(85806002)(81342001)(16799955002)(81816001)(74876001)(74502001)(31966008)(81686001)(90146001)(87266001)(74366001)(74316001)(74662001)(74706001)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR03MB593; H:BL2PR03MB210.namprd03.prod.outlook.com; CLIP:71.227.189.220; FPR:E665F15C.AC028582.68E5BCB3.40E57949.2037D; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: microsoft.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Cc: Giovanni Mattera , Manuel Martini , "freebsd-virtualization@freebsd.org" , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:01:05 -0000 Hi Giovanni,=0A= =0A= Sorry to come late to the thread. Have you integrated the Hyper-V drivers y= et or are relying on emulated mode? Let's start from there and we shall see= if the problem can be resolved.=0A= Thanks,=0A= Abhishek Gupta=0A= PM, BSD Integration Services=0A= Microsoft Corporation=0A= ________________________________________=0A= From: owner-freebsd-virtualization@freebsd.org on behalf of Maurizio Marini =0A= Sent: Tuesday, March 04, 2014 9:43 AM=0A= To: Mark Felder=0A= Cc: FreeBSD Net; freebsd-virtualization@freebsd.org; Manuel Martini; Giovan= ni Mattera=0A= Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v=0A= =0A= On Tue, 4 Mar 2014 10:25:42 -0600=0A= Mark Felder wrote:=0A= =0A= =0A= > That's unfortunate. If you do a packet dump do you even see the other nod= e's=0A= > advertisements?=0A= >=0A= > CARP is really at the mercy of these half-baked virtual switc=0A= =0A= Dear sir:=0A= =0A= https://forum.pfsense.org/index.php?topic=3D44529.0=0A= =0A= "But shouldn't the CARP state go to "master" anyway, even if it couldn't fi= nd a=0A= live partner due to network/Hyper-V problems?"=0A= =0A= this is the question, forget network/Hyper-V problems, CARP should go MASTE= R=0A= anyway=0A= =0A= =0A= =0A= =0A= --=0A= Cordiali Saluti=0A= Maurizio Marini=0A= CoST - Computers Services and Technologies S.r.l.=0A= Via Longhi, 13 - 20137 Milano=0A= P. IVA 09585780159=0A= Tel +39 02 45446.207=0A= Fax +39 02 45446.333=0A= =0A= Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone=0A= indicate. La diffusione, copia o qualsiasi altra azione derivante dalla=0A= conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te=0A= ricevuto questo documento per errore siete cortesemente pregati di darne=0A= immediata comunicazione al mittente e di provvedere alla sua distruzione.= =0A= Grazie. This e-mail and any attachment are confidential and may contain=0A= privileged information intended for the addressee(s) only. Dissemination,= =0A= copying, printing or use by anybody else is unauthorized. If you are not th= e=0A= intended recipient, please delete this message and any attachment and advis= e=0A= the sender by return e-mail. Thank you=0A= =0A= =0A= =0A= From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 18:18:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14E30852; Tue, 4 Mar 2014 18:18:26 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id A024365B; Tue, 4 Mar 2014 18:18:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id BA819A0400C; Tue, 4 Mar 2014 19:17:25 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id xeMHZV-PQ6Mo; Tue, 4 Mar 2014 19:17:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 0A994A03FFA; Tue, 4 Mar 2014 19:17:25 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 0A994A03FFA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393957045; bh=SRiT5wO0mWWcgEPfGsNcigFpLeUqAhdIY5mmZgxkwyo=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=j2Q5htckY5JbMYsdjFWcK+zcKcv6i0DNHflbF3mJx8YVEaP5JvywnRPwgfiC9QcCl WDAbL+FUWf+Age2iRUim6ZTwe86LFPYBOA8ybFfnoyFMYhVCJv+T9AxjmCWTtSip/1 i1b5JVOzouTdtP8xd2Ztcm/I1K5BAy6WZtwUpKPk= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id K5QG3tG4JzRc; Tue, 4 Mar 2014 19:17:24 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id D8D7AA03FF5; Tue, 4 Mar 2014 19:17:23 +0100 (CET) Date: Tue, 4 Mar 2014 19:18:18 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" , Manuel Martini Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304191818.46acdd8d@tikal.homenet.telecomitalia.it> In-Reply-To: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/HXy/Or554.qUqGQLMk9DG5Y"; protocol="application/pkcs7-signature" Cc: Giovanni Mattera , "freebsd-virtualization@freebsd.org" , Mark Felder , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:18:26 -0000 --Sig_/HXy/Or554.qUqGQLMk9DG5Y Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 18:00:55 +0000 "Abhishek Gupta (LIS)" wrote: > Hi Giovanni, >=20 > Sorry to come late to the thread. Have you integrated the Hyper-V drivers= yet > or are relying on emulated mode? Let's start from there and we shall see = if Yes sir, we did Best rgds Looking forward to hear from you --=20 Cordiali Saluti Maurizio Marini=20 CoST - Computers Services and Technologies S.r.l. Via Longhi, 13 - 20137 Milano P. IVA 09585780159 Tel +39 02 45446.207 Fax +39 02 45446.333 Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbia= te ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie. This e-mail and any attachment are confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorized. If you are not the intended recipient, please delete this message and any attachment and advise the sender by return e-mail. Thank you --Sig_/HXy/Or554.qUqGQLMk9DG5Y Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRjCCAkICAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTgxODE4 WjAjBgkqhkiG9w0BCQQxFgQUi8B6JCYZL1z/dEsJ1TqpbVWz9R8wKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE ggEAE4pAcwZeoNgRkQZXlkydNEiRQ4h+quS4UxUp0AJkRBwBeRb7j6uBDz22m4v6 F4rx+2i3mhQHMqXTns2XnY9B7Ycp07g9oNVXt/ToYTDgmlctgRVkmSvN2usuzI/8 GQjuxgEJLafTqriQ/QYZQgIfUq/eknGOut7Ir653u3bXHvycQZsK2o18Ufzw019n F9ZSphx8hqpd/iUh0tdkHp34wDC7O6s8Hkk2Do84q0YyHQdv0EgMEu4xVOW9H1iw jvgHZXUAAFbZD1yotOH0g2dlFVfeW9iTjupmzIXTcp+fQwzp39MZuoDZ8ivE847p ok9DX/+yZFGKGffG8PIFpOzrVQAAAAAAAA== --Sig_/HXy/Or554.qUqGQLMk9DG5Y-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 18:21:01 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC9E29D8; Tue, 4 Mar 2014 18:21:01 +0000 (UTC) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85BA7687; Tue, 4 Mar 2014 18:21:01 +0000 (UTC) Received: from compute2.internal (compute2.nyi.mail.srv.osa [10.202.2.42]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 0F25121524; Tue, 4 Mar 2014 13:20:59 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute2.internal (MEProxy); Tue, 04 Mar 2014 13:21:00 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id :references:to; s=smtpout; bh=PEGOvRkbJW77wgBMeQg3dA5yTmg=; b=X3 SS/HJbR5hcD5puhX71Xk8/jNXUbOA5yD2Yn5gR2Ax6+JhgtbOgKT2KQQc/VZIj9m FwcdQsYaudlTz56qfxBmD+fDeDBUTOUvwMJnRpf5bDrgI+TGnvIury2GIRVDiVuH spuSj4MECJxMWhq/VEV4EDyyGS1X1V/uQygFd8VS4= X-Sasl-enc: QnNOMgyTrrG7GHEmZazxglYqKynGO892VLXI7mlqifiJ 1393957259 Received: from [172.16.1.145] (unknown [68.117.126.78]) by mail.messagingengine.com (Postfix) with ESMTPA id A83C5680241; Tue, 4 Mar 2014 13:20:58 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v From: Mark Felder In-Reply-To: <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> Date: Tue, 4 Mar 2014 12:20:55 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0968D615-5C7A-4D38-A441-676B05BE2A61@FreeBSD.org> References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> To: Maurizio Marini X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , freebsd-virtualization@freebsd.org, Manuel Martini , Giovanni Mattera X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:21:01 -0000 On Mar 4, 2014, at 11:43, Maurizio Marini = wrote: > "But shouldn't the CARP state go to "master" anyway, even if it = couldn't find a > live partner due to network/Hyper-V problems?" >=20 > this is the question, forget network/Hyper-V problems, CARP should go = MASTER > anyway I will have to test again in that XenServer environment soon but I = believe it also stayed INIT there unless you forced it to MASTER. I = could be wrong, though. You're right though -- it should go to MASTER if it can't talk to the = other members.= From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 18:27:12 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6090FCA2; Tue, 4 Mar 2014 18:27:12 +0000 (UTC) Received: from webmail.cost.it (webmail.cost.it [93.62.222.3]) by mx1.freebsd.org (Postfix) with ESMTP id ED7B977B; Tue, 4 Mar 2014 18:27:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 44EBBA04027; Tue, 4 Mar 2014 19:26:13 +0100 (CET) Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id de2qwf8VFfZY; Tue, 4 Mar 2014 19:26:12 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by webmail.cost.it (Postfix) with ESMTP id 3CD01A04023; Tue, 4 Mar 2014 19:26:12 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.8.4 webmail.cost.it 3CD01A04023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cost.it; s=0DDEFADA-4B04-11E3-B1CE-8F82E0733EE7; t=1393957572; bh=gisVziYwWhl91+4Wl5nOaTukfAav3uWr9efLPmiHmEY=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=Lb/4NuHqdZS+NIWKNWB3TLN5ChAhWmEYKClrl4g7fZ7DhqkaEKHBvWozdeAVxX2a6 mnhiwRq7MFwZ2yjNH+yoe9K0y9UxX3WAhpeYQPE1yjjW1zQrjdgPiY1BretfFcQHUX YfF4kVUV3qU8C66bB7Kjuiix3LySM2SC0AvonE5c= X-Virus-Scanned: amavisd-new at webmail.cost.it Received: from webmail.cost.it ([127.0.0.1]) by localhost (webmail.cost.it [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bVIITUlnFkOG; Tue, 4 Mar 2014 19:26:11 +0100 (CET) Received: from tikal.homenet.telecomitalia.it (host151-34-dynamic.3-79-r.retail.telecomitalia.it [79.3.34.151]) by webmail.cost.it (Postfix) with ESMTPSA id E8E70A03FFA; Tue, 4 Mar 2014 19:26:10 +0100 (CET) Date: Tue, 4 Mar 2014 19:27:05 +0100 From: Maurizio Marini To: "Abhishek Gupta (LIS)" Subject: Re: freebsd 10.0 not work carp protocol on Hyper-v Message-ID: <20140304192705.476db8ef@tikal.homenet.telecomitalia.it> In-Reply-To: References: <77152523.5202330.1393757388176.JavaMail.zimbra@cost.it> <531456AA.50203@freebsd.org> <20140303122850.212f8c18@tikal.homenet.telecomitalia.it> <1393945740.22165.90432305.73C1DA68@webmail.messagingengine.com> <1492436824.5843722.1393949513726.JavaMail.zimbra@cost.it> <2AFDF12C-2655-484E-A217-6BBC2FC5E4A4@FreeBSD.org> <20140304184303.0b34e865@tikal.homenet.telecomitalia.it> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=SHA1; boundary="Sig_/DmSUn9FK1p31vlY+DkpCUmw"; protocol="application/pkcs7-signature" Cc: Giovanni Mattera , Manuel Martini , "freebsd-virtualization@freebsd.org" , Mark Felder , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 18:27:12 -0000 --Sig_/DmSUn9FK1p31vlY+DkpCUmw Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 4 Mar 2014 18:00:55 +0000 "Abhishek Gupta (LIS)" wrote: > Hi Giovanni, >=20 > Sorry to come late to the thread. Have you integrated the Hyper-V drivers= yet > or are relying on emulated mode? Let's start from there and we shall see = if > the problem can be resolved. Thanks, Abhishek Gupta Hello all I can let you access to freebsd boxes with ssh and in rdp into Windows Serv= er 2012 R2 to give a look, let me know your source ip's brgds -m --Sig_/DmSUn9FK1p31vlY+DkpCUmw Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAA oIIUFTCCB4cwggVvoAMCAQICAS0wDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMC SUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdp dGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp ZmljYXRpb24gQXV0aG9yaXR5MB4XDTA2MDkxNzE5NDYzN1oXDTM2MDkxNzE5NDYz NlowfTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMT IFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAwYjbCbxsRnx4n5V7tTOQ8nJi1sE2ICIkXs7pd/JD CqIGZKTMjjb4OOYj8G5tsTzdcqOFHKHTPbQzK9Mvr/7qsEFZZ7bEBn0KnnSF1nlM gDd63zkFUln39BtGQ6TShYXSw3HzdWI0uiyKfx6P7u000BHHls1SPboz1t1N3gs7 SkufwiYv+rUWHHI1d8o8XebK4SaLGjZ2XAHbdBQl/u21oIgP3XjKLR8HlzABLXJ5 +kbWEyqouaarg0kd5fLv3eQBjhgKj2NTFoViqQ4ZOsy1ZqbCa3QH5Cvhdj60bdj2 ROFzYh87xL6gU1YlbFEJ96qryr92/W2b853bvz1mvAxWqq+YSJU6S9+nWFDZOHWp W+pDDAL/mevobE1wWyllnN2qXcyvATHsDOvSjejqnHvmbvcnZgwaSNduQuM/3iE+ e+ENcPtjqqhsGlS0XCV6yaLJixamuyx+F14FTVhuEh0B7hIQDcYyfxj//PT6zW6R 6DZJvhpIaYvClk0aErJpF8EKkNb6eSJIv7p7afhwx/p6N9jYDdJ2T1f/kLfjkdLd 78Jgt2c63f6qnPDUi39yIs7Gn5e2+K+KoBCo2fsYxra1XFI8ibYZKnMBCg8DsxJg 8novgdujbv8mMJf1i92JV7atPbOvK8W3dgLwpdYrmoYUKnL24zOMXQlLE9+7jHQT UksCAwEAAaOCAhAwggIMMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEG MB0GA1UdDgQWBBROC+8apEBbpRdphzDKNGhD0EGu8jAfBgNVHSMEGDAWgBROC+8a pEBbpRdphzDKNGhD0EGu8jCCAVoGA1UdIASCAVEwggFNMIIBSQYLKwYBBAGBtTcB AQEwggE4MC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp Y3kucGRmMDQGCCsGAQUFBwIBFihodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRl cm1lZGlhdGUucGRmMIHPBggrBgEFBQcCAjCBwjAnFiBTdGFydCBDb21tZXJjaWFs IChTdGFydENvbSkgTHRkLjADAgEBGoGWTGltaXRlZCBMaWFiaWxpdHksIHJlYWQg dGhlIHNlY3Rpb24gKkxlZ2FsIExpbWl0YXRpb25zKiBvZiB0aGUgU3RhcnRDb20g Q2VydGlmaWNhdGlvbiBBdXRob3JpdHkgUG9saWN5IGF2YWlsYWJsZSBhdCBodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMBEGCWCGSAGG+EIBAQQEAwIA BzA4BglghkgBhvhCAQ0EKxYpU3RhcnRDb20gRnJlZSBTU0wgQ2VydGlmaWNhdGlv biBBdXRob3JpdHkwDQYJKoZIhvcNAQELBQADggIBAI6P59yUeXzxhX+fSW9ryl37 jP4ExcFi0X1CirxTt5QDZjA/secKp1AgVSV/dnoUDesEDkDmPtiIqwcng6l1pjdz x/1L0k2tF0DIRr47f1H8w7YFMdzNhSJOcbfycV6wGsa6k4t4kkqF+HgPg/4vrSz3 5KS7LdDnDTq4Ps72ePauRyTKozU2zsfGh5ja7Pvpss4nm4jDBKH2C1lor8nbEA9N 9mRjXKUSb5Kyk5THiBcOk7Z+YouQf6tOn/zjdRRPKjLfWw3g9XuTDauhz4fhpQRF 6DwSpQnFsNG3U/NgFLqFaWohfB91YRcgF3tsO0EpXOGsWtHNjJvrYB0Z7PflsNr5 eRilRT9JQ1fS3STVLKP9kY0nteXrFAaaTHshuzqtMAYYwNjBayx/WVxdkbFwIlfr imtIStUPKezGQMAviExoARd39CQZT7364bIgIUvdGtgpfaq43lTsIVWAbB71MMij EOWy5ioUMcOFLYyYsYZaT4lZLbnH9xzIin/AnQVK5kJPYqNtKaQfhavb5YHIrSo9 TF1bhCZxxIVecSTKpRts2GHTGuBU2866qTK1IvZzQQlduBddDg+ZkNZH2m8KOmIo FGeC2fHQgFmbyzHYmw+Md061aIrybPYkDi1scMVz0d4U0HGPttN7AvbjuNQJbmue dYQ55n8lpfJIAMCkAdo/MIIGNDCCBBygAwIBAgIBHjANBgkqhkiG9w0BAQUFADB9 MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi U2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3Rh cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDcxMDI0MjEwMTU1WhcN MTcxMDI0MjEwMTU1WjCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25p bmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh dGUgQ2xpZW50IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxwmD zM4t2BqxKaQuE6uWvooyg4ymiEGWVUet1G8SD+rqvyNH4QrvnEIaFHxOhESip7vM z39ScLpNLbL1QpOlPW/tFIzNHS3qd2XRNYG5Sv9RcGE+T4qbLtsjjJbi6sL7Ls/f /X9ftTyhxvxWkf8KW37iKrueKsxw2HqolH7GM6FX5UfNAwAu4ZifkpmZzU1slBhy WwaQPEPPZRsWoTb7q8hmgv6Nv3Hg9rmA1/VPBIOQ6SKRkHXG0Hhmq1dOFoAFI411 +a/9nWm5rcVjGcIWZ2v/43Yksq60jExipA4l5uv9/+Hm33mbgmCszdj/Dthf13tg Av2O83hLJ0exTqfrlwIDAQABo4IBrTCCAakwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFFNy7ZKc4NrLAVx8fpY1TvLUuFGCMB8GA1Ud IwQYMBaAFE4L7xqkQFulF2mHMMo0aEPQQa7yMGYGCCsGAQUFBwEBBFowWDAnBggr BgEFBQcwAYYbaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL2NhMC0GCCsGAQUFBzAC hiFodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcnQwWwYDVR0fBFQwUjAn oCWgI4YhaHR0cDovL3d3dy5zdGFydHNzbC5jb20vc2ZzY2EuY3JsMCegJaAjhiFo dHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwgYAGA1UdIAR5MHcwdQYL KwYBBAGBtTcBAgEwZjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjA0BggrBgEFBQcCARYoaHR0cDovL3d3dy5zdGFydHNzbC5j b20vaW50ZXJtZWRpYXRlLnBkZjANBgkqhkiG9w0BAQUFAAOCAgEACoMIfXirLAZc uGOMXq4cuSN3TaFx2H2GvD5VSy/6rV55BYHbWNaPeQn3oBSU8KgQZn/Kck1JxbLp AxVCNtsxeW1R87ifhsYZ0qjdrA9anrW2MAWCtosmAOT4OxK9QPoSjCMxM3HbkZCD JgnlE8jMopH21BbyAYr7b5EfGRQJNtgWcvqSXwKHnTutR08+Kkn0KAkXCzeQNLeA 5LlYUzFyM7kPAp8pIRMQ+seHunmyG642S2+y/qHEdMuGIwpfz3eDF1PdctL04qYK /zu+Qg1Bw0RwgigVZs/0c5HP2/e9DBHh7eSwtzYlk4AUr6yxLlcwSjOfOmKEQ/Q8 tzh0IFiNu9IPuTGAPBn4CPxD0+Ru8T2wg8/s43R/PT3kd1OEqOJUl7q+h+r6fpvU 0Fzxd2tC8Ga6fDEPme+1Nbi+03pVjuZQKbGwKJ66gEn06WqaxVZC+J8hh/jR0k9m ST1iAZPNYulcNJ8tKmVtjYsv0L1TSm2+NwON58tO+pIVzu3DWwSEXSf+qkDavQam +QtEOZxLBXI++aMUEapSn+k3Lxm48ZCYfAWLb/Xj7F5JQMbZvCexglAbYR0kIHqW 5DnsYSdMD/IplJMojx0NBrxJ3fN9dvX2Y6BIXRsF1du4qESm4/3CKuyUV7p9DW3m PlHTGLvYxnyKQy7VFBkoLINszBrOUeIwggZOMIIFNqADAgECAgMGioEwDQYJKoZI hvcNAQEFBQAwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQKEw1TdGFydENvbSBMdGQu MSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0ZSBTaWduaW5nMTgw NgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkgSW50ZXJtZWRpYXRlIENs aWVudCBDQTAeFw0xMzA1MDUxMDM3MjhaFw0xNDA1MDYwMTQ2MzFaMGUxGTAXBgNV BA0TEDg2UTJiOW9MTTExcVVudzMxIDAeBgNVBAMMF21hdXJpemlvLm1hcmluaUBj b3N0Lml0MSYwJAYJKoZIhvcNAQkBFhdtYXVyaXppby5tYXJpbmlAY29zdC5pdDCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKXx4b3RfL4AZyuKcTcf8n4f fP+pu4UvHWit7MX6Sax/Dm+RDVFb+QL8LR5kH+wV7k/u1tnKzn+u1RtVrf5uncbX m3GK2+dKsOlqwYkvHW8ubp+Y2eA+yuNSdZag6B9/NMJWxJ8/VcOZLgAPr/wQK31Q LxdspLDOCRicZLcT+wse5YXl7lbtYM9H1pE8IIyHSSDpILzyHLmmBjGbKAJHnHVr 83RXce6fZ+8QVa030z6l0rJHgjhp87vINZfR9c2lD9WDiNmGTzYH0tfxk+uZJA0E 5AlqfxXFPumtlZLY3jQjukZkOEovwajJR8u74t5Wz4HFjOqmTVAcyFSyrwQX5CcC AwEAAaOCAt0wggLZMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQUDsk9HZ3iSb0KEccS4Hj5qhB0 cRAwHwYDVR0jBBgwFoAUU3Ltkpzg2ssBXHx+ljVO8tS4UYIwIgYDVR0RBBswGYEX bWF1cml6aW8ubWFyaW5pQGNvc3QuaXQwggFMBgNVHSAEggFDMIIBPzCCATsGCysG AQQBgbU3AQIDMIIBKjAuBggrBgEFBQcCARYiaHR0cDovL3d3dy5zdGFydHNzbC5j b20vcG9saWN5LnBkZjCB9wYIKwYBBQUHAgIwgeowJxYgU3RhcnRDb20gQ2VydGlm aWNhdGlvbiBBdXRob3JpdHkwAwIBARqBvlRoaXMgY2VydGlmaWNhdGUgd2FzIGlz c3VlZCBhY2NvcmRpbmcgdG8gdGhlIENsYXNzIDEgVmFsaWRhdGlvbiByZXF1aXJl bWVudHMgb2YgdGhlIFN0YXJ0Q29tIENBIHBvbGljeSwgcmVsaWFuY2Ugb25seSBm b3IgdGhlIGludGVuZGVkIHB1cnBvc2UgaW4gY29tcGxpYW5jZSBvZiB0aGUgcmVs eWluZyBwYXJ0eSBvYmxpZ2F0aW9ucy4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDov L2NybC5zdGFydHNzbC5jb20vY3J0dTEtY3JsLmNybDCBjgYIKwYBBQUHAQEEgYEw fzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3RhcnRzc2wuY29tL3N1Yi9jbGFz czEvY2xpZW50L2NhMEIGCCsGAQUFBzAChjZodHRwOi8vYWlhLnN0YXJ0c3NsLmNv bS9jZXJ0cy9zdWIuY2xhc3MxLmNsaWVudC5jYS5jcnQwIwYDVR0SBBwwGoYYaHR0 cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3DQEBBQUAA4IBAQBScVXI8hOj Vj37HPAB66Xv9PW3PKojeShUdJwhh3BSlyABD/8/+Up0R8l8f9Ba7214ozcCkzZb bDPnrhtLqHGYKOJQ37/rCqSz31eFsBEfEbyrhldH6EYQ7hB1o2H3xigjNhAytU4a wj2D+DvHZbxh6aEZH91S6LbNV6Jax00Yxb5IWacP02NlUqR3RkObBjxvjIgjj62x 6SfcrAfWPBbCvY450zuP+rgxmdDBmKhTtaBDgz+VrsKt7ayVCg+2SY3AC82RNu0n xsyE5oHknqCpaw1zu06Dp0Qlgi9XHCQ5PjuaGoKGfznBoBenMs3GLFwKN6ekwXeO VwIU9j1CQ9g/MYICRDCCAkACAQEwgZQwgYwxCzAJBgNVBAYTAklMMRYwFAYDVQQK Ew1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZp Y2F0ZSBTaWduaW5nMTgwNgYDVQQDEy9TdGFydENvbSBDbGFzcyAxIFByaW1hcnkg SW50ZXJtZWRpYXRlIENsaWVudCBDQQIDBoqBMAkGBSsOAwIaBQCggYcwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMzA0MTgyNzA1 WjAjBgkqhkiG9w0BCQQxFgQU1W9irarUh4Gx3Wzzyc/2r2BomtowKAYJKoZIhvcN AQkPMRswGTALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDQYJKoZIhvcNAQEBBQAE gf/CR/XCs0Kf6rSp8x0D10CvVVN0NJxdbfLrT0D76nDiEnBqjeRXu9CiKQjZSqaV bjr9ULm03oQLyTFwlD4P+i3SS56MC4VJ2Amu9goEfz4hipvl/fAteXZCbRoBcod3 bHwAuBmo+tf8KbxwghC06ablO2Gnrg8SBJQE0/j1H1u7tdXKmhOL8mm8ehft4CQ5 jQpisH4s5m9byJygd1vdW/1gx+8prPAg/2CyvMyjFExsUW0sNZm1j1obrnO1wM3+ UK8qxQc08sfyxWV6kYNgDOPd5NYMKxIIuVYxSZiLU4JDh7eObacqnjb3QALCXZQh inu4umHHC6nhh+0go9vAgeMAAAAAAAA= --Sig_/DmSUn9FK1p31vlY+DkpCUmw-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 20:49:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E690A92; Tue, 4 Mar 2014 20:49:02 +0000 (UTC) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1766E8E7; Tue, 4 Mar 2014 20:49:01 +0000 (UTC) Received: by mail-la0-f41.google.com with SMTP id gl10so50974lab.14 for ; Tue, 04 Mar 2014 12:49:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=Z8P4BDrIhh9r1xUoV/0iARqfWwchpxciPoSgTl1OINU=; b=YjY3uQUoG4iVPuKAdawUdDl6YKb8SaIim/4vrwTFwOFYwKWDjX9zZURinX5bYqw6z1 +bgvQMbPUTtj8DRbVk8d6PndD6QVIVN5/vtdoar5yMMzG21Z+n/Bpav03P4u4MeAfS1N JBn2cIqnxX3Hkwkjza9VLqtrieuDIDAMAvz3Yk+0SOMxCVQYRpQsuqCbvQam9AEFNNP+ aBAxyzvFhMklmfouIskk7tEaikc4oKSkvYAM6yR1ehE4WAQGwYlO2MtbfeSojzFt3+Nh UJ8Nbz3MfyFLUMmgQdctTs0UDL3w1wy6wKbOxpWIOlD5dMOuQAJ0Z5q/42htSOU5lEOM fLnQ== MIME-Version: 1.0 X-Received: by 10.112.14.1 with SMTP id l1mr769754lbc.39.1393966140069; Tue, 04 Mar 2014 12:49:00 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Tue, 4 Mar 2014 12:48:59 -0800 (PST) Date: Tue, 4 Mar 2014 12:48:59 -0800 X-Google-Sender-Auth: 5rZCECGGu-6T2itWYahnxTFrIag Message-ID: Subject: Problem with ZFS userboot changes r262331 and VM with mixed partitions From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: Xin Li , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 20:49:03 -0000 Hi, I encountered problems with the new changes to userboot to boot ZFS. About a month ago, I created a FreeBSD 10 VM with the following layout: => 34 419430333 md0 GPT (200G) 34 128 1 freebsd-boot (64K) 162 2097152 2 freebsd-ufs (1.0G) 2097314 8388608 3 freebsd-swap (4.0G) 10485922 408944444 4 freebsd-zfs (195G) 419430366 1 - free - (512B) The FreeBSD root file system is on p4, which is ZFS. On the root file system, I have a symlink: /boot -> /bootdir /bootdir mounts p2 which is UFS. So in this scenario, the kernel lives on UFS. When I did this about a month ago, during the BHyve boot process, (1) userboot loaded the kernel from p2 (UFS) (2) The kernel booted, loaded zfs.ko, and then proceeded to mount p4 where the root file system lived (ZFS) After r262331, when I try to boot the BHyve VM (1) I can see p4 from the loader prompt (2) The kernel doesn't load (3) If I try to load /boot/kernel/kernel from the loader prompt, I get: ZFS: i/o error - all block copies unavailable I don't fully understand all this logic. Is there a bug that can be fixed here? -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 22:24:56 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0FFC5F6; Tue, 4 Mar 2014 22:24:56 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 5E5E3343; Tue, 4 Mar 2014 22:24:56 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 4ED4712212; Wed, 5 Mar 2014 08:24:49 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSG63615 (AUTH peterg@ptree32.com.au); Wed, 5 Mar 2014 08:24:48 +1000 Message-ID: <531652AE.9070506@freebsd.org> Date: Tue, 04 Mar 2014 14:24:46 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "jenkins-admin@freebsd.org" , Xin Li , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 22:24:56 -0000 Hi Craig, > ZFS: i/o error - all block copies unavailable > > I don't fully understand all this logic. Is there a bug that can be fixed here? Yes - a flag needs to be passed to userboot to tell it to ignore ZFS filesystems. This is analogous to the decision to use boot2 (implicit UFS) vs zfsboot (explicit ZFS) as the 2nd-stage loader on a real disk. It didn't look possible to do this without changing the userboot ABI, but, it's probably time to bump that. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 22:45:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B60A4B68; Tue, 4 Mar 2014 22:45:00 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ACAD3767; Tue, 4 Mar 2014 22:44:59 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so133740lan.12 for ; Tue, 04 Mar 2014 14:44:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=RQ627icaWcFVUSb7eENB4s4D1kVzQ1bBcWR7dE86Uzc=; b=MZsxCjuxu5JxF9H+WfCyrQt6UKLHfr6ZmeVahU0Qtb2hqrps74xy5XRpZAJaU6ug/V ewp0sZWYAak9zCsHncxs8+q2NcDx2dHJvRnjECmdBt/pnMTIdsCCAgySmF16ZL03H9eE mQggTO4MMI8L0G1eY22cdQF7LXEsKgJkl1VdHsJBA6NQVQlmlC4ci5uYSBH1+Py0b+e1 RunvI87fAwxaeLDSxyF26emcZUJFMJtLfu4AccP4EhsNkCLjzhHTAQ+Ij8yU09t8ke5P ugOVY3zjTbiKZQLR0c1HrWUCoA7+RS5gEhIUIEla9GjTLH5N3HuKh8LkapvnGRstC6AQ i86A== MIME-Version: 1.0 X-Received: by 10.112.160.200 with SMTP id xm8mr1087961lbb.24.1393973097808; Tue, 04 Mar 2014 14:44:57 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Tue, 4 Mar 2014 14:44:57 -0800 (PST) In-Reply-To: <531652AE.9070506@freebsd.org> References: <531652AE.9070506@freebsd.org> Date: Tue, 4 Mar 2014 14:44:57 -0800 X-Google-Sender-Auth: s_v6ZDuPm0-CMxYpL6avjj2UUYQ Message-ID: Subject: Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions From: Craig Rodrigues To: Peter Grehan Content-Type: text/plain; charset=ISO-8859-1 Cc: "jenkins-admin@freebsd.org" , Xin Li , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 22:45:00 -0000 On Tue, Mar 4, 2014 at 2:24 PM, Peter Grehan wrote: > Hi Craig, > > >> ZFS: i/o error - all block copies unavailable >> >> I don't fully understand all this logic. Is there a bug that can be fixed >> here? > > > Yes - a flag needs to be passed to userboot to tell it to ignore ZFS > filesystems. This is analogous to the decision to use boot2 (implicit UFS) > vs zfsboot (explicit ZFS) as the 2nd-stage loader on a real disk. > > It didn't look possible to do this without changing the userboot ABI, but, > it's probably time to bump that. Thanks for the explanation. Can this flag be passed via bhyveload, and if so, how? Right now the workaround was to recompile userboot with: cd /usr/src/sys/boot/userboot make clean make WITHOUT_ZFS=yes make WITHOUT_ZFS=yes install -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 4 22:47:35 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D048DEAD; Tue, 4 Mar 2014 22:47:35 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8D8C77C6; Tue, 4 Mar 2014 22:47:35 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 25B0712147; Wed, 5 Mar 2014 08:47:34 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSG65151 (AUTH peterg@ptree32.com.au); Wed, 5 Mar 2014 08:47:33 +1000 Message-ID: <53165803.2000404@freebsd.org> Date: Tue, 04 Mar 2014 14:47:31 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions References: <531652AE.9070506@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "jenkins-admin@freebsd.org" , Xin Li , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 22:47:35 -0000 Hi Craig, > Can this flag be passed via bhyveload, and if so, how? It can't yet: the API to userboot has to be changed to allow this to be passed through. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 5 05:58:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA6E5E03; Wed, 5 Mar 2014 05:58:36 +0000 (UTC) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8DF4C68F; Wed, 5 Mar 2014 05:58:36 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h18so3499520igc.2 for ; Tue, 04 Mar 2014 21:58:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=AbevK7wey7XfWFXjZHsCkw+OMJvXVov34NijrsKY1TI=; b=yHNN/egZfvAznMEsndUsTlImFScWthkEqjsWeK6Rb4DinzpbblfE+IKqvMYgjTmi32 RCHGzD7WoBPL6HdKDQcKtZD9ln57BO5+2zGtdfTvbX5foqhH+sqhlOHxpfFmm7E6cIQf 5XUS56Wz2o0rdoiB6zJPnMFfxX7PLpAe7BENI1hDFlkrR5HUrkwRICPEpgMICMCbtnZk ioLwkaIKPFVa9VECY6nuM8W5Pl7VZxB1Sncp46ZKBAeX6xtMKmac5BHQGBese1Xa6oZO jSibXwEhk7ZE7/Q/12cmHmGWGRV6F4YOtj6l8wQCKxVgrgwIxSbQt3aR8STh7d+kmnS8 wDRw== X-Received: by 10.50.66.129 with SMTP id f1mr7330118igt.26.1393999115840; Tue, 04 Mar 2014 21:58:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.192.67 with HTTP; Tue, 4 Mar 2014 21:58:15 -0800 (PST) In-Reply-To: <5315B3B1.5080909@FreeBSD.org> References: <794fb75db92a4df0991a147919727277@BL2PR03MB210.namprd03.prod.outlook.com> <5268B533.80408@FreeBSD.org> <5315B3B1.5080909@FreeBSD.org> From: Oleg Sidorkin Date: Wed, 5 Mar 2014 09:58:15 +0400 Message-ID: Subject: Re: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project To: Alexander Motin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 05:58:36 -0000 That's great! Thanks a lot. On Tue, Mar 4, 2014 at 3:06 PM, Alexander Motin wrote: > Oleg, > > It was found that real problem is not there. You do not need manual > scanning. You should remove it. All you really need is to properly report > bus to CAM: > > @@ -1147,7 +1062,7 @@ > cpi->hba_eng_cnt = 0; > cpi->max_target = STORVSC_MAX_TARGETS; > cpi->max_lun = sc->hs_drv_props->drv_max_luns_per_target; > - cpi->initiator_id = 0; > + cpi->initiator_id = cpi->max_lun + 1; > cpi->bus_id = cam_sim_bus(sim); > cpi->base_transfer_speed = 300000; > cpi->transport = XPORT_SAS; > > Default scanner skips initiator from the process, so initiator_id = 0 > excluded from scan the only really used target. > > > On 04.03.2014 12:37, Oleg Sidorkin wrote: >> >> Hi again. >> >> Disabling scan_for_luns leaves drives undetected. >> >> Calling xpt_rescan for each lun works for me. With the attached patch >> system boots and detects all configured drives. >> But also this patch introduces a race between drives detection and >> boot process, so sometimes system tries to mount undetected drive. >> I'm going to fix this by calling xpt_hold_boot() before xpt_rescan() >> and calling xpt_release_boot() in callback. >> >> Thanks >> >> On Thu, Oct 24, 2013 at 9:50 AM, Alexander Motin wrote: >>> >>> Hi. >>> >>> I took some look and think problems are in scan_for_luns() routine: >>> - After the locking changes scanning normally uses different locks, not >>> the >>> SIM one. That probably caused panic. >>> - But I think that scanning is simply not needed there -- FreeBSD CAM >>> scans >>> every new bus automatically on registration (Even for late registered >>> buses >>> it is done I think at least since FreeBSD 8). I think everything should >>> just >>> work if you remove scan_for_luns() at all. >>> - If you still wish to force scan (due to having information about >>> changed >>> list of devices, etc), then you can make CAM do all the magic for you by >>> calling xpt_rescan(). >>> >>> >>> On 24.10.2013 08:34, Oleg Sidorkin wrote: >>>> >>>> >>>> Hello again. >>>> >>>> Camlock patches are now committed and -CURRENT on Hyper-V now panics >>>> with almost the same stacktrace: >>>> >>>> FreeBSD 11.0-CURRENT #16 r257016: Wed Oct 23 21:08:44 UTC 2013 >>>> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 >>>> FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 >>>> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.57-MHz K8-class CPU) >>>> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >>>> Stepping = 7 >>>> >>>> ...... >>>> >>>> ZFS filesystem version: 5 >>>> ZFS storage pool version: features support (5000) >>>> Timecounters tick every 10.000 msec >>>> storvsc0 on vmbus0 >>>> kernel trap 12 with interrupts disabled >>>> >>>> >>>> Fatal trap 12: page fault while in kernel mode >>>> cpuid = 0; apic id = 00 >>>> fault virtual address = 0x20 >>>> fault code = supervisor read data, page not present >>>> instruction pointer = 0x20:0xffffffff804f58cc >>>> stack pointer = 0x28:0xfffffe011dd5f5d0 >>>> frame pointer = 0x28:0xfffffe011dd5f600 >>>> code segment = base 0x0, limit 0xfffff, type 0x1b >>>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>>> processor eflags = resume, IOPL = 0 >>>> current process = 0 (hv_control_1 taskq) >>>> [ thread pid 0 tid 100047 ] >>>> Stopped at turnstile_broadcast+0x8c: movq >>>> 0x20(%rbx,%rax,1),%rdx >>>> db> bt >>>> Tracing pid 0 tid 100047 td 0xfffff8000331e000 >>>> turnstile_broadcast() at turnstile_broadcast+0x8c/frame >>>> 0xfffffe011dd5f600 >>>> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfffffe011dd5f630 >>>> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011dd5f640 >>>> _sleep() at _sleep+0x18e/frame 0xfffffe011dd5f6c0 >>>> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011dd5f7f0 >>>> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011dd5f890 >>>> device_attach() at device_attach+0x3a2/frame 0xfffffe011dd5f8f0 >>>> hv_vmbus_child_device_register() at >>>> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011dd5f990 >>>> vmbus_channel_process_offer() at >>>> vmbus_channel_process_offer+0x133/frame 0xfffffe011dd5f9d0 >>>> work_item_callback() at work_item_callback+0x26/frame 0xfffffe011dd5f9f0 >>>> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >>>> 0xfffffe011dd5fa40 >>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >>>> 0xfffffe011dd5fa70 >>>> fork_exit() at fork_exit+0x9a/frame 0xfffffe011dd5fab0 >>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011dd5fab0 >>>> --- trap 0, rip = 0, rsp = 0xfffffe011dd5fb70, rbp = 0 --- >>>> >>>> >>>> Thanks >>>> >>>> On Tue, Sep 24, 2013 at 3:04 AM, Abhishek Gupta (LIS) >>>> wrote: >>>>> >>>>> >>>>> Hi Oleg, >>>>> >>>>> Please give us some time. I shall look at it. Thanks for reporting. >>>>> >>>>> Regards, >>>>> Abhishek >>>>> >>>>> -----Original Message----- >>>>> From: owner-freebsd-virtualization@freebsd.org >>>>> [mailto:owner-freebsd-virtualization@freebsd.org] On Behalf Of Oleg >>>>> Sidorkin >>>>> Sent: Monday, September 23, 2013 7:21 AM >>>>> To: freebsd-virtualization@freebsd.org >>>>> Cc: Alexander Motin >>>>> Subject: [Hyper-V][camlock] storvsc driver panics during boot with >>>>> patches from camlock project >>>>> >>>>> Hello. >>>>> >>>>> I'm running the latest current (amd64) under Hyper-V with hyper-v >>>>> services enabled. >>>>> If camlock patches are applied >>>>> >>>>> (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), >>>>> I'm hitting the following kernel panic during boot: >>>>> >>>>> FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 >>>>> olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD >>>>> clang >>>>> version 3.3 (tags/RELEASE_33/final 183502) 20130610 >>>>> CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class >>>>> CPU) >>>>> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >>>>> Stepping = >>>>> 7 >>>>> .... >>>>> Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 ZFS >>>>> NOTICE: >>>>> Prefetch is disabled by default if less than 4GB of RAM is present; >>>>> to enable, add "vfs.zfs.prefetch_disable=0" to >>>>> /boot/loader.conf. >>>>> ZFS filesystem version: 5 >>>>> ZFS storage pool version: features support (5000) Timecounters tick >>>>> every >>>>> 10.000 msec >>>>> storvsc0 on vmbus0 >>>>> Netvsc initializing... SMP: AP CPU #3 Launched! >>>>> SMP: AP CPU #2 Launched! >>>>> SMP: AP CPU #1 Launched! >>>>> kernel trap 12 with interrupts disabled >>>>> >>>>> >>>>> Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 >>>>> fault virtual address = 0x20 >>>>> fault code = supervisor read data, page not present >>>>> instruction pointer = 0x20:0xffffffff804f444c >>>>> stack pointer = 0x28:0xfffffe011df38610 >>>>> frame pointer = 0x28:0xfffffe011df38640 >>>>> code segment = base 0x0, limit 0xfffff, type 0x1b >>>>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>>>> processor eflags = resume, IOPL = 0 >>>>> current process = 0 (hv_control_1 taskq) >>>>> [ thread pid 0 tid 100046 ] >>>>> Stopped at turnstile_broadcast+0x8c: movq >>>>> 0x20(%rbx,%rax,1),%rdx >>>>> db> bt >>>>> Tracing pid 0 tid 100046 td 0xfffff80001f20490 >>>>> turnstile_broadcast() at turnstile_broadcast+0x8c/frame >>>>> 0xfffffe011df38640 >>>>> __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame >>>>> 0xfffffe011df38670 >>>>> unlock_mtx() at unlock_mtx+0x2a/frame 0xfffffe011df38680 >>>>> _sleep() at _sleep+0x18e/frame 0xfffffe011df38700 >>>>> cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfffffe011df38800 >>>>> storvsc_attach() at storvsc_attach+0x6d4/frame 0xfffffe011df388a0 >>>>> device_attach() at device_attach+0x396/frame 0xfffffe011df388f0 >>>>> hv_vmbus_child_device_register() at >>>>> hv_vmbus_child_device_register+0xdb/frame 0xfffffe011df38990 >>>>> vmbus_channel_process_offer() at >>>>> vmbus_channel_process_offer+0x133/frame 0xfffffe011df389d0 >>>>> work_item_callback() at work_item_callback+0x26/frame >>>>> 0xfffffe011df389f0 >>>>> taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame >>>>> 0xfffffe011df38a40 >>>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame >>>>> 0xfffffe011df38a70 >>>>> fork_exit() at fork_exit+0x9a/frame 0xfffffe011df38ab0 >>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011df38ab0 >>>>> --- trap 0, rip = 0, rsp = 0xfffffe011df38b70, rbp = 0 --- >>>>> db> >>>>> >>>>> >>>>> This patch is not commited yet (CFT thread with changes description is >>>>> here: >>>>> >>>>> http://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043333.html), >>>>> but it is going to be commited till the end of the year. >>>>> >>>>> As far as I understand, the invocation chain is >>>>> storvsc_attach->scan_for_luns->cam_periph_runccb >>>>> >>>>> Thanks >>>>> -- >>>>> Oleg Sidorkin >>>>> _______________________________________________ >>>>> freebsd-virtualization@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>>>> To unsubscribe, send any mail to >>>>> "freebsd-virtualization-unsubscribe@freebsd.org" >>>> >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Alexander Motin >> >> >> >> > > > -- > Alexander Motin -- Oleg Sidorkin From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 5 13:00:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8FA9FD2 for ; Wed, 5 Mar 2014 13:00:40 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4F4C5693 for ; Wed, 5 Mar 2014 13:00:40 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id pv20so674168lab.10 for ; Wed, 05 Mar 2014 05:00:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=h2jD00iW41NWBEYpZ0+49gGNwMnrt2JSnvYwTU3ZbL8=; b=VQBSGpWgDtPZs1Yr634XAuiWwYXMCXqcf9nPW6+dqyw40JmxG9LzvmDjP3rC1myCaH MRKEvUTSq5RhDEu7y7vW7L1RIRkfF+NK+ia1VkyNHjz2w2BmvwQcymg2qTVHPR8Yw9ot lHvvB4EX2U9GYbDPUZMa+QDFckoDkZ2Uj7rWQMmORd5Tu9RY3yLFTxqgIAkBAYdTFsc8 0ndcIgpaQo3R5S+0bOK+xxfridRxwlyBViXAAplmweXgCqqb5/AagERH8c8kP3OapOSk yBiN4c18P1BoiZmEO0zN9e6GeyL8qc7DFNZm2k0zXjhjSXJ9PGL6lY4yGVlWHSb9byMT rDOQ== X-Received: by 10.152.20.1 with SMTP id j1mr142283lae.83.1394024438380; Wed, 05 Mar 2014 05:00:38 -0800 (PST) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id x5sm2581706lbk.5.2014.03.05.05.00.37 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Mar 2014 05:00:37 -0800 (PST) Sender: Roman Bogorodskiy Date: Wed, 5 Mar 2014 17:00:32 +0400 From: Roman Bogorodskiy To: freebsd-virtualization@freebsd.org Subject: bhyve: allow specifiying tty by fd Message-ID: <20140305130030.GA40560@dev.san.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="gBBFr7Ir9EOA20Yy" Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 13:00:40 -0000 --gBBFr7Ir9EOA20Yy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, Currently bhyve(8) allows to specify either stdio or device path as a TTY device. However, it could be useful to specify a TTY device by file descriptor, e.g. when bhyve is being executed in an automated way by other application, so a result of openpty(3) could be passed to it. Attached a poc patch for that. It allows to specify fd this way: bhyve -s 31,lpc -l com1,fd=19 vm0 Roman Bogorodskiy --gBBFr7Ir9EOA20Yy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="bhyve_tty_fd.diff" Index: bhyve.8 =================================================================== --- bhyve.8 (revision 262780) +++ bhyve.8 (working copy) @@ -184,6 +184,8 @@ the bhyve process. .It Pa /dev/xxx Use the host TTY device for serial port I/O. +.It Li fd=N +Use a specified file descriptor of the TTY device. .El .Pp Pass-through devices: Index: uart_emul.c =================================================================== --- uart_emul.c (revision 262780) +++ uart_emul.c (working copy) @@ -585,7 +585,11 @@ retval = -1; - fd = open(opts, O_RDWR); + if (!strncmp(opts, "fd=", 3)) { + fd = atoi(opts+3); + } else { + fd = open(opts, O_RDWR); + } if (fd > 0 && isatty(fd)) { sc->tty.fd = fd; sc->tty.opened = true; --gBBFr7Ir9EOA20Yy-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 5 15:22:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8FD994DD for ; Wed, 5 Mar 2014 15:22:42 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 666A57F5 for ; Wed, 5 Mar 2014 15:22:42 +0000 (UTC) Received: from torb.pix.net (torb.pix.net [IPv6:2001:470:e254:10:12dd:b1ff:febf:eca9]) (authenticated bits=0) by hydra.pix.net (8.14.5/8.14.5) with ESMTP id s25FMeGg045307; Wed, 5 Mar 2014 10:22:40 -0500 (EST) (envelope-from lidl@pix.net) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98 at mail.pix.net Message-ID: <53174140.5040004@pix.net> Date: Wed, 05 Mar 2014 10:22:40 -0500 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 15:22:42 -0000 > Hi, > > I encountered problems with the new changes to userboot > to boot ZFS. > > About a month ago, I created a FreeBSD 10 VM with the following > layout: > > => 34 419430333 md0 GPT (200G) > 34 128 1 freebsd-boot (64K) > 162 2097152 2 freebsd-ufs (1.0G) > 2097314 8388608 3 freebsd-swap (4.0G) > 10485922 408944444 4 freebsd-zfs (195G) > 419430366 1 - free - (512B) > > > The FreeBSD root file system is on p4, which is ZFS. > On the root file system, I have a symlink: > > /boot -> /bootdir > > /bootdir mounts p2 which is UFS. > > So in this scenario, the kernel lives on UFS. > When I did this about a month ago, during the BHyve boot process, > > (1) userboot loaded the kernel from p2 (UFS) > (2) The kernel booted, loaded zfs.ko, and then proceeded to mount p4 where > the root file system lived (ZFS) > > After r262331, when I try to boot the BHyve VM > > (1) I can see p4 from the loader prompt > (2) The kernel doesn't load > (3) If I try to load /boot/kernel/kernel from the loader prompt, I get: > > ZFS: i/o error - all block copies unavailable > > I don't fully understand all this logic. Is there a bug that can be fixed here? I ran into this same problem a couple of weeks ago when I first played with the ZFS boot support in userboot.so. The following set of operations on the hypervisor can be used to add bootable support to your zfs installation. (I don't know what the name of your zpool is on the emulated host. If the name of that zpool is the same as the zpool on your hypervisor host, you might have to go at this a different way.) I have taken to naming the zpool for the any clients to be the same as the name of the virtual machine, so I can easily mount it on the host running the hypervisor and fiddle with it. For the purposes of this example, my virtual machine is called "vm0". The zpool for that host lives in a zpool called "zdata" on my hypervisor machine. In my case, my /boot was a symlink that pointed to /bootdir/boot, so you might need to adjust the following slightly. zfs import -f -R /mnt vm0 fsck_ufs /dev/zvol/zdata/vm0p2 mount /dev/zvol/zdata/vm0p2 /mnt/bootdir cd /mnt chflags -h nosunlink boot rm boot cp -R bootdir/* . umount /mnt/bootdir cd / zpool export vm0 Good luck. -Kurt From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 6 18:33:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 219BC58B; Thu, 6 Mar 2014 18:33:40 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id D8DC99FB; Thu, 6 Mar 2014 18:33:39 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id EC1581223C; Fri, 7 Mar 2014 04:33:37 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSI25746 (AUTH peterg@ptree32.com.au); Fri, 7 Mar 2014 04:33:37 +1000 Message-ID: <5318BF7F.4060306@freebsd.org> Date: Thu, 06 Mar 2014 10:33:35 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: bhyve: allow specifiying tty by fd References: <20140305130030.GA40560@dev.san.ru> In-Reply-To: <20140305130030.GA40560@dev.san.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:33:40 -0000 Hi Roman, > Currently bhyve(8) allows to specify either stdio or device path as a > TTY device. However, it could be useful to specify a TTY device by file > descriptor, e.g. when bhyve is being executed in an automated way by > other application, so a result of openpty(3) could be passed to it. > > Attached a poc patch for that. It allows to specify fd this way: > > bhyve -s 31,lpc -l com1,fd=19 vm0 Yes, useful idea, and will become more relevant in the capsicum world of passing fd's to a sandboxed bhyve. Thanks: I'll get this submitted. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 6 18:44:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04D32DB0 for ; Thu, 6 Mar 2014 18:44:14 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CC7A0B23 for ; Thu, 6 Mar 2014 18:44:13 +0000 (UTC) Received: from julian-mbp3.pixel8networks.com (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s26Ii6WA085743 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 6 Mar 2014 10:44:07 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <5318C1F3.4020808@freebsd.org> Date: Thu, 06 Mar 2014 10:44:03 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: bhyve: allow specifiying tty by fd References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> In-Reply-To: <5318BF7F.4060306@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:44:14 -0000 On 3/6/14, 10:33 AM, Peter Grehan wrote: and allows fd to be a socket! :-) > Hi Roman, >> Currently bhyve(8) allows to specify either stdio or device path as a >> TTY device. However, it could be useful to specify a TTY device by >> file >> descriptor, e.g. when bhyve is being executed in an automated way by >> other application, so a result of openpty(3) could be passed to it. >> >> Attached a poc patch for that. It allows to specify fd this way: >> >> bhyve -s 31,lpc -l com1,fd=19 vm0 > > Yes, useful idea, and will become more relevant in the capsicum > world of passing fd's to a sandboxed bhyve. > > Thanks: I'll get this submitted. > > later, > > Peter. > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 6 18:47:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64F64E86; Thu, 6 Mar 2014 18:47:28 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 26CFEB3B; Thu, 6 Mar 2014 18:47:28 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D9E1F12314; Fri, 7 Mar 2014 04:47:26 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSI26018 (AUTH peterg@ptree32.com.au); Fri, 7 Mar 2014 04:47:26 +1000 Message-ID: <5318C2BC.7010303@freebsd.org> Date: Thu, 06 Mar 2014 10:47:24 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Julian Elischer Subject: Re: bhyve: allow specifiying tty by fd References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> <5318C1F3.4020808@freebsd.org> In-Reply-To: <5318C1F3.4020808@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:47:28 -0000 Hi Julian, > and allows fd to be a socket! :-) Maybe not initially. There a bunch of code that wants to issue tty ioctls on the fd, so the code currently verifies that it is indeed a tty. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 7 06:24:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49B9D3A7 for ; Fri, 7 Mar 2014 06:24:29 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0BC0CBA for ; Fri, 7 Mar 2014 06:24:28 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 001FB1217C; Fri, 7 Mar 2014 16:24:26 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSI66856 (AUTH peterg@ptree32.com.au); Fri, 7 Mar 2014 16:24:26 +1000 Message-ID: <53196617.4090706@freebsd.org> Date: Thu, 06 Mar 2014 22:24:23 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "Rudy (bulk)" Subject: Re: [SOLVED] Re: lazy bhyve doesn't boot until I conenct to console References: <5310346B.4000001@monkeybrains.net> <531035FD.4030704@freebsd.org> <53103E8E.80709@monkeybrains.net> In-Reply-To: <53103E8E.80709@monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 06:24:29 -0000 >>> I'm seeing something odd... I am running this command: >>> >>> bhyve -c 4 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s >>> 2,virtio-net,tap1 -s 3,virtio-blk,/dev/zvol/data/vm1 -l com1,/dev/nmdm1A >>> vm1 >>> but I the VM is being lazy... >> >> A number of people have reported this, and looking at it, I think >> it's a bug in bhyve. Fixed in r262884 later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 8 06:46:31 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31D91A53 for ; Sat, 8 Mar 2014 06:46:31 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 18733132 for ; Sat, 8 Mar 2014 06:46:30 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s286kUje012032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 7 Mar 2014 22:46:30 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <531ABCC5.30801@monkeybrains.net> Date: Fri, 07 Mar 2014 22:46:29 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: [HOW-TO] CentOS on bhyve Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 06:46:31 -0000 A customer wanted CentOS, so I tried doing it like I did my Ubuntu install --- no luck with the grub loading up automatically. Here is basically what I did ... it works, but I get all these odd characters in my terminal (if I hit tab, I can guess at the menu items and successfully make it through the installation process). I need to clean up this how-to, but here it is... START INSTALL ----------------------------------------- # cd /data/images/ # truncate -s 200G vm2.img # cat vm2-device.map (hd0) /data/images/vm2.img (cd0) /data/distributions/CentOS-6.5-x86_64-minimal.iso # bhyvectl --vm=vm2 --destroy # grub-bhyve -r cd0 -m /data/images/vm2-device.map -M 2048 vm2 grub> linux /isolinux/vmlinuz grub> initrd /isolinux/initrd.img grub> boot # bhyve -c 2 -m 2048M -H -P -A \ -l com1,stdio \ -s 0:0,hostbridge \ -s 1:0,lpc -s 2:0,virtio-net,tap1 \ -s 3,ahci-cd,/data/distributions/CentOS-6.5-x86_64-minimal.iso \ -s 4,virtio-blk,/data/images/vm2.img vm2 END INSTALL ----------------------------------------- Is there something like this that will clean up the output from the CentOS install program? grub> GRUB_TERMINAL=console The boot part is messy (how to I load up the grub.conf from inside the install image? START BOOT ----------------------------------------- # grub-bhyve -r hd0,msdos1 -m vm2.map -M 2048 vm2 grub> cat /grub/ Possible files are: grub.conf splash.xpm.gz menu.lst device.map stage1 stage2 e2fs_stage1_5 fat_stage1_5 ffs_stage1_5 iso9660_stage1_5 jfs_stage1_5 minix_stage1_5 reiserfs_stage1_5 ufs2_stage1_5 vstafs_stage1_5 xfs_stage1_5 grub> cat /grub/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/mapper/VolGroup-lv_root # initrd /initrd-[generic-]version.img #boot=/dev/vda default=0 timeout=5 serial --unit=0 --speed=9600 terminal --timeout=5 serial console title CentOS (2.6.32-431.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 console=ttyS0 crashkernel=auto rd_LV M_LV=VolGroup/lv_root rd_NO_DM initrd /initramfs-2.6.32-431.el6.x86_64.img grub> linux /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_ NO_MD KEYTABLE=us rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 console=ttyS0 crashkernel=auto rd_LVM_L V=VolGroup/lv_root rd_NO_DM grub> OM error: can't find command `OM'. grub> linux /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_ NO_MD KEYTABLE=us rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 console=ttyS0 crashkernel=auto rd_LVM_L V=VolGroup/lv_root rd_NO_DM grub> initrd /initramfs-2.6.32-431.el6.x86_64.img grub> boot # bhyve -c 2 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap1 -s 3,ahci-cd,/data/distributions/CentOS-6.5-x86_64-minimal.iso -s 4,virtio-blk,/data/images/vm2.img -l com1,stdio vm2 [snip] CentOS release 6.5 (Final) Kernel 2.6.32-431.el6.x86_64 on an x86_64 localhost.localdomain login: root Password: [root@localhost ~]# END BOOT ----------------------------------------- Also, this is making me wonder... am I doing this wrong? I am using install CD images, are other people using pre-made CentOS images? I have noticed grub-bhyve requires a space at the end of the line or I get 'OM' on STDOUT and the line fails. Above, I hit the up-arrow then a space ( Rudy From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 8 07:31:49 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6D10203 for ; Sat, 8 Mar 2014 07:31:49 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8D4D866D for ; Sat, 8 Mar 2014 07:31:49 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s287VmHE014531 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 7 Mar 2014 23:31:49 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <531AC763.300@monkeybrains.net> Date: Fri, 07 Mar 2014 23:31:47 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: [HOW-TO] CentOS on bhyve References: <531ABCC5.30801@monkeybrains.net> In-Reply-To: <531ABCC5.30801@monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 07:31:49 -0000 Peter, anyway to get grub-bhyve to automatically load /grub/grub.conf from a CentOS install? A more concise boot how-to involves making a .grub file based on the contents of the grub.conf in your .img file. The first email shows me wading through the grub prompt trying to figure out what to do. I'm not sure if the SYSFONT is pertinent. :) # cat vm2.grub linux /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 console=ttyS0 crashkernel=auto rd_LVM_LV=VolGroup/lv_root rd_NO_DM initrd /initramfs-2.6.32-431.el6.x86_64.img boot # grub-bhyve -r hd0,msdos1 -m vm2.map -M 2048 vm2 < vm2.grub # bhyve -c 2 -m 2048M -H -P -A \ -l com1,stdio \ -s 0:0,hostbridge \ -s 1:0,lpc -s 2:0,virtio-net,tap2 \ -s 3,ahci-cd,/data/distributions/CentOS-6.5-x86_64-minimal.iso \ -s 4,virtio-blk,/data/images/vm2.img vm2 Rudy From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 8 07:40:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3122927E for ; Sat, 8 Mar 2014 07:40:05 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 16BCC692 for ; Sat, 8 Mar 2014 07:40:04 +0000 (UTC) Received: from Birdhouse-o-My.local (208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s287e4p3014880 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 7 Mar 2014 23:40:04 -0800 (PST) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 208-90-212-98.PUBLIC.monkeybrains.net [208.90.212.98] claimed to be Birdhouse-o-My.local Message-ID: <531AC953.4030300@monkeybrains.net> Date: Fri, 07 Mar 2014 23:40:03 -0800 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: [HOW-TO] CentOS on bhyve References: <531ABCC5.30801@monkeybrains.net> <531AC763.300@monkeybrains.net> In-Reply-To: <531AC763.300@monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 07:40:05 -0000 On 3/7/14, 11:31 PM, Rudy (bulk) wrote: > > Peter, anyway to get grub-bhyve to automatically load /grub/grub.conf > from a CentOS install? This didn't do it... I just read the --help and found --directory, but that doesn't seem to work: grub-bhyve --directory=/grub -r hd0,msdos1 -m vm2.map -M 2048 vm2 Rudy From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 8 07:48:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9E354AF for ; Sat, 8 Mar 2014 07:48:19 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9806877F for ; Sat, 8 Mar 2014 07:48:19 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id z60so14446658qgd.8 for ; Fri, 07 Mar 2014 23:48:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CzR3pfQyuLB4VYNTkPZLC0/0JS0VX/ab0VnV6KS5oIY=; b=TFi7c583Vy6Nsr4yqoeUa3xOEdLo0hunnyd4yIwSuYEZJyPQwjVjNezuMRcOFH/EM4 HzDQczJ4J4RUNg7DA6Qubvm3ZbT1C43vD7lC0pg4AgcfduqjqcB6k1c+lkDKbm9xnv1d A/fhjaQZ0RLS3P9Ig/mELndjwFn2QycXBrqT2p7Cbo+f/Oa4cnTw1Ahz0G8fzVHOpH6F WAP6R8axLO8Sk/WMNHCNJ1DykaAxvBPJpOXZfiCDLZw4i97nsnBnS8j6Q2WvioKNCyDy G4/kH9e06f3JbOGx9SUulZBDJjyE/23h7AbVbyGU8edHjZy78iwH9Oda4ZuEBkN/J0Dg pxXw== MIME-Version: 1.0 X-Received: by 10.224.11.196 with SMTP id u4mr27468652qau.4.1394264898142; Fri, 07 Mar 2014 23:48:18 -0800 (PST) Received: by 10.96.70.74 with HTTP; Fri, 7 Mar 2014 23:48:18 -0800 (PST) Received: by 10.96.70.74 with HTTP; Fri, 7 Mar 2014 23:48:18 -0800 (PST) In-Reply-To: <531AC953.4030300@monkeybrains.net> References: <531ABCC5.30801@monkeybrains.net> <531AC763.300@monkeybrains.net> <531AC953.4030300@monkeybrains.net> Date: Sat, 8 Mar 2014 02:48:18 -0500 Message-ID: Subject: Re: [HOW-TO] CentOS on bhyve From: Matt Donovan To: "Rudy (bulk)" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 07:48:20 -0000 There is no way to auto pick up grub for CentOS due to it using grub1 and not grub2. I'll look at my configuration to see how I have it setup but have to load the information manually. On Mar 8, 2014 1:40 AM, "Rudy (bulk)" wrote: > > On 3/7/14, 11:31 PM, Rudy (bulk) wrote: >> >> >> Peter, anyway to get grub-bhyve to automatically load /grub/grub.conf from a CentOS install? > > > This didn't do it... I just read the --help and found --directory, but that doesn't seem to work: > > grub-bhyve --directory=/grub -r hd0,msdos1 -m vm2.map -M 2048 vm2 > > > > Rudy > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " freebsd-virtualization-unsubscribe@freebsd.org" As CentOS uses grub1 it can not be auto picked up. Have to manually enter the grub information. From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 8 08:06:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32B846C6 for ; Sat, 8 Mar 2014 08:06:40 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id E94D18BD for ; Sat, 8 Mar 2014 08:06:39 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 16046123BA; Sat, 8 Mar 2014 18:06:32 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BSJ05127 (AUTH peterg@ptree32.com.au); Sat, 8 Mar 2014 18:06:30 +1000 Message-ID: <531ACF85.8040305@freebsd.org> Date: Sat, 08 Mar 2014 00:06:29 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "Rudy (bulk)" Subject: Re: [HOW-TO] CentOS on bhyve References: <531ABCC5.30801@monkeybrains.net> <531AC763.300@monkeybrains.net> In-Reply-To: <531AC763.300@monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 08:06:40 -0000 Hi Rudy, > Peter, anyway to get grub-bhyve to automatically load /grub/grub.conf > from a CentOS install? > > A more concise boot how-to involves making a .grub file based on the > contents of the grub.conf in your .img file. The first email shows me > wading through the grub prompt trying to figure out what to do. I'm not > sure if the SYSFONT is pertinent. :) CentOS is grub v1 which isn't quite compatible with grub2 (e.g. linux -> kernel). grub-bhyve can pick up a config file from the host system: use "-r host" to force that, and the -d option to change the default path for grub.conf. No need to redirect input. Then, in grub.conf on the host, put in the absolute path with the grub commands e.g. kernel (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=VolGroup/lv_root rd_NO_DM initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img boot (the console=ttyS0 isn't needed: grub-bhyve auto-inserts that). later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 9 02:49:43 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DFE75AF for ; Sun, 9 Mar 2014 02:49:43 +0000 (UTC) Received: from hub.org (hub.org [200.46.208.146]) by mx1.freebsd.org (Postfix) with ESMTP id 254B9788 for ; Sun, 9 Mar 2014 02:49:42 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.188]) by hub.org (Postfix) with ESMTP id 9537C15237F2 for ; Sat, 8 Mar 2014 22:42:04 -0400 (AST) Received: from hub.org ([200.46.208.146]) by maia.hub.org (mx1.hub.org [200.46.151.188]) (amavisd-maia, port 10024) with ESMTP id 31796-10 for ; Sun, 9 Mar 2014 02:42:04 +0000 (UTC) Received: from [192.168.1.5] (S01067cb21b2ff4ca.gv.shawcable.net [24.108.26.71]) by hub.org (Postfix) with ESMTPA id C8EF115237EB for ; Sat, 8 Mar 2014 22:42:03 -0400 (AST) From: Marc Fournier Content-Type: multipart/signed; boundary="Apple-Mail=_9DBE4F44-EA98-4FCE-B649-B7EB58A0191C"; protocol="application/pgp-signature"; micalg=pgp-sha1 Subject: bhyve + OpenStack Message-Id: <373466E7-F8CD-4BB3-B5FD-DCE568CA97B6@hub.org> Date: Sat, 8 Mar 2014 18:42:05 -0800 To: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 02:49:43 -0000 --Apple-Mail=_9DBE4F44-EA98-4FCE-B649-B7EB58A0191C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Has anyone started lookign into this? FreeBSD + bhyve would, IMHO, make = a perfect base for this, since we=92re not dealing wioth third party = virtualization, bu tsomething built direclty into the Host OS =85=20 --Apple-Mail=_9DBE4F44-EA98-4FCE-B649-B7EB58A0191C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlMb1P0ACgkQ4QvfyHIvDvMiWQCcCB2kjWfPw2Kjlg0Db4oWtLg0 i98An1COBsO77PTJ3CNQOHboEeWU2COe =jDLN -----END PGP SIGNATURE----- --Apple-Mail=_9DBE4F44-EA98-4FCE-B649-B7EB58A0191C-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 9 04:36:46 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B22155E6 for ; Sun, 9 Mar 2014 04:36:46 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 340F0FF7 for ; Sun, 9 Mar 2014 04:36:46 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id s7so3885159lbd.9 for ; Sat, 08 Mar 2014 20:36:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=G4G5a17kTEeMgRSjDs5x2Pr/Q3ImnEmwcoYQVOyQ+kg=; b=YALJ+YzJdotxr7/mREKf1jE0+68QyQdBe75DkB1WD+3/DKYnVmCP0TYprQS6slV1gE Qx/WhMusiua/jeHBF3O37Cyw2jfTeBR/sKXseA+3+D+GE9OJfi8S/NCOgm7pm/JViq9X qkm81kYH4iJjhgt6vKicnWP1W4J1sjBGJkbrefgGbo8Gju/AWvJVrJiGIS3jhIOfh0H0 D8QrxF+WSs+RTsHq01pYO3i6EXvMXC5Cl8tQ6NA8KlRcC64+B+MBlKa7PzbmVG7XULIj 0XxYTjwtRm7dIBaMjlMkdJmjDlYYlgu456RVToG4jz6HberSNVeV5lgIzusToPYMP4Nq HeEQ== MIME-Version: 1.0 X-Received: by 10.112.254.163 with SMTP id aj3mr17071313lbd.20.1394339802716; Sat, 08 Mar 2014 20:36:42 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sat, 8 Mar 2014 20:36:42 -0800 (PST) In-Reply-To: <373466E7-F8CD-4BB3-B5FD-DCE568CA97B6@hub.org> References: <373466E7-F8CD-4BB3-B5FD-DCE568CA97B6@hub.org> Date: Sat, 8 Mar 2014 20:36:42 -0800 X-Google-Sender-Auth: JZ5n07_VCWzCB30Y25I4OuSjugE Message-ID: Subject: Re: bhyve + OpenStack From: Craig Rodrigues To: Marc Fournier Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 04:36:46 -0000 On Sat, Mar 8, 2014 at 6:42 PM, Marc Fournier wrote: > > Has anyone started lookign into this? FreeBSD + bhyve would, Semihalf is doing work Openstack in FreeBSD. See: http://www.freebsd.org/news/status/report-2013-10-2013-12.html#FreeBSD-Host-Support-for-OpenStack-and-OpenContrail Roman Bogorodskiy is working on libvirt and FreeBSD. However, his initial focus has been on QEMU, not bhyve See: http://empt1e.blogspot.hu/2013/11/running-qemu-vms-on-freebsd.html Yes, a lot of people would like to see Openstack + FreeBSD + BHyve. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 9 04:52:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D699E84C; Sun, 9 Mar 2014 04:52:16 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A6BDE1A7; Sun, 9 Mar 2014 04:52:16 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id y13so5651854pdi.5 for ; Sat, 08 Mar 2014 20:52:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wtghYot8QEjJxQ1GSmxB+PdtqfKwlXJyNxk9zb39xtQ=; b=r2yBoHTOdrdePRswSu6/23DlSr9aCVWM70Ntezl1G31y9Ofh+1elUMYwS+AxZuRwxz koxyo/tDblGxdBO75yzcwGg+1rvUmbL0KEjARz7SOCwlNSsvTNXHrusyiUkQ7Z4Takka a+Pn7MoGRKQ6OZCqTNrbVHTY/Pp26jDaXoNZd9EeWdO2tu8IUidlSLFcJKZj21DOuINq F4giE0i/btboC4gHP892ZOZDmromo1osUlnX+JLqJpmm0l8Nz55mbpILJEdEdO2jtwPS qtQA46LYyKSBN6A38/UHTe+uOz8dcRNO4V56jDiHzJMTw5IbfMfSds1QccCTUkeKaVeN B4qg== MIME-Version: 1.0 X-Received: by 10.68.172.37 with SMTP id az5mr32068273pbc.139.1394340736325; Sat, 08 Mar 2014 20:52:16 -0800 (PST) Received: by 10.68.155.38 with HTTP; Sat, 8 Mar 2014 20:52:16 -0800 (PST) In-Reply-To: References: <373466E7-F8CD-4BB3-B5FD-DCE568CA97B6@hub.org> Date: Sat, 8 Mar 2014 23:52:16 -0500 Message-ID: Subject: Re: bhyve + OpenStack From: Aryeh Friedman To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 04:52:16 -0000 We have DevStack running under bhyve on PetiteCloud [ http://www.petitecloud.org ] and once we do a little bit more debugging of our API we will test a complete ground up install (with docs)... from my experience with both DevStack and a full OpenStack install (manually on a bare metal linux host {note}) works just fine Note: kvm runs fine under kvm and under bhyve but not on raw hardware nor can bhyve be nested.... but you can simulate this with host-->linux instance-->OpenStack On Sat, Mar 8, 2014 at 11:36 PM, Craig Rodrigues wrote: > On Sat, Mar 8, 2014 at 6:42 PM, Marc Fournier wrote: > > > > Has anyone started lookign into this? FreeBSD + bhyve would, > > Semihalf is doing work Openstack in FreeBSD. See: > > http://www.freebsd.org/news/status/report-2013-10-2013-12.html#FreeBSD-Host-Support-for-OpenStack-and-OpenContrail > > Roman Bogorodskiy is working on libvirt and FreeBSD. > However, his initial focus has been on QEMU, not bhyve > See: > http://empt1e.blogspot.hu/2013/11/running-qemu-vms-on-freebsd.html > > Yes, a lot of people would like to see Openstack + FreeBSD + BHyve. > > -- > Craig > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 9 11:18:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D37E3A8; Sun, 9 Mar 2014 11:18:27 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 98D17D22; Sun, 9 Mar 2014 11:18:26 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so4044764lab.27 for ; Sun, 09 Mar 2014 04:18:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ci81uo05lF0kWQGNPsF6KxPEREvkCC8FkzPUfw5gNjs=; b=jHq7w88+n0Ot5jbem7IiBu051v/gXAuJbRxUos5mGNb1wlnYjJl4yXyg9uocn5yFrF 0GHFLYnGGTTDTLSH1nAmMot/tSOjAiuFP+0nCxaEBKebICbHQ57L91KGcnjQBFopS0JM cDnwCVh9M5yo9j+PoCtk9RrZ0+GtY44VDvfz/1vkmGGhM2DPjfB75QkAKp4rBvO03KV+ Jag4mq9Wn/h2Mlx8Rib+xOPP/sppkx3wTdMjpFFhHMhBPojAioUrr3nXvV79ohsPqkiK nizuWwe3D4RjNRWglw6Owa5FQMGEjnukanil4LlF7Ohthl66MMPkdbXp6/ITwcV3lEGu aZGg== X-Received: by 10.152.42.196 with SMTP id q4mr11013248lal.14.1394363903842; Sun, 09 Mar 2014 04:18:23 -0700 (PDT) Received: from kloomba ([95.104.135.120]) by mx.google.com with ESMTPSA id q6sm24560420lal.3.2014.03.09.04.18.22 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Mar 2014 04:18:22 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 9 Mar 2014 15:15:09 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: bhyve + OpenStack Message-ID: <20140309111507.GA76858@kloomba> References: <373466E7-F8CD-4BB3-B5FD-DCE568CA97B6@hub.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="FCuugMFkClbJLl1L" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 11:18:27 -0000 --FCuugMFkClbJLl1L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Sat, Mar 8, 2014 at 6:42 PM, Marc Fournier wrote: > > > > Has anyone started lookign into this? FreeBSD + bhyve would, >=20 > Semihalf is doing work Openstack in FreeBSD. See: > http://www.freebsd.org/news/status/report-2013-10-2013-12.html#FreeBSD-Ho= st-Support-for-OpenStack-and-OpenContrail >=20 > Roman Bogorodskiy is working on libvirt and FreeBSD. > However, his initial focus has been on QEMU, not bhyve > See: > http://empt1e.blogspot.hu/2013/11/running-qemu-vms-on-freebsd.html >=20 > Yes, a lot of people would like to see Openstack + FreeBSD + BHyve. Hi, Actually, I'm working on the bhyve driver for libvirt as well. libvirt 1.2.2 released this March includes initial support for the bhyve driver. It's in a basic state so far though. OpenStack upstream prefers to see new hypervisor driver support through libvirt, so once libvirt bhyve driver would be suitable for that, I'm going to work in that direction. Roman Bogorodskiy --FCuugMFkClbJLl1L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTHE07AAoJEMltX/4IwiJqCa0H/A2q+49vgVx5N2x1TubYBPIp KzAncdaykgzzLlAAFw3tJC31h67x2ob6k3KSven4d/5As4wF0QucHPVthiGEU4bK xszvh2c3uCRp15OuZGxLs+UcPr5vAvBGhySWZ5b+INZieC2TpB0H7ykp/z1mv6b2 WZ8gfZz0J9ahCck949aeLSfGB+z85Gi313f2uTEHXsMP4GYb/BmcZVsVDTK+PvVB RCdzh3MMhiLDLheApHF/0Kev38OPzjH+F530pk2wgAPmS55Jhm2fjjGf+OP4hN0H K8sQLcl7V3xlKBRam01DDUL/BK8928y+OJdeBN/qIIe6waP5aefRSCdaAtR1qzs= =dOAT -----END PGP SIGNATURE----- --FCuugMFkClbJLl1L-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 10 11:06:55 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D33151DC for ; Mon, 10 Mar 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C04CD825 for ; Mon, 10 Mar 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2AB6tT4043404 for ; Mon, 10 Mar 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2AB6tDF043402 for freebsd-virtualization@FreeBSD.org; Mon, 10 Mar 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Mar 2014 11:06:55 GMT Message-Id: <201403101106.s2AB6tDF043402@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 14 00:26:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEE818E4 for ; Fri, 14 Mar 2014 00:26:28 +0000 (UTC) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 948DE2D5 for ; Fri, 14 Mar 2014 00:26:27 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id lf10so1842375pab.41 for ; Thu, 13 Mar 2014 17:26:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=t8678nwfWUynifWBxj8QZhZIFhPQ8REjDBWs1rUsVOw=; b=ae6euYF1dLZqfnZ04m45Ltt3RR0kePCfaYm1IsdM2Iew118j9GlqjGIgHhd7Fp0Ipt sPd7uCzBi+jVoZyG9mg339rUOGz2JITgRYMelAA2L3QtKMK0vZ6ATeUYpBtsCIEJVDUb KktgMGnbPkE49vTDAui9LfTACX510b8IupEXGVjpK0kJcgu+QswdNMYTq2K0J+aKp3P3 K7InIA/Eu/EQeY8BCbgyBDekW80gqwVT/SmWuaMv+DVLLTq9t7b7hvQUe6joEql/0dqa 7TsczGAcGnP+o7raCB8rpppgZ42D+Ni39NUVWMwYBDmittf1zm6bnVDzLWEgfHwHCTAY Ncow== X-Gm-Message-State: ALoCoQmr0yX0CqJ9c5u+B0w2vKynutQSqAoQdphh5/KvV9uEdsG36C50NP/Xi2JMOQklN33rAb6y X-Received: by 10.68.244.229 with SMTP id xj5mr5851468pbc.108.1394756787404; Thu, 13 Mar 2014 17:26:27 -0700 (PDT) Received: from Michaels-MacBook-Pro.local ([180.42.49.96]) by mx.google.com with ESMTPSA id qh2sm16749417pab.13.2014.03.13.17.26.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Mar 2014 17:26:26 -0700 (PDT) Message-ID: <53224CB1.4040808@callfortesting.org> Date: Fri, 14 Mar 2014 09:26:25 +0900 From: Michael Dexter User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: "freebsd-virtualization@freebsd.org" Subject: bhyvecon slides: Up-to-date bhyve information Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 00:26:28 -0000 Hello all, Peter and Takuya have provided good up-to-date bhyve information in their bhyvecon slides at: http://bhyvecon.org You may also find the other topics interesting. Thank you everyone who helped putting bhyvecon together! Michael Dexter bhyve volunteer From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 14 06:09:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A331BC7; Fri, 14 Mar 2014 06:09:14 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2FBE75F0; Fri, 14 Mar 2014 06:09:13 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so1403082lan.40 for ; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=EJA3OnlupPDI2EsH0T1+qIW/MoSVYJem5Vf5P/ZKc8c=; b=Xa3kYCutRRrLwLVz7h2TEUAIJu//YfDkIiEPz0tXQUM8fdRubcOq3iTPa3YneAMgO2 7Oi2okaB83cmnFjVioNT5vHGGWuRcvxa03Pq6A2CHiOsftQEaBDoNc2cEo+Hd8OVR0u/ W8d8NjGrvr+zEFFGJaKPWItdROiMEm+yWq+PM73vqZdCyZgaQ++M7PScZZKlWWeIRO47 E0+z6H6CWSTkUm64fxOQgSq02SUEtjMHQhrpNcTLuO9IUqRWwI/Z/7xvMMeajkuXJ3aZ 5bsodq8w0Ogi8Cu1K4q59kJ5cynm0DGRgnK/rsPhx2jHut43GHN3iXp+m/Fia5WEnKuk RLUg== MIME-Version: 1.0 X-Received: by 10.153.8.135 with SMTP id dk7mr4208838lad.18.1394777351184; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) In-Reply-To: References: Date: Thu, 13 Mar 2014 23:09:11 -0700 X-Google-Sender-Auth: XdssAH0JaveZ3LE3JX0VwvtJGw8 Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 06:09:14 -0000 On March 13, 2014, I gave a talk at the Bay Area FreeBSD User Group on the current state of Jenkins and BHyve work in the FreeBSD cluster. The slides are now online: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 15 17:29:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45E29121 for ; Sat, 15 Mar 2014 17:29:15 +0000 (UTC) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10817FC7 for ; Sat, 15 Mar 2014 17:29:15 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so4013167oag.13 for ; Sat, 15 Mar 2014 10:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=8ckLJxdNwk3redFdY5Fcf8pglvStiXydY3d7SubSaxQ=; b=FlYHgqWqCZWYBXx9Hw4CxzvEAMtzvtnUdq1ysQ9//tBTN0tHh4KXEPgk5+6drXIugT 4A2T67LREx2mV4V8PsVMhE/tgcACAkY2257ul2AkGQVhGoasxZE/fOFDvCIE2xr6iP8L 4lWbTN0t7cpfZedo+Mi5nfF1uCGnrdMRDsAMNnUajosq8pcq7626AjVOXv2s2pgW/aQL leGv+LC6x+ff8Hwx/9ucgLYwZIkyzn78+uVL55lbcKU0h+eYX2gtkXCxpJ5Z9OUzBoQZ dmHTctAzZw8zN2yBGiTcrILnKDsO+hmtY5C1C9HVQx9bsURJ0pvuC2C2FSOeIH1MyZ1w dCVg== MIME-Version: 1.0 X-Received: by 10.60.115.129 with SMTP id jo1mr12965588oeb.0.1394904554438; Sat, 15 Mar 2014 10:29:14 -0700 (PDT) Received: by 10.182.33.136 with HTTP; Sat, 15 Mar 2014 10:29:14 -0700 (PDT) Date: Sat, 15 Mar 2014 18:29:14 +0100 Message-ID: Subject: CfP 9th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '14) From: VHPC 14 To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 17:29:15 -0000 we apologize if you receive multiple copies of this CfP ================================================================= CALL FOR PAPERS 9th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '14) held in conjunction with Euro-Par 2014, August 25-29, Porto, Portugal ================================================================= Date: August 26, 2014 Workshop URL: http://vhpc.org Paper Submission Deadline: May 30, 2014 CALL FOR PAPERS Virtualization technologies constitute a key enabling factor for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to dynamically manage complex infrastructures in a seamless fashion for varying workloads and hosted applications, independently of the customers deploying software or users submitting highly dynamic and heterogeneous workloads. Thanks to virtualization, we have the ability to manage vast computing and networking resources dynamically and close to the marginal cost of providing the services, which is unprecedented in the history of scientific and commercial computing. Various virtualization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple under-utilized servers with heterogeneous software and operating systems (OSes), and its capability to live-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-level virtualization, with its capability to isolate multiple user-space environments and to allow for their co-existence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical NICs/HBAs to take traffic from multiple VMs; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology and IP addressing, provides the fundamental ground on top of which evolved network services can be realized with an unprecedented level of dynamicity and flexibility; the increasingly adopted paradigm of Software-Defined Networking (SDN) promises to extend this flexibility to the control and data planes of network paths. These technologies have to be inter-mixed and integrated in an intelligent way, to support workloads that are increasingly demanding in terms of absolute performance, responsiveness and interactivity, and have to respect well-specified Service- Level Agreements (SLAs), as needed for industrial-grade provided services. Indeed, among emerging and increasingly interesting application domains for virtualization, we can find big-data application workloads in cloud infrastructures, interactive and real-time multimedia services in the cloud, including real-time big-data streaming platforms such as used in real-time analytics supporting nowadays a plethora of application domains. Distributed cloud infrastructures promise to offer unprecedented responsiveness levels for hosted applications, but that is only possible if the underlying virtualization technologies can overcome most of the latency impairments typical of current virtualized infrastructures (e.g., far worse tail-latency). What is more, in data communications Network Function Virtualization (NFV) is becoming a key technology enabling a shift from supplying hardware-based network functions, to providing them in a software-based and elastic way. In conjunction with (public and private) cloud technologies, NFV may be used for constructing the foundation for cost-effective network functions that can easily and seamlessly adapt to demand, still keeping their major carrier-grade characteristics in terms of QoS and reliability. The Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, and lightning talks, limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. TOPICS Topics of interest include, but are not limited to: - Management, deployment and monitoring of virtualized environments - Language-process virtual machines - Performance monitoring for virtualized/cloud workloads - Virtual machine monitor platforms - Topology management and optimization for distributed virtualized applications - Paravirtualized I/O - Improving I/O and network virtualization including use of RDMA, Infiniband, PCIe - Improving performance in VM access to GPUs, GPU clusters, GP-GPUs - HPC storage virtualization - Virtualized systems for big-data and analytics workloads - Optimizations and enhancements to OS virtualization support - Improving OS-level virtualization and its integration within cloud management - Performance modelling for virtualized/cloud applications - Heterogeneous virtualized environments - Network virtualization - Software defined networking - Network function virtualization - Hypervisor and network virtualization QoS and SLAs - Cloudbursting - Evolved European grid architectures including such based on network virtualization - Workload characterization for VM-based environments - Optimized communication libraries/protocols in the cloud - System and process/bytecode VM convergence - Cloud frameworks and APIs - Checkpointing/migration of VM-based large compute jobs - Job scheduling/control/policy with VMs - Instrumentation interfaces and languages - VMM performance (auto-)tuning on various load types - Cloud reliability, fault-tolerance, and security - Research, industrial and educational use cases - Virtualization in cloud, cluster and grid environments - Cross-layer VM optimizations - Cloud HPC use cases including optimizations - Services in cloud HPC - Hypervisor extensions and tools for cluster and grid computing - Cluster provisioning in the cloud - Performance and cost modelling - Languages for describing highly-distributed compute jobs - VM cloud and cluster distribution algorithms, load balancing - Instrumentation interfaces and languages - Energy-aware virtualization Important Dates Rolling Paper registration May 30, 2014 - Full paper submission July 4, 2014 - Acceptance notification October 3, 2014 - Camera-ready version due August 26, 2014 - Workshop Date TPC CHAIR Michael Alexander (chair), TU Wien, Austria Anastassios Nanos (co-chair), NTUA, Greece Tommaso Cucinotta (co-chair), Bell Labs, Dublin, Ireland PROGRAM COMMITTEE Costas Bekas, IBM Jakob Blomer, CERN Roberto Canonico, University of Napoli Federico II, Italy Paolo Costa, MS Research Cambridge, England Jorge Ejarque Artigas, Barcelona Supercomputing Center, Spain William Gardner, University of Guelph, USA Balazs Gerofi, University of Tokyo, Japan Krishna Kant, Temple University, USA Romeo Kinzler, IBM Nectarios Koziris, National Technical University of Athens, Greece Giuseppe Lettieri, University of Pisa, Italy Jean-Marc Menaud, Ecole des Mines de Nantes, France Christine Morin, INRIA, France Dimitrios Nikolopoulos, Queen's University of Belfast, UK Herbert Poetzl, VServer, Austria Luigi Rizzo, University of Pisa, Italy Josh Simons, VMWare, USA Borja Sotomayor, University of Chicago, USA Vangelis Tasoulas, Simula Research Lab, Norway Yoshio Turner, HP Labs, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden Chao-Tung Yang, Tunghai University, Taiwan PAPER SUBMISSION-PUBLICATION Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, key words, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in the Springer LNCS series - the format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Format Guidelines: http://www.springer.de/comp/lncs/authors.html EasyChair Abstract Submission Link: https://www.easychair.org/conferences/?conf=europar2014ws GENERAL INFORMATION The workshop is one day in length and will be held in conjunction with Euro-Par 2014, 25-29 August, Porto, Portugal From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 15 18:27:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A2856E2 for ; Sat, 15 Mar 2014 18:27:15 +0000 (UTC) Received: from litio.roma.schema31.it (2-228-74-184.ip190.fastwebnet.it [2.228.74.184]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 867F27A8 for ; Sat, 15 Mar 2014 18:27:13 +0000 (UTC) Received: from litio.roma.schema31.it (localhost [127.0.0.1]) by litio.roma.schema31.it (8.14.5/8.14.5) with ESMTP id s2FIEH2c087367 for ; Sat, 15 Mar 2014 19:14:18 +0100 (CET) (envelope-from abrancatelli@schema31.it) Received: (from www@localhost) by litio.roma.schema31.it (8.14.5/8.14.5/Submit) id s2FIEHfC087366; Sat, 15 Mar 2014 19:14:17 +0100 (CET) (envelope-from abrancatelli@schema31.it) X-Authentication-Warning: litio.roma.schema31.it: www set sender to abrancatelli@schema31.it using -f To: freebsd-virtualization@freebsd.org Subject: Clock in bhyve X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Date: Sat, 15 Mar 2014 19:14:17 +0100 From: Andrea Brancatelli Organization: Schema31 s.r.l. Message-ID: <766b303ae006c121f8bebe077193e4fa@schema31.it> X-Sender: abrancatelli@schema31.it User-Agent: Roundcube Webmail/0.9.4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 18:27:15 -0000 Hello everybody. I have a stupid question for you :) Is the clock in bhyve virtual or just a hook the host? I mean, should I run ntpd inside the VMs as I would do, let's say, with VMWare, or is it enough to run it on the host? -- Andrea Brancatelli From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 15 18:34:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31797B40 for ; Sat, 15 Mar 2014 18:34:15 +0000 (UTC) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 08E5D872 for ; Sat, 15 Mar 2014 18:34:15 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id md12so3984679pbc.7 for ; Sat, 15 Mar 2014 11:34:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=jgrKY4n0N+bSWTjx3/TpYIp41IXKNt2x7+KzZEyqrEM=; b=eBEgtFY22jlX3HAanagKSU2TjXAYy1weesH51dfe5zAWzkrCdiUl2DrvMJHgXoKJGf +hZxwyc2LPIHWYqTGgtb6eelK7RLI5kElIJtB7WH9kP7e0P4AOHknybf7+ncBQPgPhVw cvaJ+x1Xorlo7Zllx+LtI68u+Bmg7HOlJ3qlFOHpMuhG4htVdaqI/tHBCGVihwL2nadJ oDsQoPhwOKy/09XiXu0sgmQnrsZXsYM2N+1R6jb2d3xKecl6Q80FL0pX0QkOid9N3v1u DwycZ0Yc07HXr7eNj0X9CpP87Sm2RB0XcqEpMv0dJqTzt8fbpuJ1F5icBEPd26Newt3T rClw== MIME-Version: 1.0 X-Received: by 10.68.13.42 with SMTP id e10mr16307716pbc.118.1394908454604; Sat, 15 Mar 2014 11:34:14 -0700 (PDT) Received: by 10.68.155.38 with HTTP; Sat, 15 Mar 2014 11:34:14 -0700 (PDT) In-Reply-To: <766b303ae006c121f8bebe077193e4fa@schema31.it> References: <766b303ae006c121f8bebe077193e4fa@schema31.it> Date: Sat, 15 Mar 2014 14:34:14 -0400 Message-ID: Subject: Re: Clock in bhyve From: Aryeh Friedman To: Andrea Brancatelli , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 18:34:15 -0000 Yes there is always a time drift on a guest/instance... in specialized cases it makes sense to modify the instance by requiring it run ntpd.... but if it is general purpose and for general "public" (whatever that means in your context) instances it is likely to turn people away to know that they are required to take a Red Pill. On Sat, Mar 15, 2014 at 2:14 PM, Andrea Brancatelli < abrancatelli@schema31.it> wrote: > > > Hello everybody. > > I have a stupid question for you :) > > Is the clock in bhyve virtual or just a hook the host? > > I mean, should I run ntpd inside the VMs as I would do, let's say, with > VMWare, or is it enough to run it on the host? > -- > > Andrea Brancatelli > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 15 23:35:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0ACB49AB for ; Sat, 15 Mar 2014 23:35:33 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BD9931AF for ; Sat, 15 Mar 2014 23:35:32 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so4021926qae.31 for ; Sat, 15 Mar 2014 16:35:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XFbk9gNpe6DTvKY3ETol342X7wURVw+sPvnmC8AmLps=; b=rDIdAvbSyHPr6nsHxJvh4Q/l+egKhcJShWuM6YVJo0JdCGT/3yyffWlknSm1yFSkbJ /ZjE8HAXDxQd1Eyl6ML6f6cjGWOPDHAyOe9ww/oK/bMkJAR2SxgkWxFUDJHDK2oE4/1e Cr0bxAvAbwqxXu8j1VIVFYrTxtUzeyrziv4RDebd5MzQ5AAVEZNkGfC3bCdsKX2bRftA 4fTSu3BBNH5ohisyy93qALqmOYBzquT4ijiVomWzPNMrOPqfA1Nar4H4FzqS+0A3I1Cd dpUB3uX1W5OzCImEZkB3olxqByopXl8LFqQFrjTw+EAvBT0bxJRyCasYQdYNkujOZoZB vkXg== MIME-Version: 1.0 X-Received: by 10.140.27.193 with SMTP id 59mr17995009qgx.18.1394926531898; Sat, 15 Mar 2014 16:35:31 -0700 (PDT) Received: by 10.140.87.74 with HTTP; Sat, 15 Mar 2014 16:35:31 -0700 (PDT) In-Reply-To: <766b303ae006c121f8bebe077193e4fa@schema31.it> References: <766b303ae006c121f8bebe077193e4fa@schema31.it> Date: Sat, 15 Mar 2014 16:35:31 -0700 Message-ID: Subject: Re: Clock in bhyve From: Neel Natu To: Andrea Brancatelli Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 23:35:33 -0000 Hi Andrea, On Sat, Mar 15, 2014 at 11:14 AM, Andrea Brancatelli wrote: > > > Hello everybody. > > I have a stupid question for you :) > > Is the clock in bhyve virtual or just a hook the host? > > I mean, should I run ntpd inside the VMs as I would do, let's say, with > VMWare, or is it enough to run it on the host? You should also run it inside the guest. best Neel > -- > > Andrea Brancatelli > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sun Mar 16 00:12:47 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 678C4292 for ; Sun, 16 Mar 2014 00:12:47 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E55C0648 for ; Sun, 16 Mar 2014 00:12:46 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id 10so2708783lbg.21 for ; Sat, 15 Mar 2014 17:12:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=0ohbFSKXMPLIYAArT1UTtM9u5RJ3aI7ionEmnG4ommE=; b=UGVCNk2miC79cDdVqUeGQQhDIBDAtbfQY0TxQRg5imNI1qkJfhVp81Mrd6Dh/FwQN4 KBmdxBUUql6DWFJrMunb9UZuQH+cC68l6Q0G331y8hN//7+P5KSDdxjcl9Ob0Fv3MySA 3M/TRFnJFkIo2Au9JIJjN3RHY9GAi6/tyOSy8Yj4C7WNfTxxgto1L4yVl3jwhrWbKDCx y/63MowSW+SPlGPyTREq+7XledGA1/kegq0jT4cv9PpDfPWLWx8tPQd7kplEgCHae0Tx crE8NSrkEi2bU+4jIjn9IJCnzeKbtEa8DRZPzrEEz/9RurZQYYwLZcTbDcSH8huKNKId HIzw== MIME-Version: 1.0 X-Received: by 10.152.37.137 with SMTP id y9mr11100536laj.8.1394928765010; Sat, 15 Mar 2014 17:12:45 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Sat, 15 Mar 2014 17:12:44 -0700 (PDT) In-Reply-To: <53174140.5040004@pix.net> References: <53174140.5040004@pix.net> Date: Sat, 15 Mar 2014 17:12:44 -0700 X-Google-Sender-Auth: _ThJzUnpEn8ofEO4a1ebWHvcuTw Message-ID: Subject: Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions From: Craig Rodrigues To: Kurt Lidl Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 00:12:47 -0000 On Wed, Mar 5, 2014 at 7:22 AM, Kurt Lidl wrote: > > > I ran into this same problem a couple of weeks ago when I first played > with the ZFS boot support in userboot.so. > > The following set of operations on the hypervisor can be used to add > bootable support to your zfs installation. > > (I don't know what the name of your zpool is on the emulated host. > If the name of that zpool is the same as the zpool on your hypervisor > host, you might have to go at this a different way.) I have taken > to naming the zpool for the any clients to be the same as the name > of the virtual machine, so I can easily mount it on the host running > the hypervisor and fiddle with it. > > For the purposes of this example, my virtual machine is called "vm0". > The zpool for that host lives in a zpool called "zdata" on my > hypervisor machine. In my case, my /boot was a symlink that > pointed to /bootdir/boot, so you might need to adjust the > following slightly. > > zfs import -f -R /mnt vm0 > > fsck_ufs /dev/zvol/zdata/vm0p2 > mount /dev/zvol/zdata/vm0p2 /mnt/bootdir > cd /mnt > chflags -h nosunlink boot > rm boot > cp -R bootdir/* . > umount /mnt/bootdir > cd / > zpool export vm0 > > Good luck. Hi, Thanks for breaking down the steps so nicely! I followed similar steps and got my VM to boot with the latest userboot. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 17 11:06:55 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD46CBC8 for ; Mon, 17 Mar 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A141E2AF for ; Mon, 17 Mar 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2HB6tmK011449 for ; Mon, 17 Mar 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2HB6tZp011447 for freebsd-virtualization@FreeBSD.org; Mon, 17 Mar 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Mar 2014 11:06:55 GMT Message-Id: <201403171106.s2HB6tZp011447@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 17 13:46:39 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 317CC884 for ; Mon, 17 Mar 2014 13:46:39 +0000 (UTC) Received: from smtp149.ord.emailsrvr.com (smtp149.ord.emailsrvr.com [173.203.6.149]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 045196A8 for ; Mon, 17 Mar 2014 13:46:38 +0000 (UTC) Received: from smtp27.relay.ord1a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp27.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 71DBC2A8696; Mon, 17 Mar 2014 09:39:49 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp27.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 5AC722A869F; Mon, 17 Mar 2014 09:39:49 -0400 (EDT) X-Virus-Scanned: OK Received: from smtp192.mex05.mlsrvr.com (unknown [184.106.31.85]) by smtp27.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTPS id A0BC42A86A0; Mon, 17 Mar 2014 09:39:48 -0400 (EDT) Received: from ORD2MBX05C.mex05.mlsrvr.com ([fe80::90e2:baff:fe30:7498]) by ORD2HUB21.mex05.mlsrvr.com ([fe80::be30:5bff:feee:18f4%15]) with mapi id 14.03.0169.001; Mon, 17 Mar 2014 08:39:48 -0500 From: Michael Berman To: Neel Natu , Andrea Brancatelli Subject: Re: Clock in bhyve Thread-Topic: Clock in bhyve Thread-Index: AQHPQHw0zJ3BvE84yUGizPIVRpSfPprjIVuAgAII3oA= Date: Mon, 17 Mar 2014 13:39:48 +0000 Message-ID: References: <766b303ae006c121f8bebe077193e4fa@schema31.it> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [71.198.217.229] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <21FA7C6BED13A14E94A23D1169944347@mex05.mlsrvr.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 13:46:39 -0000 In my humble experience, the clock inside a VM can jump badly enough that NTP complains and exits. You=B9ll need to configure NTP to be very forgiving. VMware does a lot of work in its tools/driver to avoid this. Michael On 3/15/14, 4:35 PM, "Neel Natu" wrote: >Hi Andrea, > >On Sat, Mar 15, 2014 at 11:14 AM, Andrea Brancatelli > wrote: >> >> >> Hello everybody. >> >> I have a stupid question for you :) >> >> Is the clock in bhyve virtual or just a hook the host? >> >> I mean, should I run ntpd inside the VMs as I would do, let's say, with >> VMWare, or is it enough to run it on the host? > >You should also run it inside the guest. > >best >Neel > >> -- >> >> Andrea Brancatelli >> >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >>"freebsd-virtualization-unsubscribe@freebsd.org" >_______________________________________________ >freebsd-virtualization@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >To unsubscribe, send any mail to >"freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 18 06:17:57 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9BD9ED46 for ; Tue, 18 Mar 2014 06:17:57 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 62B67E6E for ; Tue, 18 Mar 2014 06:17:57 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so7269904qcy.14 for ; Mon, 17 Mar 2014 23:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Oy+mbmZ3SGEONmM/MpZ9Uv1mbZTMYl8H/ob4YXzpy0U=; b=NK9FVvhshFZHwv0Pl48JnxEEuV4rv3BWP8QPhgcosDFBUOUPqYGopE00HeBQkbLPs7 hvGhsbim8ElKH7lBx/oshY5wVpkYX8LCPg3eAa3GptXMP8dD/0c288qkXN3U0nrvVkci BAOWvmI5ev9K9SpUoilinrXd0pfgVhhAM3OeoF+LeuJauTHQns114PW7XMjCY1DDepVf XM80YWKBBQjUZux/uaAizjIWAp97aTycCCDm2OEQrnghBTOmYHBK3e5FV2ID2OKnbzA8 tfkgGbjRBObsYT+Ln+/y4tdhAUzxkoOGJREl9zjszTNtJEkLUZrQwzWQHsTR6WAzuqvC me9A== MIME-Version: 1.0 X-Received: by 10.224.63.131 with SMTP id b3mr33882476qai.63.1395123475764; Mon, 17 Mar 2014 23:17:55 -0700 (PDT) Received: by 10.140.25.243 with HTTP; Mon, 17 Mar 2014 23:17:55 -0700 (PDT) Date: Tue, 18 Mar 2014 08:17:55 +0200 Message-ID: Subject: [GSOC] Instruction caching for bhyve From: Mihai Carabas To: virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 06:17:57 -0000 Hi, My name is Mihai Carabas and I'm a first year student at PhD in University POLITEHNICA from Bucharest, Romania. After some discussions with Neel Natu about choosing a bhyve project, I came with a GSOC proposal for the project "Instruction caching for bhyve" which is publicly available here [1]. Any feedback on the proposal would be great. Thanks, Mihai [1] http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/mihaicarabas/5698390809640960 From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 19 18:44:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D16D2FE; Wed, 19 Mar 2014 18:44:27 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 70009D99; Wed, 19 Mar 2014 18:44:26 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so6273096lbj.31 for ; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=bsV2TpV77A89Mt5Q5VKNPhZR+XFhoynSkyV5nfUiMO4=; b=NImjUr3/UVNL+ybBvvhgJOo4oykODbgaAmph6mof+Kbk6SESoZqzj3tW9SLX4jaOGW t7WtWQuz7bQ2RPyGAhDcEee9rjgXsmtbJ5CB9DU+LxJlpOkwOdz3LC/jYeKECWk7unA3 71IqDpzrQ/bJUna1zXHokFZHtYvMGmv+Xbyoshe5XU6jj/Iigw9ozqaeJ83qsE05E+Oc mYGBHOqT+cDWo2bYYXm+NhfvJEYDu3UQ9gZPc2h9PAZHs89cK/f7ozKFCOEypVSdmlOR zPxSMqibCkLa+8rZY7hXMh06bRfrSbmwOoTEdJNsMOW07y1oI+mYz7ClluKQbh+Ha39X 1qJg== MIME-Version: 1.0 X-Received: by 10.152.22.37 with SMTP id a5mr26923345laf.4.1395254663500; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Mar 2014 11:44:23 -0700 X-Google-Sender-Auth: u1Ssv9mrZ-n0HRptuHoSPYZQGZc Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 18:44:27 -0000 On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ Thanks to Annie Zhang and the rest of the iXsystems marketing department who recorded the video and did the editing, the video for this presentation is now online: Go to: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups and click on the link for the video. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 19 22:00:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8C1E8DF for ; Wed, 19 Mar 2014 22:00:05 +0000 (UTC) Received: from mb2i1.ns.pitt.edu (mb2i1mx.ns.pitt.edu [136.142.11.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B760B2B0 for ; Wed, 19 Mar 2014 22:00:05 +0000 (UTC) Received: from [10.0.0.4] (c-24-2-126-242.hsd1.pa.comcast.net [24.2.126.242]) by pitt.edu (PMDF V6.5-x7 #31901) with ESMTPSA id <0N2P00D2NDD2H9@mb2i1.ns.pitt.edu> for freebsd-virtualization@freebsd.org; Wed, 19 Mar 2014 17:36:39 -0400 (EDT) Date: Wed, 19 Mar 2014 17:36:38 -0400 From: John Matty Subject: Google Summer of Code 2014 question To: freebsd-virtualization@freebsd.org Message-id: <532A0DE6.2020801@pitt.edu> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7bit User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 22:00:05 -0000 Hello everybody, My name is John, and I was interested in doing Google Summer of Code for FreeBSD this Summer. I am currently interested in writing a virtual machine execution language. A rough draft of my proposal can be found on the website for GSoC 2014 (I am not exactly sure how to get to it from the outside though, sorry), however the brief description of my project is to specify and implement a domain specific language to start a virtual machine, execute given commands, and store the results of the given command. Then, using a text comparison tool (maybe something similar to `diff'), compare the obtained output to the expected output. I was currently thinking that this would mainly be used for doing repeated testing of software on FreeBSD running in a virtual machine, however I was interested to see if anyone had any ideas for other uses or applications for this language so that I could focus on making it as useful as possible for what people would actually be using it for, or if anyone had any other comments or suggestions? I was also wondering if anyone would be interested in mentoring me for my project this summer? Finally, as I am new to this mailing list, if this is not the correct place to ask this question or anything else like that please let me know so I can move my question to the correct place. Thank you! --John From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 16:12:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3D946A7 for ; Thu, 20 Mar 2014 16:12:25 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 868BEA2A for ; Thu, 20 Mar 2014 16:12:25 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so1270819qcr.11 for ; Thu, 20 Mar 2014 09:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lJc+Y7UcUqNGEvgC6Hu0yBnFUE32XQtLCLQ827WbDSg=; b=P2BWbdY7XjTtlqxc4Zxj4PFqbmq6T1038PYTed6CL6VsAOeEje9HuBpi8pMt/H2umr TYme5fQgvq46/JjMUIg/gi/DvcyNn7jR7W8cQRqM629f4mWONq3cedfBCK/DJrkU3kRw jexhtdA0r8+e8CfyNQQDKDt0AeE92+HIFR78KK5/Xfn/trV62KdRejSnysnvTayWaOU3 w2hw+pHWjBTp+3QcYA9v/nQgirDXorHskaeyDudhCyhJSuf/tqwaqBwTO9vF1To1admO +ioPHRcYqyHkn6V40/RYRVsoCSHvKHDkB0ptWby0nPCHiWry32PNHujq19HlWNxkBe01 WM6w== MIME-Version: 1.0 X-Received: by 10.229.116.193 with SMTP id n1mr51445514qcq.10.1395331944725; Thu, 20 Mar 2014 09:12:24 -0700 (PDT) Received: by 10.140.87.74 with HTTP; Thu, 20 Mar 2014 09:12:24 -0700 (PDT) In-Reply-To: <532A0DE6.2020801@pitt.edu> References: <532A0DE6.2020801@pitt.edu> Date: Thu, 20 Mar 2014 09:12:24 -0700 Message-ID: Subject: Re: Google Summer of Code 2014 question From: Neel Natu To: John Matty Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 16:12:25 -0000 Hi John, On Wed, Mar 19, 2014 at 2:36 PM, John Matty wrote: > Hello everybody, > > My name is John, and I was interested in doing Google Summer of Code for > FreeBSD this Summer. I am currently interested in writing a virtual machine > execution language. A rough draft of my proposal can be found on the > website for GSoC 2014 (I am not exactly sure how to get to it from the > outside though, sorry), however the brief description of my project is to > specify and implement a domain specific language to start a virtual machine, > execute given commands, and store the results of the given command. Then, > using a text comparison tool (maybe something similar to `diff'), compare > the obtained output to the expected output. > > I was currently thinking that this would mainly be used for doing repeated > testing of software on FreeBSD running in a virtual machine, however I was > interested to see if anyone had any ideas for other uses or applications for > this language so that I could focus on making it as useful as possible for > what people would actually be using it for, or if anyone had any other > comments or suggestions? > > I was also wondering if anyone would be interested in mentoring me for my > project this summer? > > Finally, as I am new to this mailing list, if this is not the correct place > to ask this question or anything else like that please let me know so I can > move my question to the correct place. > > Thank you! > I could certainly make use of this in bhyve development. For e.g., whenever we make changes to bhyve there is a large test matrix that needs to be addressed: - various guest OSes: FreeBSD (various releases), Linux (various flavors), OpenBSD etc. - 32-bit or 64-bit virtual machine - number of vcpus and guest memory sizes - multiple device emulations (virtio, ahci, passthru) - timer modes (PIT, LAPIC, HPET, ACPI) - interrupt delivery (8259, legacy, MSI, MSI-X) - shutdown types (reboot, halt, acpi power off) A tool like the one you are proposing would be very useful to catch regressions. best Neel > --John > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 18:39:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3806A746; Thu, 20 Mar 2014 18:39:15 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10A3EDF8; Thu, 20 Mar 2014 18:39:15 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E0463B9B7; Thu, 20 Mar 2014 14:39:13 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: Google Summer of Code 2014 question Date: Thu, 20 Mar 2014 11:24:07 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <532A0DE6.2020801@pitt.edu> In-Reply-To: <532A0DE6.2020801@pitt.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403201124.07472.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 20 Mar 2014 14:39:14 -0400 (EDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 18:39:15 -0000 On Wednesday, March 19, 2014 5:36:38 pm John Matty wrote: > Hello everybody, > > My name is John, and I was interested in doing Google Summer of Code for > FreeBSD this Summer. I am currently interested in writing a virtual > machine execution language. A rough draft of my proposal can be found > on the website for GSoC 2014 (I am not exactly sure how to get to it > from the outside though, sorry), however the brief description of my > project is to specify and implement a domain specific language to start > a virtual machine, execute given commands, and store the results of the > given command. Then, using a text comparison tool (maybe something > similar to `diff'), compare the obtained output to the expected output. > > I was currently thinking that this would mainly be used for doing > repeated testing of software on FreeBSD running in a virtual machine, > however I was interested to see if anyone had any ideas for other uses > or applications for this language so that I could focus on making it as > useful as possible for what people would actually be using it for, or if > anyone had any other comments or suggestions? > > I was also wondering if anyone would be interested in mentoring me for > my project this summer? > > Finally, as I am new to this mailing list, if this is not the correct > place to ask this question or anything else like that please let me know > so I can move my question to the correct place. This isn't a bad place to ask this question. I think a person you might want to talk to is Craig Rodrigues (cc'd) who has recently setup Jenkins with the goal of doing automated testing using virtual machines. I'm not sure exactly what design he is considering, but he is the person to ask. -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 20:54:55 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9033E739 for ; Thu, 20 Mar 2014 20:54:55 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1A8A1179 for ; Thu, 20 Mar 2014 20:54:54 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so1037283lan.2 for ; Thu, 20 Mar 2014 13:54:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jXt+NbVSZMHd4CKys6PvAhN21IcgR5Iu+d/9zVFbOTU=; b=DlIsgoj4Fg8uAZgLWpB1KOIUqZSgnot2CxYkRATQkpZ2NLzs6LEndMCGW2pU0ytSs4 FD0v5bqrHJ1SRAAxdkjvyocP55YAyIsYAXkdx/8OKOhbo1dISBs0GvakeEtVqEUnS0wc ktYjeB3IDneXzGGUIam+8U5Di5dycc31GmVAjHjqm6TtsZK3dsBXvxE7uSNVtbPV2Mi5 fhg3D4HAZYp2nWZP25xz6VhM7rRQtFlrDNG8bEOHtowS0JhebEJQNq+GN4Pe1sF9On2n XUN0uGmA1F9NQV4rX/QJIVU59WwqYUHVmrhUmom6sIKDG7xu8l28MIrO9TyjfMlk4dSH X8dg== MIME-Version: 1.0 X-Received: by 10.112.154.202 with SMTP id vq10mr29697473lbb.3.1395348893112; Thu, 20 Mar 2014 13:54:53 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 20 Mar 2014 13:54:53 -0700 (PDT) In-Reply-To: <201403201124.07472.jhb@freebsd.org> References: <532A0DE6.2020801@pitt.edu> <201403201124.07472.jhb@freebsd.org> Date: Thu, 20 Mar 2014 13:54:53 -0700 X-Google-Sender-Auth: EOo29ovou4xnvuO-95dOmP1ENZw Message-ID: Subject: Re: Google Summer of Code 2014 question From: Craig Rodrigues To: John Matty Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 20:54:55 -0000 On Thu, Mar 20, 2014 at 8:24 AM, John Baldwin wrote: > > This isn't a bad place to ask this question. I think a person you might > want to talk to is Craig Rodrigues (cc'd) who has recently setup Jenkins > with the goal of doing automated testing using virtual machines. I'm not > sure exactly what design he is considering, but he is the person to ask. Last year I signed up to be a GSoC mentor, but this year I unfortunately do not have time, so haven't done so. Unfortunately, since I did not sign up as a mentor, I can't view your proposal on the google-melange web site. If you look at the following link, you can see the March 13 presentation which describes some work I did setting up Jenkins and BHyve for continuous integration and testing on FreeBSD: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups If you look at this link, item 22: https://wiki.freebsd.org/Jenkins#Jenkins_for_FreeBSD_status you will see that we are trying to work at integrating Jenkins with the Kyua framework, so that Jenkins can run Kyua, and report the results in the Jenkins web UI. Jenkins is used by hundreds of companies, not just in the FreeBSD world. Kyua is gaining popularity in the FreeBSD community, and is already used by major companies (EMC/Isilon and Spectralogic are the ones that I know of). There are already many systems out there for doing automation, starting/stopping VM's, and running tests. I'm focusing on integrating as many existing technologies and getting them to run under FreeBSD to put together a more modern continuous integration and testing pipeline for the FreeBSD project. I cannot comment on the specifics your proposal without reading it. However, I don't see a lot of value in coming up with a new Domain Specific Language for starting/stopping VM's and sending commands to VM's. There are already many people doing this already with other frameworks and automation API's. For example, libvirt ( http://libvirt.org ) is one popular API specifically for controlling VM's. In terms of frameworks that can send remote commands to VM's, there are already plenty of automation systems to choose from: Puppet, Chef, SaltStack, CFEngine are some examples. Creating a new Domain Specific Language for VM control sounds like a more academic/research exercise, so if you can get someone to mentor you for such a project, maybe you can target the results of your work for publication, if that is of interest to you. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 21:40:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88378CD4; Thu, 20 Mar 2014 21:40:38 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C72F48D8; Thu, 20 Mar 2014 21:40:37 +0000 (UTC) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) by freebsd.czest.pl (8.14.5/8.14.5) with ESMTP id s2KLcmkd010431; Thu, 20 Mar 2014 21:38:49 GMT (envelope-from wkoszek@freebsd.czest.pl) Received: (from wkoszek@localhost) by freebsd.czest.pl (8.14.5/8.14.5/Submit) id s2KLcmD7010430; Thu, 20 Mar 2014 21:38:48 GMT (envelope-from wkoszek) Date: Thu, 20 Mar 2014 21:38:48 +0000 From: "Wojciech A. Koszek" To: Craig Rodrigues Subject: Re: Google Summer of Code 2014 question Message-ID: <20140320213848.GU37327@FreeBSD.org> References: <532A0DE6.2020801@pitt.edu> <201403201124.07472.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-0.4 required=5.0 tests=RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on freebsd.czest.pl X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (freebsd.czest.pl [212.87.224.105]); Thu, 20 Mar 2014 21:38:55 +0000 (UTC) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:40:38 -0000 On Thu, Mar 20, 2014 at 01:54:53PM -0700, Craig Rodrigues wrote: > On Thu, Mar 20, 2014 at 8:24 AM, John Baldwin wrote: > > > > This isn't a bad place to ask this question. I think a person you might > > want to talk to is Craig Rodrigues (cc'd) who has recently setup Jenkins > > with the goal of doing automated testing using virtual machines. I'm not > > sure exactly what design he is considering, but he is the person to ask. > > Last year I signed up to be a GSoC mentor, but this year I unfortunately > do not have time, so haven't done so. Unfortunately, since I did not > sign up as a mentor, I can't view your proposal on the google-melange web site. > > If you look at the following link, you can see the March 13 > presentation which describes > some work I did setting up Jenkins and BHyve for continuous > integration and testing on FreeBSD: > > https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups > > If you look at this link, item 22: > > https://wiki.freebsd.org/Jenkins#Jenkins_for_FreeBSD_status > > you will see that we are trying to work at integrating Jenkins with the Kyua > framework, so that Jenkins can run Kyua, and report the results in the > Jenkins web UI. > > Jenkins is used by hundreds of companies, not just in the FreeBSD world. > Kyua is gaining popularity in the FreeBSD community, and is already used by > major companies (EMC/Isilon and Spectralogic are the ones that I know of). > > There are already many systems out there for doing automation, > starting/stopping VM's, > and running tests. I'm focusing on integrating as many existing > technologies and > getting them to run under FreeBSD to put together a more modern > continuous integration and testing > pipeline for the FreeBSD project. > > I cannot comment on the specifics your proposal without reading it. > However, I don't see a lot of value in coming up with a new Domain > Specific Language > for starting/stopping VM's and sending commands to VM's. There are > already many people > doing this already with other frameworks and automation API's. For > example, libvirt ( http://libvirt.org ) > is one popular API specifically for controlling VM's. > > In terms of frameworks that can send remote commands to VM's, there > are already plenty > of automation systems to choose from: Puppet, Chef, SaltStack, > CFEngine are some examples. > > Creating a new Domain Specific Language for VM control sounds like a > more academic/research exercise, > so if you can get someone to mentor you for such a project, maybe you > can target the results > of your work for publication, if that is of interest to you. John, Craig, When I initially proposed the idea: https://wiki.freebsd.org/SummerOfCode2014#TEST-o-steron_for_FreeBSD_.28node.js.2FLua.29 it was meant to be mainly a regression testing tool for FreeBSD. Something that would let me build and run continuous builds for FreeBSD ARM (ZEDBOARD kernel), and then make a SD image out of it, which I can boot in QEMU/real hardware. It would be nice if we payed attention to BHyVE in this project, but not limit ourselves to it. E.g.: ARM testing was my original drive towards proposing this. For such a system I'd like to be able to recognize what is being printed on UART at the boot time. Then register the output in a file and later compare it. And I'd like enough expressiveness to say: "if there's no login screen after 30s of bot time, fail and let me know what the screen content at the time of failure was". DSL isn't a must here, I agree, but you'll be limited in testing otherwise. Unless some technology already has what I want, I suggested getting this functionality: vm0 = vm.start("vm0"); vm1 = vm.start("vm1"); vm0.power(1); vm1.power(1); vm0.login(); vm1.login(); vm0.cmd("ifconfig ....."); vm1.cmd("ifconfig ...."); vm0.cmd("ping vm1"); vm0.register(function(data) { if (data =~ /*Not*/) { error(); } }); Once this would work for VMs, I'd be interesting in proposing the creation of the physical infrastructure for continuous FreeBSD testing. We could start this from Atom-based PC and Raspberry PI netbooting latest FreeBSD. If John sees it in a similar way and he'd like to explore this path, I'd be very interested in mentoring this idea. But once again: only if there isn't something of a similar functionality. Thanks, -- Wojciech A. Koszek wkoszek@FreeBSD.czest.pl http://FreeBSD.czest.pl/~wkoszek/ From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 20 22:29:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A450CFE6; Thu, 20 Mar 2014 22:29:19 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F3A9CD91; Thu, 20 Mar 2014 22:29:18 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id hr17so1119390lab.32 for ; Thu, 20 Mar 2014 15:29:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VBP68Fe1BFCL4P3NgR2dEqIVxtTwzCS1DLCClhWUjXA=; b=zaAFuAiKT3VPVRAGpe1Bf3RVjbX5o0Qvw/W5I8MBcGH/EZpdqnGZD2s4wcesMiKNeC dg3VR26g0hZw2u9GN4FVUJpWWXMI2+/LiMPGMLHZdlAI2lxcRsDRXHCTnev4nBo8OSEV jEq6e+YBSkL1CaoMXqDpzkZJbPMiEMth+VcvNW/M7Y/1p4qwizTxQqs4Q+jeGhQ/BO4G gU8BDL9ADwN8hlcTx0epW+mt9YMFu7yLiMbRVK+PVEUUutshpFIVluiudz4d9Yq3za8L DEBsDm60Brv0S/AP86r1WAeaXywdldQ4DQ673IwT53/Ge3/OutvB4Lb2kI+0onaDPuGv 4//A== MIME-Version: 1.0 X-Received: by 10.112.92.172 with SMTP id cn12mr183909lbb.73.1395354557088; Thu, 20 Mar 2014 15:29:17 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 20 Mar 2014 15:29:17 -0700 (PDT) In-Reply-To: <20140320213848.GU37327@FreeBSD.org> References: <532A0DE6.2020801@pitt.edu> <201403201124.07472.jhb@freebsd.org> <20140320213848.GU37327@FreeBSD.org> Date: Thu, 20 Mar 2014 15:29:17 -0700 X-Google-Sender-Auth: r-j2RITa7tSmhzjl1b3O5vZYHbE Message-ID: Subject: Re: Google Summer of Code 2014 question From: Craig Rodrigues To: "Wojciech A. Koszek" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 22:29:19 -0000 On Thu, Mar 20, 2014 at 2:38 PM, Wojciech A. Koszek wrote: > DSL isn't a must here, I agree, but you'll be limited in testing otherwise. > Unless some technology already has what I want, I suggested getting this > functionality: > > vm0.cmd("ifconfig ....."); > vm1.cmd("ifconfig ...."); > > vm0.cmd("ping vm1"); > > vm0.register(function(data) { > if (data =~ /*Not*/) { > error(); > } > }); For the above pseudo-code which you listed, you've pretty much got that functionality in existing automation tools like Puppet, SaltStack, Chef, etc. For unit testing, Kyua is quite good, and people are writing lots of unit tests for it, and even committing some of them directly to FreeBSD. > Once this would work for VMs, I'd be interesting in proposing the creation > of the physical infrastructure for continuous FreeBSD testing. For the above item, we've already started putting that together. See: http://wiki.freebsd.org/Jenkins and the presentation I gave on March 13. At the next Devsummit in Ottawa, we want to talk about this further and get people who are interested in this topic to participate in the conversation: https://wiki.freebsd.org/201405DevSummit/Jenkins It would be interesting to get Raspberry PI hardware being tested regularly as part of continuous integration. We don't have Raspberry PI hardware available in the FreeBSD cluster, so someone who is interested would have to take the action item to acquire the hardware and set things up so the hardware is integrated with continuous integration/testing. Looking at the SummerOfCode2014 ideas page, my opinion is that for this item: https://wiki.freebsd.org/SummerOfCode2014#TEST-o-steron_for_FreeBSD_.28node.js.2FLua.29 -> has already been done with things like libvirt, which is being ported to FreeBSD by Roman Bogorodskiy. See: http://empt1e.blogspot.com/2014/03/bhyve-in-libvirt.html -> existing automation frameworks can already be used to send remote commands to VM's and for this item: https://wiki.freebsd.org/SummerOfCode2014#CON-tinuous_INtegration_for_FreeBSD_aka_improved_TinderBox_.28node.js.2FLua.29 -> better Tinderbox systems already exist. Jenkins and Buildbot are two commonly available open source implementations. -> the jenkins-admin team set up Jenkins ( http://jenkins.freebsd.org ) because we were familiar it, and so far it seems to work well My focus is to get existing software that people in the non-FreeBSD world have developed to solve these types of problems, and get them up and running in FreeBSD. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 21 02:40:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2804C147; Fri, 21 Mar 2014 02:40:10 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 20F7A934; Fri, 21 Mar 2014 02:40:08 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so1226777lan.26 for ; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=D0o61ra25zJw+RuT/2p57l9/QikuqC1mfgf/oCgsYZI=; b=GG/QVdsidnJNW1TasrVZuBLbr0KhLaL9RpzSsmV6B/1npOcQkDFDXvdivzp0PNOaHh 9uqC9HI8D7Pvn6cXwZlbbqEccjskUd439ktSR783coR3j3abNE7cfG46Djy+YB8s1gnK g8Q3LFVcxYgnom8GKwq/zWSbysHuNAKJqrvCuG4miID+DOtNQihgHileZIqsJuuy4uMG nuO8kmsrojIv5WYkNuwjeaEPLXMDwqNaQEawClZVawUA5S9WDYL24vWfUg3b6YjoTKyZ kupsjf5fub6gltNLNVgM0IZdGyQM2ixoNZVm3YhDqBUJVZmq9XJRHDBDBi+tkKqX4yni qISQ== MIME-Version: 1.0 X-Received: by 10.112.209.5 with SMTP id mi5mr22676854lbc.30.1395369606730; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) In-Reply-To: <20140318102615.21fa7fde@suse3.ewadmin.local> References: <20140318102615.21fa7fde@suse3.ewadmin.local> Date: Thu, 20 Mar 2014 19:40:06 -0700 X-Google-Sender-Auth: 7C8mgzzGL5O7id0sadDZPa4DAAw Message-ID: Subject: Re: Scripts for booting FreeBSD images from the install ISO for use in Jenkins? From: Craig Rodrigues To: freebsd-current Current Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:40:10 -0000 On Tue, Mar 18, 2014 at 2:26 AM, Rainer Duffner wrote: > Am Mon, 17 Mar 2014 19:30:01 -0700 > schrieb Craig Rodrigues : > >> Hi, >> >> For the BSD DevSummit in May, one of the items >> on our agenda: >> >> https://wiki.freebsd.org/201405DevSummit/Jenkins >> >> is to talk about writing scripts which can take a FreeBSD ISO image, >> and then boot it and run it on a remote system or in a VM >> to install the OS. After the OS is up, we would like to run tests. >> All of this would be triggered from Jenkins. >> >> Does anyone have scripts which can do this? >> Can they be contributed to the Jenkins effort on FreeBSD? >> >> If you have scripts in Python, Ruby, Bourne shell, etc. are all fine, >> or even recipes in automation frameworks like Puppet, Ansible, Chef, >> SaltStack, etc., >> please let us know! :) > > > > I would have loved to attend this talk: > > > http://2014.asiabsdcon.org/timetable.html.en#P7A > > > Hopefully, more documentation and/or the slides/the video for this talk > will become available. Rainer, Thanks for posting that link! It is highly relevant to my original posting. That looks like a really good presentation, and I also wish I could have attended the talk! There seem to be many automation frameworks for provisioning and booting VM's and real machines. I just need to learn one of them. :) -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 21 08:46:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9295977A for ; Fri, 21 Mar 2014 08:46:02 +0000 (UTC) Received: from frv198.fwdcdn.com (frv198.fwdcdn.com [212.42.77.198]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4572EBD7 for ; Fri, 21 Mar 2014 08:46:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=nv6MqQkNpIgahObvaWj5GL3BfM/091CWUZTFs1+NAnM=; b=V4vnec5dOH9cpFAyA/mwEptKAi1Car5kczCRLraMaL39t2q+uKqZ+4fG/kzZ32+hm8fAKUYm5i0EBCL+d6rp5OZ6mT3K28r+8ah7wvLiO3K6LTR1Ms88+tg6zjddDbwT/M72FovSuOzCPozN3h+mYJjp5+yYdAFSES+lV24YJz4=; Received: from [10.10.10.34] (helo=frv34.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1WQv5M-0003kq-5q for freebsd-virtualization@freebsd.org; Fri, 21 Mar 2014 10:45:52 +0200 Date: Fri, 21 Mar 2014 10:45:51 +0200 From: wishmaster Subject: Re[2]: misc/187665: unable to disable IPFW with VIMAGE To: "Andrey V. Elsukov" X-Mailer: mail.ukr.net 5.0 Message-Id: <1395391117.294356046.7txcun6f@frv34.fwdcdn.com> In-Reply-To: <532BF77D.6070007@FreeBSD.org> References: <532BF77D.6070007@FreeBSD.org> MIME-Version: 1.0 Received: from artemrts@ukr.net by frv34.fwdcdn.com; Fri, 21 Mar 2014 10:45:51 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline Cc: freebsd-virtualization@freebsd.org, bug-followup@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 08:46:02 -0000 --- Original message --- From: "Andrey V. Elsukov" Date: 21 March 2014, 10:26:23 > Hello, > > please, check the securelevel in your jail. What value does it have? > I use securelevel 2 (in jail only. In host - default), but securelevel does not impact to net.inet.ip.fw.enable at all, because this OID is absent in base system too. This happens only if options VIMAGE present in kernel! Cheers, Vitaliy From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 24 11:06:54 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABA10289 for ; Mon, 24 Mar 2014 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 97FD918E for ; Mon, 24 Mar 2014 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2OB6sml014035 for ; Mon, 24 Mar 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2OB6suX014033 for freebsd-virtualization@FreeBSD.org; Mon, 24 Mar 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Mar 2014 11:06:54 GMT Message-Id: <201403241106.s2OB6suX014033@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Wed Mar 26 15:49:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2EF5A5A8 for ; Wed, 26 Mar 2014 15:49:22 +0000 (UTC) Received: from mail-vc0-f176.google.com (mail-vc0-f176.google.com [209.85.220.176]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DFF2BE7D for ; Wed, 26 Mar 2014 15:49:21 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id lc6so2635836vcb.35 for ; Wed, 26 Mar 2014 08:49:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=r1v3ezgmolbLAMHEnzF0A03hNd0e4U2vLltZIJdU/cE=; b=h7riVZc/aCRvD07TY7+X+nXMbZ53saWlcUOR69WC9XTb8c2tzO13SmWYlzNwCmpTzT CBbzc04OtAzg54X1abhLQepwrMg+/K+V5dOqO0lEAX0Pn5necM13t3wgj2mj5PrB1cmn G/J5TqTcOFiE0oAbrHw3yFRIy/cG64yQCmii+iHOdM8jAoP461agHrvhKDgx/nMcyf5o zprDKsnUA+V0S1MMV+ibDjX18EH5xCHRKQPFEy6KLiBMKUCTfhvs12/AWTGKFLk896Ky 3KF8zPs44nBEGrtGJIjWgyYFs4dUC9RkWuKYyO0xs9AxUTgouOC30MaXdavhXyVHScnP tVoQ== X-Gm-Message-State: ALoCoQnvjYWeaaFsW5/bh2Xl5obbVwIvXvRxSQC7urxx9yDIbKJjKEajXqfAy4D3adrmejV47v2q X-Received: by 10.220.159.4 with SMTP id h4mr50213791vcx.1.1395848954769; Wed, 26 Mar 2014 08:49:14 -0700 (PDT) Received: from torb.pix.net (verizon.pix.net. [71.178.232.3]) by mx.google.com with ESMTPSA id dv1sm39010900vdd.21.2014.03.26.08.49.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Mar 2014 08:49:14 -0700 (PDT) Message-ID: <5332F6F8.1030400@pi-coral.com> Date: Wed, 26 Mar 2014 11:49:12 -0400 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: booting bhyve off zfs - a cautionary tale Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 15:49:22 -0000 Greetings all - I'm writing to report a solution to a puzzling problem that I've had for a while and just worked out a solution over the last two days. I've been working with the boot-bhyve-from-zfs patch that has been floating around for a while, and had good success with it initially. My configuration is such: the machine that run the hypervisor only has ZFS filesystems. As such, I just create new zvols on that machine, and map those into bhyve as the various vtbdX devices. I have a script that will create the zvols, then carefully partitions the zvol, create a zpool on the zvol, create a few other zfs filesystems on the zpool, and unpack the distribution .txz files into the resulting zpool. This worked well for a while and I could create new virtual machines without issue. And then, after an upgrade of the hosting machine to a more recent stable/10 image, all the new images I created would fail to start properly. The userboot.so support would locate the kernel, and load it, but the mounting of the root partition would fail: --- snip, snip --- ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec Timecounter "TSC-low" frequency 1149929960 Hz quality 800 Trying to mount root from zfs:nine5/ROOT/default []... Mounting from zfs:nine5/ROOT/default failed with error 45. Loader variables: vfs.root.mountfrom=zfs:nine5/ROOT/default --- snip, snip --- After far too much time groveling through the boot code, I finally had the realization of what was going on. Since I was creating the new zpool for the client machine under stable/10, and I couldn't boot it under stable/9, there must have been some incompatibility introduced. Anyway, to make a long debugging story shorter, the newest zpool-features (enabled by default) in stable/10 prevent the stable/9 kernel from successfully being about to mountroot() the zpool that created under stable/10. It turns out there is a flag to suppress enabling all the zpool-features when a new zpool is created. So I switched my script to generate the zpool, specifying "zpool create -d ...", and then enabling the features that are OK for stable/9: zpool set feature@async_destroy=enabled ${zroot} zpool set feature@empty_bpobj=enabled ${zroot} zpool set feature@lz4_compress=enabled ${zroot} After these small changes with regards to the creation of the zpool, I can now successfully boot the resulting zpool-on-a-zvol without issue. Hopefully this information will save someone a bunch of time debugging this problem in the future. -Kurt From owner-freebsd-virtualization@FreeBSD.ORG Thu Mar 27 15:40:02 2014 Return-Path: Delivered-To: freebsd-virtualization@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5AB7613 for ; Thu, 27 Mar 2014 15:40:02 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 865EAF2D for ; Thu, 27 Mar 2014 15:40:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2RFe2Ev027442 for ; Thu, 27 Mar 2014 15:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2RFe2if027441; Thu, 27 Mar 2014 15:40:02 GMT (envelope-from gnats) Date: Thu, 27 Mar 2014 15:40:02 GMT Message-Id: <201403271540.s2RFe2if027441@freefall.freebsd.org> To: freebsd-virtualization@FreeBSD.org Cc: From: Philipp Schmid Subject: Re: kern/160496: [vimage] [pf] [patch] kernel panic with pf + VIMAGE X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Philipp Schmid List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 15:40:02 -0000 The following reply was made to PR kern/160496; it has been noted by GNATS. From: Philipp Schmid To: bug-followup@FreeBSD.org, nvass@gmx.com Cc: Subject: Re: kern/160496: [vimage] [pf] [patch] kernel panic with pf + VIMAGE Date: Thu, 27 Mar 2014 16:36:58 +0100 --Apple-Mail=_E95A62E0-814B-4C29-9BAB-EABD685BD926 Content-Type: multipart/alternative; boundary="Apple-Mail=_1C9488DE-00AC-4D94-BB20-D4456B20AAC8" --Apple-Mail=_1C9488DE-00AC-4D94-BB20-D4456B20AAC8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I think I have the same problem and also created a bug report: = http://www.freebsd.org/cgi/query-pr.cgi?pr=3D188018 Also this thread seems to be related: = http://lists.freebsd.org/pipermail/freebsd-current/2014-March/048787.html All the best, Philipp --Apple-Mail=_1C9488DE-00AC-4D94-BB20-D4456B20AAC8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii I = think I have the same problem and also created a bug report: http://www.fr= eebsd.org/cgi/query-pr.cgi?pr=3D188018

Also this = thread seems to be related:


All the = best,

Philipp


= --Apple-Mail=_1C9488DE-00AC-4D94-BB20-D4456B20AAC8-- --Apple-Mail=_E95A62E0-814B-4C29-9BAB-EABD685BD926 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTNEWaAAoJEFOb3dIWHqZVaBsP/2JSpHz5gwc74R5ZgNX18V7S kldvkXY5wl+8PWbzCbiAdyQ+Pt1nJIfda8Qr6qh5XH3hkAn03UUXc52nfGYjbe4L OIqjEWoLeVTy+LYFHJJDqn3vi82IO6RLW0WxFyLnwDsqi0ZltlNiOY7Ck/07CuGt QrAg+2hcPLGRjP9/g8tBXQSNFHE1DJNNcCFmbh3eY2YD7h1ntpAjuWxZ1RnFQvi9 RUkZVoRFUv/zqTwJwYKt+hEo9tc7Y5NgYnI0CIFgHQFsJtxT4HAPT2Khnw2KWVf9 NgHLRnoO0oO7RahUFcoLN7EU/2BLRPyHZ7T9fKS9jQ9t0/FX/ynSboA7Igjpler/ dlmgZKiBOrayiwnyWzsGBj25DXpYTNON36AMHK/j0NNfsT9bHs+n3LN8S2OPaqQg HV/3GubQYh/mWHB9bsaj0f/6+l2dDShK5jCsiqxgCF3oJ7uE19yuSqgnplY9DPPQ 0Cq+43CVyCsKIFfK0xDHEJPkgvVV3aNzOE4Sd6ttgpb+HRSyZwPqoYbe10l/Ylpe ycddsF+SF+ynOZqcehZZpz/AXIQg9jU/YdgP9PAhKrsqI44GCHKN+xGVMZiZ+IHc k+1sIhJaELCucOCYWznucp1hs6iV48ekpiHWuKdMWiV3KfMo7sFMGiQvkMeir7+g 6fPsDROYzv2KRwPGmd7B =xIYn -----END PGP SIGNATURE----- --Apple-Mail=_E95A62E0-814B-4C29-9BAB-EABD685BD926-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 29 17:58:16 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0262D70F; Sat, 29 Mar 2014 17:58:16 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.fisglobal.com", Issuer "VeriSign Class 3 Secure Server CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6486B8E; Sat, 29 Mar 2014 17:58:12 +0000 (UTC) Received: from smarthost.fisglobal.com ([10.132.206.193]) by ltcfislmsgpa04.fnfis.com (8.14.5/8.14.5) with ESMTP id s2THw8NV013930 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 29 Mar 2014 12:58:08 -0500 Received: from THEMADHATTER (10.242.181.54) by smarthost.fisglobal.com (10.132.206.193) with Microsoft SMTP Server id 14.3.174.1; Sat, 29 Mar 2014 12:58:07 -0500 From: Sender: Devin Teske To: "'Palle Girgensohn'" References: <4FD66519.8030503@FreeBSD.org> In-Reply-To: <4FD66519.8030503@FreeBSD.org> Subject: RE: VIMAGE, epair/if_bridge or netgraph? Date: Sat, 29 Mar 2014 10:58:01 -0700 Message-ID: <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGojnTtmr+0A7SxHiVcb6yw3Zz8PptF0jlA Content-Language: en-us X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87, 1.0.14, 0.0.0000 definitions=2014-03-29_02:2014-03-28,2014-03-29,1970-01-01 signatures=0 Cc: 'Devin Teske' , freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Mar 2014 17:58:16 -0000 > -----Original Message----- > From: owner-freebsd-virtualization@freebsd.org [mailto:owner-freebsd- > virtualization@freebsd.org] On Behalf Of Palle Girgensohn > Sent: Monday, June 11, 2012 2:37 PM > To: freebsd-virtualization@FreeBSD.org > Subject: VIMAGE, epair/if_bridge or netgraph? >=20 > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > Hi, >=20 > I'm updating some jail servers, and want to use VIMAGE. Compiled it into = the > kernel, learned the hard way not to even include PF in the same kernel [1= ], > so now it works quite well. >=20 > I am setting up many similar jails, some for testing, some for production= . The > applications are web servers, som tomcat+apache's, and some other > standard type of services like email and ldap, simple stuff. > I need no fancy network control, I just need it to work. For each jail th= ere are > two interfaces, one public, connected to a software bridge (if_bridge or > ng_bridge) acting as a switch, and one internal, for maintenance, connect= ed > to a different software bridge. To each software bridge, I connect a phys= ical > external interface from the jail host. >=20 > I am trying to decide whether to use epair and if_bridge, or to use netgr= aph. > For netgraph, there is a nice package at DruidBSD [3]. When I found that,= I > had already rewritten the standard jail script, using the > v2 patches from polymorf [4]. They work equally fine for my purpose. >=20 > So now I need to know which scales best, is there a difference in > performance or stability between netgraph and epair/if_bridge? >=20 > Cheers, > Palle >=20 >=20 > [1] http://forums.freebsd.org/showthread.php?t=3D31765 >=20 > [2] http://forums.freebsd.org/showthread.php?t=3D31949 >=20 > [3] http://druidbsd.sourceforge.net/vimage.shtml >=20 > [4] http://wiki.polymorf.fr/index.php?title=3DHowto:FreeBSD_jail_vnet [Devin Teske]=20 Never saw a reply to this and I'm locating round-tuits to tackle e-mails that I've marked as "needing reply": I have not profiled netgraph to have a limitation of 65530 eiface devices off a single if_bridge, but are allowed multiple bridges with that many devices. The problems that you run into with that many devices is that if all the interfaces are visible to a single jail or single host... your "ifconfig" command could take several hours (about 4) to enumerate each iface to the screen. I didn't mess much with epair because it failed to produce a situation where I could speak separate subnets over the same wire. Netgraph made it easy by way of being able to enable promiscuous and disable the "autosrc" feature (as you perhaps already found in my code you linked to above). --=20 Cheers, Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-virtualization@FreeBSD.ORG Sat Mar 29 18:08:24 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEACB841; Sat, 29 Mar 2014 18:08:24 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.fisglobal.com", Issuer "VeriSign Class 3 Secure Server CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8928BC51; Sat, 29 Mar 2014 18:08:24 +0000 (UTC) Received: from smarthost.fisglobal.com ([10.132.206.193]) by ltcfislmsgpa03.fnfis.com (8.14.5/8.14.5) with ESMTP id s2TI8NPh019453 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 29 Mar 2014 13:08:23 -0500 Received: from THEMADHATTER (10.242.181.54) by smarthost.fisglobal.com (10.132.206.193) with Microsoft SMTP Server id 14.3.174.1; Sat, 29 Mar 2014 13:08:22 -0500 From: Sender: Devin Teske To: , "'Palle Girgensohn'" References: <4FD66519.8030503@FreeBSD.org> <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> In-Reply-To: <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> Subject: RE: VIMAGE, epair/if_bridge or netgraph? Date: Sat, 29 Mar 2014 11:08:16 -0700 Message-ID: <036601cf4b79$dc61d9c0$95258d40$@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGojnTtmr+0A7SxHiVcb6yw3Zz8PgLj99Ormy62pIA= Content-Language: en-us X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87, 1.0.14, 0.0.0000 definitions=2014-03-29_02:2014-03-28,2014-03-29,1970-01-01 signatures=0 Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Mar 2014 18:08:24 -0000 > -----Original Message----- > From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] > Sent: Saturday, March 29, 2014 10:58 AM > To: 'Palle Girgensohn' > Cc: freebsd-virtualization@FreeBSD.org; 'Devin Teske' > Subject: RE: VIMAGE, epair/if_bridge or netgraph? >=20 >=20 >=20 > > -----Original Message----- > > From: owner-freebsd-virtualization@freebsd.org [mailto:owner-freebsd- > > virtualization@freebsd.org] On Behalf Of Palle Girgensohn > > Sent: Monday, June 11, 2012 2:37 PM > > To: freebsd-virtualization@FreeBSD.org > > Subject: VIMAGE, epair/if_bridge or netgraph? > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Hi, > > > > I'm updating some jail servers, and want to use VIMAGE. Compiled it > > into the kernel, learned the hard way not to even include PF in the > > same kernel [1], so now it works quite well. > > > > I am setting up many similar jails, some for testing, some for > > production. The applications are web servers, som tomcat+apache's, and > > some other standard type of services like email and ldap, simple stuff. > > I need no fancy network control, I just need it to work. For each jail > > there are two interfaces, one public, connected to a software bridge > > (if_bridge or > > ng_bridge) acting as a switch, and one internal, for maintenance, > > connected to a different software bridge. To each software bridge, I > > connect a physical external interface from the jail host. > > > > I am trying to decide whether to use epair and if_bridge, or to use > netgraph. > > For netgraph, there is a nice package at DruidBSD [3]. When I found > > that, I had already rewritten the standard jail script, using the > > v2 patches from polymorf [4]. They work equally fine for my purpose. > > > > So now I need to know which scales best, is there a difference in > > performance or stability between netgraph and epair/if_bridge? > > > > Cheers, > > Palle > > > > > > [1] http://forums.freebsd.org/showthread.php?t=3D31765 > > > > [2] http://forums.freebsd.org/showthread.php?t=3D31949 > > > > [3] http://druidbsd.sourceforge.net/vimage.shtml > > > > [4] http://wiki.polymorf.fr/index.php?title=3DHowto:FreeBSD_jail_vnet >=20 > [Devin Teske] >=20 > Never saw a reply to this and I'm locating round-tuits to tackle e-mails = that > I've marked as "needing reply": >=20 > I have not profiled Ugh, that was originally "I have not profiled [epair but I have profiled] n= etgraph" --=20 Cheers, Devin > netgraph to have a limitation of 65530 eiface devices off a > single if_bridge, but are allowed multiple bridges with that many devices. >=20 > The problems that you run into with that many devices is that if all the > interfaces are visible to a single jail or single host... your "ifconfig" > command could take several hours (about 4) to enumerate each iface to the > screen. >=20 > I didn't mess much with epair because it failed to produce a situation wh= ere I > could speak separate subnets over the same wire. Netgraph made it easy by > way of being able to enable promiscuous and disable the "autosrc" feature > (as you perhaps already found in my code you linked to above). > -- > Cheers, > Devin >=20 >=20 >=20 > _____________ > The information contained in this message is proprietary and/or confident= ial. > If you are not the intended recipient, please: (i) delete the message and= all > copies; (ii) do not disclose, distribute or use the message in any manner= ; and > (iii) notify the sender immediately. In addition, please be aware that any > message addressed to our domain is subject to archiving and review by > persons other than the intended recipient. Thank you. _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 11:06:53 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB356BF3 for ; Mon, 31 Mar 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C762BBB3 for ; Mon, 31 Mar 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2VB6r8I058873 for ; Mon, 31 Mar 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2VB6rcL058871 for freebsd-virtualization@FreeBSD.org; Mon, 31 Mar 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 31 Mar 2014 11:06:53 GMT Message-Id: <201403311106.s2VB6rcL058871@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 11:53:57 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2EB2EF7A; Mon, 31 Mar 2014 11:53:57 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id C3BEC171; Mon, 31 Mar 2014 11:53:56 +0000 (UTC) Received: from [10.0.0.167] (citron2.pingpong.net [195.178.173.68]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id 8DF54353A9; Mon, 31 Mar 2014 13:44:01 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_22885038-3993-4C64-A244-C5F55FE221EE"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: VIMAGE, epair/if_bridge or netgraph? From: Palle Girgensohn In-Reply-To: <036601cf4b79$dc61d9c0$95258d40$@FreeBSD.org> Date: Mon, 31 Mar 2014 13:44:00 +0200 Message-Id: <2E1F87DA-0CC6-4BEE-BF82-2210D49643BF@FreeBSD.org> References: <4FD66519.8030503@FreeBSD.org> <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> <036601cf4b79$dc61d9c0$95258d40$@FreeBSD.org> To: dteske@freebsd.org X-Mailer: Apple Mail (2.1874) Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 11:53:57 -0000 --Apple-Mail=_22885038-3993-4C64-A244-C5F55FE221EE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii 29 mar 2014 kl. 19:08 skrev dteske@freebsd.org: >=20 >=20 >> -----Original Message----- >> From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] >> Sent: Saturday, March 29, 2014 10:58 AM >> To: 'Palle Girgensohn' >> Cc: freebsd-virtualization@FreeBSD.org; 'Devin Teske' >> Subject: RE: VIMAGE, epair/if_bridge or netgraph? >>=20 >>=20 >>=20 >>> -----Original Message----- >>> From: owner-freebsd-virtualization@freebsd.org = [mailto:owner-freebsd- >>> virtualization@freebsd.org] On Behalf Of Palle Girgensohn >>> Sent: Monday, June 11, 2012 2:37 PM >>> To: freebsd-virtualization@FreeBSD.org >>> Subject: VIMAGE, epair/if_bridge or netgraph? >>>=20 >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>>=20 >>> Hi, >>>=20 >>> I'm updating some jail servers, and want to use VIMAGE. Compiled it >>> into the kernel, learned the hard way not to even include PF in the >>> same kernel [1], so now it works quite well. >>>=20 >>> I am setting up many similar jails, some for testing, some for >>> production. The applications are web servers, som tomcat+apache's, = and >>> some other standard type of services like email and ldap, simple = stuff. >>> I need no fancy network control, I just need it to work. For each = jail >>> there are two interfaces, one public, connected to a software bridge >>> (if_bridge or >>> ng_bridge) acting as a switch, and one internal, for maintenance, >>> connected to a different software bridge. To each software bridge, I >>> connect a physical external interface from the jail host. >>>=20 >>> I am trying to decide whether to use epair and if_bridge, or to use >> netgraph. >>> For netgraph, there is a nice package at DruidBSD [3]. When I found >>> that, I had already rewritten the standard jail script, using the >>> v2 patches from polymorf [4]. They work equally fine for my purpose. >>>=20 >>> So now I need to know which scales best, is there a difference in >>> performance or stability between netgraph and epair/if_bridge? >>>=20 >>> Cheers, >>> Palle >>>=20 >>>=20 >>> [1] http://forums.freebsd.org/showthread.php?t=3D31765 >>>=20 >>> [2] http://forums.freebsd.org/showthread.php?t=3D31949 >>>=20 >>> [3] http://druidbsd.sourceforge.net/vimage.shtml >>>=20 >>> [4] http://wiki.polymorf.fr/index.php?title=3DHowto:FreeBSD_jail_vnet >>=20 >> [Devin Teske] >>=20 >> Never saw a reply to this and I'm locating round-tuits to tackle = e-mails that >> I've marked as "needing reply": >>=20 >> I have not profiled >=20 > Ugh, that was originally "I have not profiled [epair but I have = profiled] netgraph" > --=20 > Cheers, > Devin >=20 >> netgraph to have a limitation of 65530 eiface devices off a >> single if_bridge, but are allowed multiple bridges with that many = devices. >>=20 >> The problems that you run into with that many devices is that if all = the >> interfaces are visible to a single jail or single host... your = "ifconfig" >> command could take several hours (about 4) to enumerate each iface to = the >> screen. >>=20 >> I didn't mess much with epair because it failed to produce a = situation where I >> could speak separate subnets over the same wire. Netgraph made it = easy by >> way of being able to enable promiscuous and disable the "autosrc" = feature >> (as you perhaps already found in my code you linked to above). >> -- >> Cheers, >> Devin >>=20 Thanks for the response. I have since created a setup with epair, only to abandon it and pursue a = setup with netgraph instead. I can't yet say which will best serve my = needs, I can get back to that when I have more data.=20 I do know that shutting down a jail that has epairs enabled very likely = will panic the kernel. I'm not certain that netgraph is any different, = but I have no data yey. I do know that some fixes have been made to = kernel to avoid crashes. I'll get back with more info as I have more info to reveal. :) Cheers, Palle --Apple-Mail=_22885038-3993-4C64-A244-C5F55FE221EE Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTOVUAAAoJEIhV+7FrxBJDcVAH/25G7vo/KlIhR84aXVe+NB60 I6HABRYid8YdiYJ+Nz+IVwB0U7Ipr12020UJrwBWOz3IdTnnxu9hdE4XAYjRa8K3 3jGJ31RLWt7LPnirtAgr+nEwsGrj995AT0F0QRMP2yLuXQ5C0cDT/TGELO87lxLN rnjeh91TU9BkvaSVg1xZRDwoVyyBIjBQGrwKvDwXRgP+DSoW/khOWiJmnEelHeQ4 MtOiQGLo47b/DkgxwABZNXCvHKGdm/V7MO5DUUIcB9ct//KWul9ds0SrEAVhFdsw Q0pMPxnKwJcVhT3ZxdXi9bREj/oCCZ+rpZMmwa5PHGiGeyeQcc5uVxLmk97a8C4= =kmjg -----END PGP SIGNATURE----- --Apple-Mail=_22885038-3993-4C64-A244-C5F55FE221EE-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 15:18:48 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C9D6B98; Mon, 31 Mar 2014 15:18:48 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.fisglobal.com", Issuer "VeriSign Class 3 Secure Server CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 348B9BF1; Mon, 31 Mar 2014 15:18:47 +0000 (UTC) Received: from smarthost.fisglobal.com ([10.132.206.192]) by ltcfislmsgpa03.fnfis.com (8.14.5/8.14.5) with ESMTP id s2VFIkgq032196 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Mon, 31 Mar 2014 10:18:46 -0500 Received: from THEMADHATTER (10.242.181.54) by smarthost.fisglobal.com (10.132.206.192) with Microsoft SMTP Server id 14.3.174.1; Mon, 31 Mar 2014 10:18:44 -0500 From: Sender: Devin Teske To: "'Palle Girgensohn'" , References: <4FD66519.8030503@FreeBSD.org> <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> <036601cf4b79$dc61d9c0$95258d40$@FreeBSD.org> <2E1F87DA-0CC6-4BEE-BF82-2210D49643BF@FreeBSD.org> In-Reply-To: <2E1F87DA-0CC6-4BEE-BF82-2210D49643BF@FreeBSD.org> Subject: RE: VIMAGE, epair/if_bridge or netgraph? Date: Mon, 31 Mar 2014 08:18:37 -0700 Message-ID: <065e01cf4cf4$7e0bd6b0$7a238410$@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGojnTtmr+0A7SxHiVcb6yw3Zz8PgLj99OrANV2hGkBa4f+BZsfo26A Content-Language: en-us X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87, 1.0.14, 0.0.0000 definitions=2014-03-31_02:2014-03-31,2014-03-31,1970-01-01 signatures=0 Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 15:18:48 -0000 > -----Original Message----- > From: Palle Girgensohn [mailto:girgen@FreeBSD.org] > Sent: Monday, March 31, 2014 4:44 AM > To: dteske@freebsd.org > Cc: freebsd-virtualization@FreeBSD.org > Subject: Re: VIMAGE, epair/if_bridge or netgraph? > > > 29 mar 2014 kl. 19:08 skrev dteske@freebsd.org: > > > > > > >> -----Original Message----- > >> From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] > >> Sent: Saturday, March 29, 2014 10:58 AM > >> To: 'Palle Girgensohn' > >> Cc: freebsd-virtualization@FreeBSD.org; 'Devin Teske' > >> Subject: RE: VIMAGE, epair/if_bridge or netgraph? > >> > >> > >> > >>> -----Original Message----- > >>> From: owner-freebsd-virtualization@freebsd.org > >>> [mailto:owner-freebsd- virtualization@freebsd.org] On Behalf Of > >>> Palle Girgensohn > >>> Sent: Monday, June 11, 2012 2:37 PM > >>> To: freebsd-virtualization@FreeBSD.org > >>> Subject: VIMAGE, epair/if_bridge or netgraph? > >>> > >>> -----BEGIN PGP SIGNED MESSAGE----- > >>> Hash: SHA1 > >>> > >>> Hi, > >>> > >>> I'm updating some jail servers, and want to use VIMAGE. Compiled it > >>> into the kernel, learned the hard way not to even include PF in the > >>> same kernel [1], so now it works quite well. > >>> > >>> I am setting up many similar jails, some for testing, some for > >>> production. The applications are web servers, som tomcat+apache's, > >>> and some other standard type of services like email and ldap, simple > stuff. > >>> I need no fancy network control, I just need it to work. For each > >>> jail there are two interfaces, one public, connected to a software > >>> bridge (if_bridge or > >>> ng_bridge) acting as a switch, and one internal, for maintenance, > >>> connected to a different software bridge. To each software bridge, I > >>> connect a physical external interface from the jail host. > >>> > >>> I am trying to decide whether to use epair and if_bridge, or to use > >> netgraph. > >>> For netgraph, there is a nice package at DruidBSD [3]. When I found > >>> that, I had already rewritten the standard jail script, using the > >>> v2 patches from polymorf [4]. They work equally fine for my purpose. > >>> > >>> So now I need to know which scales best, is there a difference in > >>> performance or stability between netgraph and epair/if_bridge? > >>> > >>> Cheers, > >>> Palle > >>> > >>> > >>> [1] > >>> https://urldefense.proofpoint.com/v1/url?u=http://forums.freebsd.org > >>> > /showthread.php?t%3D31765&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D > %0A&r=Mr > >>> > js6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S > %2BolAmp > >>> > ovPuqBDL%2FSJ3eiM%3D%0A&s=14d4e7005de0720881a8a37c21d7738c5efac > 19fd3 > >>> 6a40fd9d86339469412b1c > >>> > >>> [2] > >>> https://urldefense.proofpoint.com/v1/url?u=http://forums.freebsd.org > >>> > /showthread.php?t%3D31949&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D > %0A&r=Mr > >>> > js6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S > %2BolAmp > >>> > ovPuqBDL%2FSJ3eiM%3D%0A&s=526e98adfe7b28bb2e9387eda1ad4745c142 > 4e8662 > >>> 2109a1b26d53e1ed4526b3 > >>> > >>> [3] > >>> https://urldefense.proofpoint.com/v1/url?u=http://druidbsd.sourcefor > >>> > ge.net/vimage.shtml&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r= > Mrjs6vR4 > >>> > %2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S%2Bol > AmpovPuqB > >>> > DL%2FSJ3eiM%3D%0A&s=fa628e6b3896b8f1b75b2eda070a9b65375e564e736 > 21da1 > >>> ddf12c18fe56c612 > >>> > >>> [4] > >>> https://urldefense.proofpoint.com/v1/url?u=http://wiki.polymorf.fr/i > >>> > ndex.php?title%3DHowto:FreeBSD_jail_vnet&k=%2FbkpAUdJWZuiTILCq%2F > FnQ > >>> > g%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWz > TO24ZzksQH > >>> > z%2Bx0S%2BolAmpovPuqBDL%2FSJ3eiM%3D%0A&s=2762f34c39dd7b58b8b3 > 98d89fa > >>> 0f7fe7e4900978664f25eafb66e1d4aedcdeb > >> > >> [Devin Teske] > >> > >> Never saw a reply to this and I'm locating round-tuits to tackle > >> e-mails that I've marked as "needing reply": > >> > >> I have not profiled > > > > Ugh, that was originally "I have not profiled [epair but I have profiled] > netgraph" > > -- > > Cheers, > > Devin > > > >> netgraph to have a limitation of 65530 eiface devices off a single > >> if_bridge, but are allowed multiple bridges with that many devices. > >> > >> The problems that you run into with that many devices is that if all > >> the interfaces are visible to a single jail or single host... your "ifconfig" > >> command could take several hours (about 4) to enumerate each iface to > >> the screen. > >> > >> I didn't mess much with epair because it failed to produce a > >> situation where I could speak separate subnets over the same wire. > >> Netgraph made it easy by way of being able to enable promiscuous and > >> disable the "autosrc" feature (as you perhaps already found in my code > you linked to above). > >> -- > >> Cheers, > >> Devin > >> > > > Thanks for the response. > > I have since created a setup with epair, only to abandon it and pursue a setup > with netgraph instead. I can't yet say which will best serve my needs, I can > get back to that when I have more data. > > I do know that shutting down a jail that has epairs enabled very likely will > panic the kernel. I'm not certain that netgraph is any different, but I have no > data yey. I do know that some fixes have been made to kernel to avoid > crashes. > > I'll get back with more info as I have more info to reveal. :) > In my experience (which has been with 8.1, 8.3, 8.4, stable/8, 9.0, 9.1, 9.2 and stable/9) is that when you shut down a jail that still has a netgraph eiface in it is that (if the management script didn't reclaim the interface properly using "ifconfig IFACE -vnet JID") the eiface is still active but not visible to any jail nor to the host machine (read: no crash). You can still see the eiface with "ngctl ls -l" and can do "shutdown" commands on it without a panic. You can also (if you know the JID) manually reclaim it (also without panic). -- Devin _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 17:00:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05FF1649; Mon, 31 Mar 2014 17:00:18 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53A3BA0C; Mon, 31 Mar 2014 17:00:17 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so5973138lab.41 for ; Mon, 31 Mar 2014 10:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=UuIiAQfCNdEGv4jSjFTJ6HKQLoyQ3R6YvepZEIkElws=; b=Af6clRT+qADDuSOfE91lrOwznnSZ5sddjG18jlLYKYBqqOp+7lzSNNT5I6OfZWqnMU 3XDqWHKe4ySwYlZbKa3ZTzUQBcPMkSOrydIhg/L5iKwI/T7NGneRybKlaHj12+Kal0nH tvjoZkGj5UhN87gBO3hOJEMK4Nuosp8ZqlBoz/S0gEgHfxHdW7yk740QzCTI29Cy0Pn4 SuqTd1N765xwEh8b1fwdbbSVmg1liJJ6kn6OAx8RRm/dJNcV5SBQE31p/JYgblc0P1Ph ZBlejdRj7qoSS3jjq7lFGTM+LWHq0fAKAlVmPpKhBL9Tqhykjp9DbjAyJfWGZGMayezQ u0sg== X-Received: by 10.112.141.10 with SMTP id rk10mr3152969lbb.26.1396285215264; Mon, 31 Mar 2014 10:00:15 -0700 (PDT) Received: from kloomba ([77.94.196.45]) by mx.google.com with ESMTPSA id n9sm10285186lbg.6.2014.03.31.10.00.13 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 31 Mar 2014 10:00:14 -0700 (PDT) Sender: Roman Bogorodskiy Date: Mon, 31 Mar 2014 21:00:07 +0400 From: Roman Bogorodskiy To: freebsd-virtualization@FreeBSD.org Subject: Bhyve support in Libvirt Message-ID: <20140331170006.GD1275@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rqzD5py0kzyFAOWN" Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 17:00:18 -0000 --rqzD5py0kzyFAOWN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi! I was suggested it'd be useful to share that info on this list. The news is that libvirt 1.2.2 released around a month ago was the first libvirt version with Bhyve support! Libvirt is the virtualization library which aims to provide a unified API for hypervisors (and actually even more things, storage for example). Libvirt webpage is here: http://libvirt.org/ I've provided some more details in a blog post I wrote couple of weeks ago: http://empt1e.blogspot.ru/2014/03/bhyve-in-libvirt.html also, there is some info on the Qemu driver status as well in the blog, if you're interested. Roman Bogorodskiy --rqzD5py0kzyFAOWN Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTOZ8WAAoJEMltX/4IwiJqaMgH/267u9V62Z6b1ikjVbhZjXf9 RfjYd0K5t7sVTBFVqhWHAkq+9vIQDjObu0YxBPOWJ7FAxv+kVgrEWLTHqJH1lsKy 2ELnVYOvVEdzKJV61KZgLgrQWoER9tceZ+O2N+ODZRsE7sEAZArcX2Wf9JBQk4IH kEcwi1+cambllyWelP5s8CEWzvh9HFtnj9laCj4v5pB23qtwqog0HTi+taiNOPD5 xwPLtncADOcGxYZg9n7nA4g9X/IB+GuLJzO8cXrCT9avxcJkbEy83z51yjcItafs TABQj4DZ9l5eki1h5AdRHdSpt+jwb6aCqRg5Cudi62WV/+3C2cDQOJ4sFc2pdUE= =ihiu -----END PGP SIGNATURE----- --rqzD5py0kzyFAOWN-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 18:13:59 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E111BF2 for ; Mon, 31 Mar 2014 18:13:59 +0000 (UTC) Received: from nexus.itaserv.net (nexus.itaserv.net [5.9.118.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8E941B8 for ; Mon, 31 Mar 2014 18:13:58 +0000 (UTC) Received: from [78.134.26.143] by itaserv.net with HTTP; Mon, 31 Mar 2014 20:08:45 +0200 From: bRa B Date: Mon, 31 Mar 2014 20:08:45 +0200 X-Mailer: Axigen WebMail To: freebsd-virtualization@freebsd.org Message-ID: <1396289325336661500@itaserv.net> Subject: Bhyve on a Core i7-920 Importance: Normal MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-AxigenSpam-Level: 4 X-AxigenVirus-Level: 1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 18:13:59 -0000 =E2=80=8BHowdy folks, i have encountered a strange issue when i was trying to run CentOS 6.5 with= bhyve on a server with 11.0-CURRENT FreeBSD Installed: VM unrestricted guest capability required Error in initializing VM the machine has 48GB of ram, and the cpu is a i7 920 as stated in the subje= ct of this email.=20 It has the instructions required to run bhyve ( VMX and POPCNT, as they app= ear in the dmesg log and in the Intel's Ark: http://ark.intel.com/products/= 37147/Intel-Core-i7-920-Processor-8M-Cache-2_66-GHz-4_80-GTs-Intel-QPI?q=3D= core%20i7-920 ). Is there something wrong with this CPU?=20 Thanks Brando From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 18:20:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32FE7CB9; Mon, 31 Mar 2014 18:20:58 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8579E201; Mon, 31 Mar 2014 18:20:57 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf5so6108669lab.35 for ; Mon, 31 Mar 2014 11:20:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=YyiEg1TKWyClKVD7lJYmY0gziR69sCJy55/EDOWQra4=; b=VM27V6c2KIJAPcdKNzV2a7EUIKrdar90eKoeMdqhm88TJqK3qDJS7HH4XnSJTQxHuL lp/LgOnFs1f9LPTnz+xG4DgVF2x8shdbVGjs+M9TI5n/a3Ie6aXBvDkaSrxdkBXvLn5T WbvfwQxQV19l5SWWYPacyfKGblyBQvkzvfJNly0ha2gwmMU5XzLwJCRaD+P0j7cAKp9Z e0OSpdnORirPwq0i6nW4I65b0EcGeXMl1+J4A1WbSUscA3PuQiZm539DLJbMy+gHPfV1 zXiVexFAWPsI7AQC9Cst2F1Bn4WWxKn2EjiGzkmZDWCpRwzjXcpbuUDrBdCqcKQoi7Ex mX7Q== MIME-Version: 1.0 X-Received: by 10.112.13.101 with SMTP id g5mr2286486lbc.39.1396290055573; Mon, 31 Mar 2014 11:20:55 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Mon, 31 Mar 2014 11:20:55 -0700 (PDT) In-Reply-To: <20140331170006.GD1275@kloomba> References: <20140331170006.GD1275@kloomba> Date: Mon, 31 Mar 2014 11:20:55 -0700 X-Google-Sender-Auth: 8kW-bF0ZZ_GJ8Obgid3QoWjc2EU Message-ID: Subject: Re: Bhyve support in Libvirt From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 18:20:58 -0000 On Mon, Mar 31, 2014 at 10:00 AM, Roman Bogorodskiy wrote: > > http://empt1e.blogspot.ru/2014/03/bhyve-in-libvirt.html > This is a huge step forward for bhyve! Thank you! There is a sizable ecosystem of software for managing hypervisors built on top of libvirt. Have you tested any of this software, such as virsh? Are there any GUI or web UI utilities built on top of libvirt which work with bhyve? You might want to consider submitting a status report at: https://www.freebsd.org/news/status/status.html -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 18:29:24 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 779BEF91 for ; Mon, 31 Mar 2014 18:29:24 +0000 (UTC) Received: from a0i308.smtpcorp.com (a0i308.smtpcorp.com [216.22.15.140]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D34C2CE for ; Mon, 31 Mar 2014 18:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpcorp.com; s=a0_1; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:Subject:Cc:To:From; bh=LiuhrlORhuXrEde06KO2vM0+QTHaHAJQtMoGpYW38mc=; b=VpOClIhMdCqrNanhZubEh2MJLkG8X9t/ybjX+COSp5QgfPLMDsavbXeWEdxDFmZUORM3PM4DH9XylTCW1+YxVZJW6IheEu5A05JA2r8l0z7mDKgPCUtkH4Fa0Rd6Tjbdc7w/7E1snCQLfj2BBBsBGdcaULJwq+EVow3pX7f05K4=; From: Daniel Corbe To: bRa B Subject: Re: Bhyve on a Core i7-920 References: <1396289325336661500@itaserv.net> Date: Mon, 31 Mar 2014 14:25:00 -0400 In-Reply-To: <1396289325336661500@itaserv.net> (bRa B.'s message of "Mon, 31 Mar 2014 20:08:45 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Smtpcorp-Track: 454298245.1.28608399 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 18:29:24 -0000 Do you have VT-d enabled in your bios? A disturbingly large number of servers and workstations come set factory default with virtualization support disabled. -Daniel bRa B writes: > =E2=80=8BHowdy folks, > i have encountered a strange issue when i was trying to run CentOS 6.5 > with bhyve on a server with 11.0-CURRENT FreeBSD Installed: > > VM unrestricted guest capability required > Error in initializing VM > > the machine has 48GB of ram, and the cpu is a i7 920 as stated in the sub= ject of this email.=20 > It has the instructions required to run bhyve ( VMX and POPCNT, as > they appear in the dmesg log and in the Intel's Ark: > http://ark.intel.com/products/37147/Intel-Core-i7-920-Processor-8M-Cache-= 2_66-GHz-4_80-GTs-Intel-QPI?q=3Dcore%20i7-920 > ). > > Is there something wrong with this CPU?=20 > > Thanks > Brando > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Mon Mar 31 18:33:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1BDF143 for ; Mon, 31 Mar 2014 18:33:09 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id A4D9D37D for ; Mon, 31 Mar 2014 18:33:09 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 1A901122B3; Tue, 1 Apr 2014 04:25:09 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTA22826 (AUTH peterg@ptree32.com.au); Tue, 1 Apr 2014 04:25:08 +1000 Message-ID: <5339B302.6000905@freebsd.org> Date: Mon, 31 Mar 2014 11:25:06 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: bRa B Subject: Re: Bhyve on a Core i7-920 References: <1396289325336661500@itaserv.net> In-Reply-To: <1396289325336661500@itaserv.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 18:33:09 -0000 Hi Brando, > ​Howdy folks, i have encountered a strange issue when i was trying to > run CentOS 6.5 with bhyve on a server with 11.0-CURRENT FreeBSD > Installed: > > VM unrestricted guest capability required Error in initializing VM > > the machine has 48GB of ram, and the cpu is a i7 920 as stated in the > subject of this email. It has the instructions required to run bhyve > ( VMX and POPCNT, as they appear in the dmesg log and in the Intel's > Ark: > http://ark.intel.com/products/37147/Intel-Core-i7-920-Processor-8M-Cache-2_66-GHz-4_80-GTs-Intel-QPI?q=core%20i7-920 > ). > > Is there something wrong with this CPU? The 920 aka 'Bloomfield' is a first-generation Nehalem: http://en.wikipedia.org/wiki/Intel_Core#Core_i7 That particular model doesn't have support for 16-bit real mode/32-bit protected/unpaged mode (aka "flat"). This is required for MP support for FreeBSD, or Linux support, since grub starts Linux in 32-bit flat mode. The VT-x feature that enables supports for this is known as 'unrestricted guest', and was introduced in the Westmere microarch. It *may* be possible to get Linux working with grub-bhyve by adding support to start it directly in 64-bit mode, but it would be UP only. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 1 04:05:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAAAF5E4; Tue, 1 Apr 2014 04:05:18 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41EF1EA1; Tue, 1 Apr 2014 04:05:18 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id n15so6451111lbi.41 for ; Mon, 31 Mar 2014 21:05:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=Q2F8ekjXMdKfJ1qNV41exIy5ByxhnkUGauTLr8LnXio=; b=sPi5rAn8Oxj2GXtlHWUsgK4FX9nkUR3cZRxcd7+C6a2963UBHmFKyAqk7fsXp7Emma 6zBJIOWmi7pns8dlwxvUUf26VGCN8frhtyaqzbfDLZyrTsYcaLWGJu0Ii5p6MUDpx07G a61uDI+qympfigcOYc3i7RJ/RcGFZOsLCgnbV+MrlxVwEb1nZVHwhW4SfmYwhob7LjiN l/KjUrNXi/T4/TzhFJ5oxARotOhoxhjF900p3wlKGancKdJiJT+fIFKc7BhzZekoEmyF EfE8SIcGKMpLc7TzfyPhprTC2UeflFIdrBf66yx7tdqhLSpJ00G4SpqWLbj0kmUw5bv6 R0Zg== X-Received: by 10.152.2.131 with SMTP id 3mr21243580lau.20.1396325116042; Mon, 31 Mar 2014 21:05:16 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id t3sm15857122laj.15.2014.03.31.21.05.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 31 Mar 2014 21:05:15 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 1 Apr 2014 08:05:09 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: Bhyve support in Libvirt Message-ID: <20140401040508.GA96324@dev.san.ru> References: <20140331170006.GD1275@kloomba> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 04:05:19 -0000 Craig Rodrigues wrote: > On Mon, Mar 31, 2014 at 10:00 AM, Roman Bogorodskiy w= rote: > > > > http://empt1e.blogspot.ru/2014/03/bhyve-in-libvirt.html > > >=20 > This is a huge step forward for bhyve! Thank you! > There is a sizable ecosystem of software for managing hypervisors built on > top of libvirt. >=20 > Have you tested any of this software, such as virsh? > Are there any GUI or web UI utilities built on top of libvirt > which work with bhyve? virsh is an official client that is a part of libvirt. I use it for development and testing and also it's the main tool I use to manage VMs with libvirt. I know there's a GUI application called virt-manager (deskutils/virt-manager port), but I didn't try that. I'm not aware of web UI applications, but I didn't looks specifically. > You might want to consider submitting a status report at: > https://www.freebsd.org/news/status/status.html I'll take a look, thanks for a suggestion.=20 Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 2 16:26:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7B56D81 for ; Wed, 2 Apr 2014 16:26:40 +0000 (UTC) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8FC3A823 for ; Wed, 2 Apr 2014 16:26:40 +0000 (UTC) Received: from john by potato.growveg.org with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WVNRS-0003GK-Nu for freebsd-virtualization@freebsd.org; Wed, 02 Apr 2014 16:51:06 +0100 Date: Wed, 2 Apr 2014 16:51:06 +0100 From: John To: freebsd-virtualization@freebsd.org Subject: virtualbox and bhyve questions Message-ID: <20140402155106.GA64789@potato.growveg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Sender: John X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: john@potato.growveg.org X-SA-Exim-Scanned: No (on potato.growveg.org); SAEximRunCond expanded to false X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 16:26:40 -0000 Hello virtuals, I'm running stable-10 r262917 on an i7 machine with 192GB RAM and 10Tb zfs. I'd like to run bhyve but I'm struggling a little bit because the latest information is scattered about instead of all in one place. 1. I've read that virtualbox and bhyve cannot run at the same time. Is this still the case? (because I've managed to start a bhyve instance with virtualbox still running - is this a fluke?) 2. Can I take a system made in virtualbox and boot with it using bhyve? The system 'hard drives' are all VDI (the default in virtualbox) 3. if I can't boot it without conversion to something else, how do I go about it? 4. what's the maximum number of cpu and the maximum ram and HD space that I can give to a bhyve VM? 5. if I make a VM and put opensuse (latest) on it, is it preferable to have the HD as raw or as ext-3 (or whatever the default for opensuse is)? If the latter, is there a penalty in terms of performance or security? The host runs zfs. thanks for your input -- John From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 2 18:33:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8D3AB2C for ; Wed, 2 Apr 2014 18:33:41 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7B73D909 for ; Wed, 2 Apr 2014 18:33:41 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 3925B1244E; Thu, 3 Apr 2014 04:33:34 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTB71905 (AUTH peterg@ptree32.com.au); Thu, 3 Apr 2014 04:33:33 +1000 Message-ID: <533C57FB.3000308@freebsd.org> Date: Wed, 02 Apr 2014 11:33:31 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: John , freebsd-virtualization@freebsd.org Subject: Re: virtualbox and bhyve questions References: <20140402155106.GA64789@potato.growveg.org> In-Reply-To: <20140402155106.GA64789@potato.growveg.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 18:33:41 -0000 Hi John, > 1. I've read that virtualbox and bhyve cannot run at the same time. Is > this still the case? (because I've managed to start a bhyve instance > with virtualbox still running - is this a fluke?) If you use VirtualBox with VT-x disabled, i.e. binary translation mode, it will work. Otherwise, all bets are off. There is no way to share VT-x resources (vCPU TLB tags, CPU VMCS state etc) so eventually the hypervisors will collide. > 2. Can I take a system made in virtualbox and boot with it using bhyve? > The system 'hard drives' are all VDI (the default in virtualbox) You would have to convert the VDI file to a flat file with a tool like qemu-img. Depending on the o/s, the disk and network adapter type may have to be the same in vbox and bhyve or init scripts may not be able to locate interface/disk names. Also, the vbox guest should be configured with a serial console. > 3. if I can't boot it without conversion to something else, how do I go > about it? > > 4. what's the maximum number of cpu and the maximum ram and HD space > that I can give to a bhyve VM? bhyve's vCPU limit is 16. There are no limits on HD space. > 5. if I make a VM and put opensuse (latest) on it, is it preferable to > have the HD as raw or as ext-3 (or whatever the default for opensuse > is)? If the latter, is there a penalty in terms of performance or > security? The host runs zfs. Do you mean, the virtual disk type for the guest, something like vmdk/vdi/vhd/raw ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 3 18:29:30 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B2E441A for ; Thu, 3 Apr 2014 18:29:30 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D31D2E4E for ; Thu, 3 Apr 2014 18:29:29 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6FF71B94B for ; Thu, 3 Apr 2014 14:29:28 -0400 (EDT) From: John Baldwin To: virtualization@freebsd.org Subject: [PATCH] Tweaks to vmrun.sh Date: Thu, 3 Apr 2014 14:26:40 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201404031426.40882.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 03 Apr 2014 14:29:28 -0400 (EDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 18:29:30 -0000 I have a few tweaks to vmrun.sh in the patch below: - Format the usage so that it fits in 80 cols and follows the standard convention for long usage lines in manpages. - Sort the option string passed to getopts and the case statements for the option returned by getopts. - Add a -C option to specify the device to be used for the console (defaults to 'stdio') (This could be let vmrun be run in the background by using /dev/nmdm0B or the like) - Add a -H option to specify a host path to pass to bhyveload(8) via -h to back the host0: filesystem in bhyveload(8) (useful for loading kernels from the host into the guest without having to copy them into the guest's disk image first) Index: vmrun.sh =================================================================== --- vmrun.sh (revision 264082) +++ vmrun.sh (working copy) @@ -34,18 +34,25 @@ FBSDRUN=/usr/sbin/bhyve DEFAULT_MEMSIZE=512M DEFAULT_CPUS=2 DEFAULT_TAPDEV=tap0 +DEFAULT_CONSOLE=stdio DEFAULT_VIRTIO_DISK="./diskdev" DEFAULT_ISOFILE="./release.iso" usage() { - echo "Usage: vmrun.sh [-hai][-g ][-m ][-d ][-e ][-I ][-t ] " + echo "Usage: vmrun.sh [-ahi] [-c ] [-C ] [-d ]" + echo " [-e ] [-g ] [-H ]" + echo " [-I ] [-m ]" + echo " [-t ] " + echo "" echo " -h: display this help message" - echo " -a: force memory mapped local apic access" + echo " -a: force memory mapped local APIC access" echo " -c: number of virtual cpus (default is ${DEFAULT_CPUS})" + echo " -C: console device (default is ${DEFAULT_CONSOLE})" echo " -d: virtio diskdev file (default is ${DEFAULT_VIRTIO_DISK})" echo " -e: set FreeBSD loader environment variable" echo " -g: listen for connection from kgdb at " + echo " -H: host filesystem to export to the loader" echo " -i: force boot of the Installation CDROM image" echo " -I: Installation CDROM image location (default is ${DEFAULT_ISOFILE})" echo " -m: memory size (default is ${DEFAULT_MEMSIZE})" @@ -69,29 +76,37 @@ fi force_install=0 isofile=${DEFAULT_ISOFILE} memsize=${DEFAULT_MEMSIZE} +console=${DEFAULT_CONSOLE} cpus=${DEFAULT_CPUS} virtio_diskdev=${DEFAULT_VIRTIO_DISK} tapdev=${DEFAULT_TAPDEV} apic_opt="" gdbport=0 -env_opt="" +loader_opt="" -while getopts haic:e:g:I:m:d:t: c ; do +while getopts ac:C:d:e:g:hH:iI:m:t: c ; do case $c in - h) - usage - ;; a) apic_opt="-a" ;; + c) + cpus=${OPTARG} + ;; + C) + console=${OPTARG} + ;; d) virtio_diskdev=${OPTARG} ;; e) - env_opt="${env_opt} -e ${OPTARG}" + loader_opt="${loader_opt} -e ${OPTARG}" ;; - g) gdbport=${OPTARG} + g) + gdbport=${OPTARG} ;; + H) + host_base=`realpath ${OPTARG}` + ;; i) force_install=1 ;; @@ -98,9 +113,6 @@ gdbport=0 I) isofile=${OPTARG} ;; - c) - cpus=${OPTARG} - ;; m) memsize=${OPTARG} ;; @@ -107,7 +119,7 @@ gdbport=0 t) tapdev=${OPTARG} ;; - \?) + *) usage ;; esac @@ -120,6 +132,9 @@ if [ $# -ne 1 ]; then fi vmname="$1" +if [ -n "${host_base}" ]; then + loader_opt="${loader_opt} -h ${host_base}" +fi # Create the virtio diskdev file if needed if [ ! -f ${virtio_diskdev} ]; then @@ -168,7 +183,8 @@ while [ 1 ]; do installer_opt="" fi - ${LOADER} -m ${memsize} -d ${BOOTDISK} ${env_opt} ${vmname} + ${LOADER} -c ${console} -m ${memsize} -d ${BOOTDISK} ${loader_opt} \ + ${vmname} if [ $? -ne 0 ]; then break fi @@ -179,7 +195,7 @@ while [ 1 ]; do -s 1:0,lpc \ -s 2:0,virtio-net,${tapdev} \ -s 3:0,virtio-blk,${virtio_diskdev} \ - -l com1,stdio \ + -l com1,${console} \ ${installer_opt} \ ${vmname} if [ $? -ne 0 ]; then -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 3 18:57:37 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C98CBA3B for ; Thu, 3 Apr 2014 18:57:37 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AC15712A for ; Thu, 3 Apr 2014 18:57:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id C7FA922834 for ; Thu, 3 Apr 2014 14:56:48 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 21360-05 for ; Thu, 3 Apr 2014 14:56:48 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id B505122831; Thu, 3 Apr 2014 14:56:48 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: KVM virtio filesystem sharing (was Re: (no subject)) X-PHP-Originating-Script: 0:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 03 Apr 2014 14:56:48 -0400 From: Andre Goree In-Reply-To: References: Message-ID: <3d69f89ff092fdaa3317ca5ededcb72b@drenet.net> X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 18:57:37 -0000 On 04/03/2014 2:18 pm, Andre Goree wrote: > Hello list. I wanted to ask if anyone has been able to make > filesystem mounting work under a FreeBSD guest? For example, I've > added the following to the guest's xml config using 'virsh edit': > > > > > > > > However, I don't know how to mount the above from within the FreeBSD > guest. That which is instructed to do so on Linux guests does not > work on the FreeBSD guest: > > root@freebsd9-test:~ # mount -t 9p -o trans=virtio,version=9p2000.L > tag /mnt/shared/ > mount: tag: Operation not supported by device > > The hosts OS is Ubuntu 12.04 LTS. Thanks in advance for any answers on > this. My apologies, just realized I left out the Subject -__- -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 3 19:01:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D67BB00 for ; Thu, 3 Apr 2014 19:01:09 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42E411B7 for ; Thu, 3 Apr 2014 19:01:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id 80ED522833 for ; Thu, 3 Apr 2014 14:50:56 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 21560-02 for ; Thu, 3 Apr 2014 14:50:56 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 5F10922831; Thu, 3 Apr 2014 14:50:56 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: KVM filesystem sharing X-PHP-Originating-Script: 0:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 03 Apr 2014 14:50:56 -0400 From: Andre Goree Message-ID: X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 19:01:09 -0000 Hello list. I wanted to ask if anyone has been able to make filesystem mounting work under a FreeBSD guest? For example, I've added the following to the guest's xml config using 'virsh edit': However, I don't know how to mount the above from within the FreeBSD guest. That which is instructed to do so on Linux guests does not work on the FreeBSD guest: root@freebsd9-test:~ # mount -t 9p -o trans=virtio,version=9p2000.L tag /mnt/shared/ mount: tag: Operation not supported by device The hosts OS is Ubuntu 12.04 LTS. Thanks in advance for any answers on this -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 3 19:14:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5BA27E5 for ; Thu, 3 Apr 2014 19:14:10 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5CB72A6 for ; Thu, 3 Apr 2014 19:14:09 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id w62so2367974wes.28 for ; Thu, 03 Apr 2014 12:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=886m1QL/IT5jPPHBseP2FPp5Zk74YDGtJzd9uPsflsU=; b=MCVbI+ioI6nnmNNShr5El4tclxWeweJykUcyk7aWMuExTn+QAjSjDaklTtXTjPj4gA pOCYxbjcb07qstehyKPuDorc32MJuXePZXboBVhUQ+nPFjIv7obdKJ8fGvSgWQLCffxM x1P8wA3Sh5e+cet6CEgJuolzFvO4A0BZdEoq8vfQvAXT0/K8vGSWREtP7utXAXuRGvpk 1NpucvOSdV5EdEPje3ThvBZZqj0E7gYGgKaXPcndxbgbbSqlLPObRgg1BupL++3tFUzY 1sAFYoNIRyH24NxHJlwnPXw6CUZUYPOLitD08CoCplEudv/fJ08NC7NaLbTgiXRRSQa3 /oVQ== X-Received: by 10.194.89.40 with SMTP id bl8mr8822867wjb.90.1396552448254; Thu, 03 Apr 2014 12:14:08 -0700 (PDT) Received: from strashydlo.home (abwv188.neoplus.adsl.tpnet.pl. [83.8.245.188]) by mx.google.com with ESMTPSA id g3sm14018487eet.35.2014.04.03.12.14.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 03 Apr 2014 12:14:07 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: KVM filesystem sharing Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= In-Reply-To: Date: Thu, 3 Apr 2014 21:14:06 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7D9BADB3-3A53-4C8C-B86B-12F3C18F9235@FreeBSD.org> References: To: Andre Goree X-Mailer: Apple Mail (2.1283) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 19:14:10 -0000 Wiadomo=B6=E6 napisana przez Andre Goree w dniu 3 kwi 2014, o godz. = 20:50: > Hello list. I wanted to ask if anyone has been able to make = filesystem mounting work under a FreeBSD guest? For example, I've added = the following to the guest's xml config using 'virsh edit': >=20 > > > > >=20 >=20 > However, I don't know how to mount the above from within the FreeBSD = guest. That which is instructed to do so on Linux guests does not work = on the FreeBSD guest: >=20 > root@freebsd9-test:~ # mount -t 9p -o trans=3Dvirtio,version=3D9p2000.L = tag /mnt/shared/ > mount: tag: Operation not supported by device The 9P protocol is not supported under FreeBSD. Perhaps you could try to use NFS instead? Generally speaking, why do Linux folks use 9P? From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 3 19:36:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1FC97E8; Thu, 3 Apr 2014 19:36:42 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B443C66A; Thu, 3 Apr 2014 19:36:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id 7466D22833; Thu, 3 Apr 2014 15:35:58 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 21360-09; Thu, 3 Apr 2014 15:35:58 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 5904A22831; Thu, 3 Apr 2014 15:35:58 -0400 (EDT) To: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: KVM filesystem sharing X-PHP-Originating-Script: 0:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Thu, 03 Apr 2014 15:35:58 -0400 From: Andre Goree In-Reply-To: <7D9BADB3-3A53-4C8C-B86B-12F3C18F9235@FreeBSD.org> References: <7D9BADB3-3A53-4C8C-B86B-12F3C18F9235@FreeBSD.org> Message-ID: <26931f62daacb23daff55f1fb8c8a34a@drenet.net> X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 Cc: =?UTF-8?Q?Edward_Tomasz_Napiera?= =?UTF-8?Q?=C5=82a?= , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 19:36:42 -0000 On 04/03/2014 3:14 pm, Edward Tomasz Napierała wrote: > Wiadomość napisana przez Andre Goree w dniu 3 kwi 2014, o godz. 20:50: >> Hello list. I wanted to ask if anyone has been able to make >> filesystem mounting work under a FreeBSD guest? For example, I've >> added the following to the guest's xml config using 'virsh edit': >> >> >> >> >> >> >> >> However, I don't know how to mount the above from within the FreeBSD >> guest. That which is instructed to do so on Linux guests does not >> work on the FreeBSD guest: >> >> root@freebsd9-test:~ # mount -t 9p -o trans=virtio,version=9p2000.L >> tag /mnt/shared/ >> mount: tag: Operation not supported by device > > The 9P protocol is not supported under FreeBSD. Perhaps you could try > to use NFS instead? > > Generally speaking, why do Linux folks use 9P? Thanks! I'm thinking I may use guestfish instead to accomplish what it is I was wanting to do. I've never personally used 9P, but thought it might be a way I could use to get around a problem I've run into. Thanks again for the info. -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 7 11:06:53 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C04ABBEE for ; Mon, 7 Apr 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AB453C18 for ; Mon, 7 Apr 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s37B6rrL071248 for ; Mon, 7 Apr 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s37B6rw7071246 for freebsd-virtualization@FreeBSD.org; Mon, 7 Apr 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 7 Apr 2014 11:06:53 GMT Message-Id: <201404071106.s37B6rw7071246@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 7 16:52:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82427709 for ; Mon, 7 Apr 2014 16:52:41 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AEDA8A1 for ; Mon, 7 Apr 2014 16:52:40 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id hm4so5432291wib.2 for ; Mon, 07 Apr 2014 09:52:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=AWX3Tbq+dStn6Ua62CCNvpX2TJ7yMFgPCzrXG8nttkY=; b=sO8kIlaIg0Eqh2Vb698xGC1RjMccDzzgcte+80kdUh5QQhN+iWL4wSxO9IclhQJrnP JcZyDdqOLjet/y9Gb2d1NKxGfpY5/VQh+qBlVKuPhZx0st7pVDuHet49knbNaYcFZwpy VIwXVs3zrbMX9VlbANSfmVGwb96IoqYwsfgdi3tdIvlNQgduETM5AmcxW+4Jj6MuwqiM +d5tE66CkLfcnfy4t4CvDSSS+xTn4es5/S5EnfPe3yEOEknOh/u81mx9ma70EodR1z/p sHZlUKSFrC4vLISDqfvKSUY8f0j0LeQzN7rao5lHUjOl6eFPrxFFf2zNRwhKelmt48De cqSg== X-Received: by 10.195.12.33 with SMTP id en1mr10011920wjd.31.1396889559454; Mon, 07 Apr 2014 09:52:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.217.55.138 with HTTP; Mon, 7 Apr 2014 09:52:19 -0700 (PDT) From: Miguel Clara Date: Mon, 7 Apr 2014 17:52:19 +0100 Message-ID: Subject: FreeBSD 10 XENHVM panic under NetBSD Dom0 (xn_txeof: WARNING: response is -1) To: "freebsd-virtualization@freebsd.org" , "port-xen@netbsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 16:52:41 -0000 Hi, I've reported this to xen-devel, but it seems not much people use NetBSD as a Dom0 and I was wondering if anyone is? I'm getting a panic and an error message (xn_txeof: WARNING: response is -1) when I try to use ssh (the error also show for other network operation but not the panic) Here goes the panic I get after I try ssh: xn_txeof: WARNING: response is -1! panic: mbuf already on the free list, but we're trying to free it again! cpuid = 0 KDB: enter: panic [ thread pid 12 tid 100026 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why db> bt Tracing pid 12 tid 100026 td 0xfffffe000be79920 kdb_enter() at kdb_enter+0x3e/frame 0xffffff80002fe8c0 vpanic() at vpanic+0x146/frame 0xffffff80002fe900 kassert_panic() at kassert_panic+0x136/frame 0xffffff80002fe970 xn_txeof() at xn_txeof+0x99/frame 0xffffff80002fe9c0 xn_intr() at xn_intr+0x59/frame 0xffffff80002feab0 evtchn_interrupt() at evtchn_interrupt+0x1e6/frame 0xffffff80002feb30 intr_event_execute_handlers() at intr_event_execute_handlers+0x90/frame 0xffffff80002feb70 ithread_loop() at ithread_loop+0x148/frame 0xffffff80002febb0 fork_exit() at fork_exit+0x84/frame 0xffffff80002febf0 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002febf0 --- trap 0, rip = 0, rsp = 0xffffff80002fecb0, rbp = 0 --- I've tried a few revisions and actually revert way back to the last one that was working on 9.x which in stable/10 would be Revision 251297 but no luck. So probably the issue is somewhere else then netfront, but I have no idea where to look next. If anyone has a netbsd dom0 that could test this would be much appreciated... I'm using NetBSD 6.1.2 btw From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 18:28:54 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEDA5AB7 for ; Tue, 8 Apr 2014 18:28:54 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AEFA2125E for ; Tue, 8 Apr 2014 18:28:54 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s38ISju0089219 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 8 Apr 2014 18:28:50 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s38ISj9a089218 for freebsd-virtualization@FreeBSD.org; Tue, 8 Apr 2014 18:28:45 GMT (envelope-from swills) Date: Tue, 8 Apr 2014 18:28:45 +0000 From: Steve Wills To: freebsd-virtualization@FreeBSD.org Subject: bhyve VM crashed Message-ID: <20140408182844.GC84921@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Tue, 08 Apr 2014 18:28:50 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 18:28:55 -0000 Hi, I experienced a bhyve VM crash with this message: vm exit[1] reason VMX rip 0xffffffff80c7490b inst_length 3 status 0 exit_reason 2 qualification 0x0000000000000000 inst_type 0 inst_error 0 Not sure what that means, anyone have any ideas? This VM is running FreeBSD 10.0-RELEASE, 64bit. Let me know what other info would be helpful. Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 18:39:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F461E0F; Tue, 8 Apr 2014 18:39:02 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C01411343; Tue, 8 Apr 2014 18:39:01 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id il7so1134051vcb.18 for ; Tue, 08 Apr 2014 11:39:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nGr79olcOvjTc4E+x0ZV7x2rJLXTfqZf+fY734eiWdU=; b=JisnauVURfw9iy878yeCrja+X1cUt1IUo0FG+BJZEoRPOVfgFrJtH9FL9vUIJ1kECa UjmubUOfXthdquVyrzkpzpWoawXSeHc/JNqwRiBbdnQJ9heXWcz4rDvtgo1TtfJxr9Hv re5rVSzN0tv7/o09VN764Wwd9sqirWAyay4EuahXqPqGo43xAP7+thPBWs09U3PO+7cK Jh0+AfoSTzgluRm1rzPcqWZlul6H7JKyg+1tS8M7z2enb5I+TNoIxkPbTAU1n8H7wRHW /5vZ28W4Nfqirym6zgoHYaJ3gPS5p1XWnwDR5d8LKQrqS+wff8qYex5+VUnO/dX0Ay2Q Muvg== MIME-Version: 1.0 X-Received: by 10.58.230.103 with SMTP id sx7mr1601476vec.28.1396982340858; Tue, 08 Apr 2014 11:39:00 -0700 (PDT) Received: by 10.220.193.200 with HTTP; Tue, 8 Apr 2014 11:39:00 -0700 (PDT) In-Reply-To: <20140408182844.GC84921@mouf.net> References: <20140408182844.GC84921@mouf.net> Date: Tue, 8 Apr 2014 14:39:00 -0400 Message-ID: Subject: Re: bhyve VM crashed From: Aryeh Friedman To: Steve Wills Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 18:39:02 -0000 What was the guest OS? On Tue, Apr 8, 2014 at 2:28 PM, Steve Wills wrote: > Hi, > > I experienced a bhyve VM crash with this message: > > vm exit[1] > reason VMX > rip 0xffffffff80c7490b > inst_length 3 > status 0 > exit_reason 2 > qualification 0x0000000000000000 > inst_type 0 > inst_error 0 > > Not sure what that means, anyone have any ideas? This VM is running FreeBSD > 10.0-RELEASE, 64bit. Let me know what other info would be helpful. > > Thanks, > Steve > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 18:43:32 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 958EC263; Tue, 8 Apr 2014 18:43:32 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 58439144D; Tue, 8 Apr 2014 18:43:32 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D8A0A1236F; Wed, 9 Apr 2014 04:43:30 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-69-181-164-196.hsd1.ca.comcast.net [69.181.164.196]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTF21543 (AUTH peterg@ptree32.com.au); Wed, 9 Apr 2014 04:43:30 +1000 Message-ID: <5344434F.6000004@freebsd.org> Date: Tue, 08 Apr 2014 11:43:27 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steve Wills Subject: Re: bhyve VM crashed References: <20140408182844.GC84921@mouf.net> In-Reply-To: <20140408182844.GC84921@mouf.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 18:43:32 -0000 Hi Steve, > I experienced a bhyve VM crash with this message: > > vm exit[1] > reason VMX > rip 0xffffffff80c7490b > inst_length 3 > status 0 > exit_reason 2 That's a triple-fault :( (the exit reasons are the VMX ones in Appendix C, Vol 3 of the Intel SDM). Any idea what was happening at the time ? Also, for the guest, do you have the config that was being used (e.g. bhyve command line) ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 19:17:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61A0C5B6 for ; Tue, 8 Apr 2014 19:17:41 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E0AC1841 for ; Tue, 8 Apr 2014 19:17:41 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s38JHU4w089956 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 8 Apr 2014 19:17:35 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s38JHUNT089955; Tue, 8 Apr 2014 19:17:30 GMT (envelope-from swills) Date: Tue, 8 Apr 2014 19:17:30 +0000 From: Steve Wills To: Aryeh Friedman Subject: Re: bhyve VM crashed Message-ID: <20140408191729.GA89356@mouf.net> References: <20140408182844.GC84921@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Tue, 08 Apr 2014 19:17:36 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 19:17:41 -0000 The host is running FreeBSD 11.0-CURRENT r264056. The guest is running FreeBSD 10.0-RELEASE r260789. Steve On Tue, Apr 08, 2014 at 02:39:00PM -0400, Aryeh Friedman wrote: > What was the guest OS? > > > On Tue, Apr 8, 2014 at 2:28 PM, Steve Wills wrote: > > > Hi, > > > > I experienced a bhyve VM crash with this message: > > > > vm exit[1] > > reason VMX > > rip 0xffffffff80c7490b > > inst_length 3 > > status 0 > > exit_reason 2 > > qualification 0x0000000000000000 > > inst_type 0 > > inst_error 0 > > > > Not sure what that means, anyone have any ideas? This VM is running FreeBSD > > 10.0-RELEASE, 64bit. Let me know what other info would be helpful. > > > > Thanks, > > Steve > > > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 19:26:45 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AAE1A6F; Tue, 8 Apr 2014 19:26:45 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CD7D31955; Tue, 8 Apr 2014 19:26:44 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s38JQZ1e090120 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 8 Apr 2014 19:26:40 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s38JQZNf090119; Tue, 8 Apr 2014 19:26:35 GMT (envelope-from swills) Date: Tue, 8 Apr 2014 19:26:35 +0000 From: Steve Wills To: Peter Grehan Subject: Re: bhyve VM crashed Message-ID: <20140408192635.GB89356@mouf.net> References: <20140408182844.GC84921@mouf.net> <5344434F.6000004@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5344434F.6000004@freebsd.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Tue, 08 Apr 2014 19:26:40 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 19:26:45 -0000 On Tue, Apr 08, 2014 at 11:43:27AM -0700, Peter Grehan wrote: > Hi Steve, > > > I experienced a bhyve VM crash with this message: > > > > vm exit[1] > > reason VMX > > rip 0xffffffff80c7490b > > inst_length 3 > > status 0 > > exit_reason 2 > > That's a triple-fault :( > > (the exit reasons are the VMX ones in Appendix C, Vol 3 of the Intel SDM). > > Any idea what was happening at the time ? The VM was building ports. I could try to deduce which ports, but I wouldn't be 100% sure. > Also, for the guest, do you > have the config that was being used (e.g. bhyve command line) ? /usr/sbin/bhyve -c 4 -m 12G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap1,mac=58:9c:fc:01:01:02 -s 3:0,virtio-blk,/vm/100amd64/disk.img -l com1,/dev/nmdm1A 100amd64 Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 19:55:17 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EC123F4 for ; Tue, 8 Apr 2014 19:55:17 +0000 (UTC) Received: from mail-ig0-x232.google.com (mail-ig0-x232.google.com [IPv6:2607:f8b0:4001:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13B361C23 for ; Tue, 8 Apr 2014 19:55:16 +0000 (UTC) Received: by mail-ig0-f178.google.com with SMTP id hn18so1521617igb.11 for ; Tue, 08 Apr 2014 12:55:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=+T1Liomt/aj09CkjbmmQlFSWv7ePW/ujtXmr2/OmOiI=; b=s+B9GLAzzVuZtEtSYl1AlphrMnOaDXaaLe/pBP4+ZAzT013sVm4PB7JxCkCBbVq9xH yc04dEo7Rd8VmZAtoTQz6VWJG0kvlxx/UvO/3rjiDOp/wMEkAJjXelztT4ro7R9eOg7Q F8K2QZIl1edvHHxUaR1CEUHRiUmPxHYqApl2o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=+T1Liomt/aj09CkjbmmQlFSWv7ePW/ujtXmr2/OmOiI=; b=hs44Lj/hgMGKwoOcgJwLgATmF+zJGHleFZjKbuO5uqnxKQzUx8ZcaxGLpwe1tKY7Kk aIVXV+k1yaufGfi1d4mRbliMPq3i9i6p2OnKuS66JyswAqa60QiZEGaoivVVfSaa9xLq +OZF76q+psUWnpC5ZkMNJ5OdoUZcYgFnNZWR6rLICGHJSwxDjEMjHMbDpLxigsMGXvFH KDV7NGZCDDX04nT0kAocztCkZz6KJ3JabdYHFhQCy8TVq7UtUNqn0N+BweqCq+03Emjc JsIYNmwgSwLaAoaU/K9NyBPty1OH9inEWjW5Xgv98jjK9/6a0AX8zF4qMuii7P0rI9D2 fS1A== X-Gm-Message-State: ALoCoQkYwhDgi0v4cdtloN+FqlUtF50DUu++l0meQSfe0fFptFxHhcZ+V8DgpLkIrYzNTU4NeXrb X-Received: by 10.50.47.12 with SMTP id z12mr529198igm.37.1396986916111; Tue, 08 Apr 2014 12:55:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.181.168 with HTTP; Tue, 8 Apr 2014 12:54:36 -0700 (PDT) In-Reply-To: <20140408192635.GB89356@mouf.net> References: <20140408182844.GC84921@mouf.net> <5344434F.6000004@freebsd.org> <20140408192635.GB89356@mouf.net> From: Takuya ASADA Date: Wed, 9 Apr 2014 04:54:36 +0900 Message-ID: Subject: Re: bhyve VM crashed To: Steve Wills Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 19:55:17 -0000 Did you execute bhyveload before bhyve? 2014-04-09 4:26 GMT+09:00 Steve Wills : > On Tue, Apr 08, 2014 at 11:43:27AM -0700, Peter Grehan wrote: > > Hi Steve, > > > > > I experienced a bhyve VM crash with this message: > > > > > > vm exit[1] > > > reason VMX > > > rip 0xffffffff80c7490b > > > inst_length 3 > > > status 0 > > > exit_reason 2 > > > > That's a triple-fault :( > > > > (the exit reasons are the VMX ones in Appendix C, Vol 3 of the Intel > SDM). > > > > Any idea what was happening at the time ? > > The VM was building ports. I could try to deduce which ports, but I > wouldn't be > 100% sure. > > > Also, for the guest, do you > > have the config that was being used (e.g. bhyve command line) ? > > /usr/sbin/bhyve -c 4 -m 12G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc -s > 2:0,virtio-net,tap1,mac=58:9c:fc:01:01:02 -s > 3:0,virtio-blk,/vm/100amd64/disk.img -l com1,/dev/nmdm1A 100amd64 > > Thanks, > Steve > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 20:37:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 663EE1C4; Tue, 8 Apr 2014 20:37:40 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 091AE114D; Tue, 8 Apr 2014 20:37:39 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s38KbNsV091230 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 8 Apr 2014 20:37:28 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s38KbNuJ091229; Tue, 8 Apr 2014 20:37:23 GMT (envelope-from swills) Date: Tue, 8 Apr 2014 20:37:23 +0000 From: Steve Wills To: Takuya ASADA Subject: Re: bhyve VM crashed Message-ID: <20140408203722.GC89356@mouf.net> References: <20140408182844.GC84921@mouf.net> <5344434F.6000004@freebsd.org> <20140408192635.GB89356@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Tue, 08 Apr 2014 20:37:28 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 20:37:40 -0000 Yep, /usr/sbin/bhyveload -m 12G -d /vm/100amd64/disk.img -c /dev/nmdm1A 100amd64 Steve On Wed, Apr 09, 2014 at 04:54:36AM +0900, Takuya ASADA wrote: > Did you execute bhyveload before bhyve? > > > 2014-04-09 4:26 GMT+09:00 Steve Wills : > > > On Tue, Apr 08, 2014 at 11:43:27AM -0700, Peter Grehan wrote: > > > Hi Steve, > > > > > > > I experienced a bhyve VM crash with this message: > > > > > > > > vm exit[1] > > > > reason VMX > > > > rip 0xffffffff80c7490b > > > > inst_length 3 > > > > status 0 > > > > exit_reason 2 > > > > > > That's a triple-fault :( > > > > > > (the exit reasons are the VMX ones in Appendix C, Vol 3 of the Intel > > SDM). > > > > > > Any idea what was happening at the time ? > > > > The VM was building ports. I could try to deduce which ports, but I > > wouldn't be > > 100% sure. > > > > > Also, for the guest, do you > > > have the config that was being used (e.g. bhyve command line) ? > > > > /usr/sbin/bhyve -c 4 -m 12G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc -s > > 2:0,virtio-net,tap1,mac=58:9c:fc:01:01:02 -s > > 3:0,virtio-blk,/vm/100amd64/disk.img -l com1,/dev/nmdm1A 100amd64 > > > > Thanks, > > Steve > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org" > > From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 8 21:14:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F0EAAD8; Tue, 8 Apr 2014 21:14:38 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id D4B971635; Tue, 8 Apr 2014 21:14:37 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 260A71234B; Wed, 9 Apr 2014 07:14:36 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTF24710 (AUTH peterg@ptree32.com.au); Wed, 9 Apr 2014 07:14:35 +1000 Message-ID: <534466B8.9060708@freebsd.org> Date: Tue, 08 Apr 2014 14:14:32 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steve Wills Subject: Re: bhyve VM crashed References: <20140408182844.GC84921@mouf.net> <5344434F.6000004@freebsd.org> <20140408192635.GB89356@mouf.net> In-Reply-To: <20140408192635.GB89356@mouf.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 21:14:38 -0000 Hi Steve, > The VM was building ports. I could try to deduce which ports, but I > wouldn't be 100% sure. Probably no need, unless this can be repro'd by building that particular port. Some more questions: Were there other VMs running at the same time ? What's the type of host CPU/number of CPUs/amount of RAM ? Did the VM have much uptime before the crash ? Have you been able to run similar VMs on prior CURRENT releases without this issue (i.e. have we introduced a regression) ? >> Also, for the guest, do you have the config that was being used >> (e.g. bhyve command line) ? > > /usr/sbin/bhyve -c 4 -m 12G -A -H -P -g 0 -s 0:0,hostbridge -s > 1:0,lpc -s 2:0,virtio-net,tap1,mac=58:9c:fc:01:01:02 -s > 3:0,virtio-blk,/vm/100amd64/disk.img -l com1,/dev/nmdm1A 100amd64 Thanks for that. If you see this again, would you be able to grab VM state for all the vcpus with bhyvectl --get-all --cpu=0 --vm=100amd64 bhyvectl --get-all --cpu=1 --vm=100amd64 bhyvectl --get-all --cpu=2 --vm=100amd64 bhyvectl --get-all --cpu=3 --vm=100amd64 later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 9 00:14:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D73EAC41; Wed, 9 Apr 2014 00:14:14 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 79F281B0C; Wed, 9 Apr 2014 00:14:14 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s390DwVC094163 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Apr 2014 00:14:03 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s390DwJD094162; Wed, 9 Apr 2014 00:13:58 GMT (envelope-from swills) Date: Wed, 9 Apr 2014 00:13:58 +0000 From: Steve Wills To: Peter Grehan Subject: Re: bhyve VM crashed Message-ID: <20140409001355.GE89356@mouf.net> References: <20140408182844.GC84921@mouf.net> <5344434F.6000004@freebsd.org> <20140408192635.GB89356@mouf.net> <534466B8.9060708@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <534466B8.9060708@freebsd.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Wed, 09 Apr 2014 00:14:03 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 00:14:14 -0000 On Tue, Apr 08, 2014 at 02:14:32PM -0700, Peter Grehan wrote: > Hi Steve, > > > The VM was building ports. I could try to deduce which ports, but I > > wouldn't be 100% sure. > > Probably no need, unless this can be repro'd by building that > particular port. Ok, I'll keep an eye out for that. > Some more questions: Were there other VMs running at the same time ? Yes, there's currently a 9.2 amd64 guest as well, which was also running then and is still running. > What's the type of host CPU/number of CPUs/amount of RAM ? Did the VM hw.model: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz hw.ncpu: 32 hw.physmem: 68644569088 > have much uptime before the crash ? Have you been able to run similar Was up maybe 4 days, not 100% sure, but I think that's right. > VMs on prior CURRENT releases without this issue (i.e. have we > introduced a regression) ? Don't know, just started doing this. > >> Also, for the guest, do you have the config that was being used > >> (e.g. bhyve command line) ? > > > > /usr/sbin/bhyve -c 4 -m 12G -A -H -P -g 0 -s 0:0,hostbridge -s > > 1:0,lpc -s 2:0,virtio-net,tap1,mac=58:9c:fc:01:01:02 -s > > 3:0,virtio-blk,/vm/100amd64/disk.img -l com1,/dev/nmdm1A 100amd64 > > Thanks for that. > > If you see this again, would you be able to grab VM state for all the > vcpus with > > bhyvectl --get-all --cpu=0 --vm=100amd64 > bhyvectl --get-all --cpu=1 --vm=100amd64 > bhyvectl --get-all --cpu=2 --vm=100amd64 > bhyvectl --get-all --cpu=3 --vm=100amd64 Will do, thanks! Steve From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 9 17:09:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF402DEE; Wed, 9 Apr 2014 17:09:02 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0411B136E; Wed, 9 Apr 2014 17:09:01 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id pv20so1367625lab.24 for ; Wed, 09 Apr 2014 10:09:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=eoOeHE5RUHz2MrjFv4RxfZbhKtWVwlRMncbBewsfe+Q=; b=Z6o13qH4V1Rwkqz13lfggxGX3cuEM4dnIIXBFEnVRDRUI0UBOTYEaESqpHdZc58fsM uL0xY3UI+ajyBBhd00A35k6lFszUKd21BJHZUP9CwU9yiRSitur5MOg0nBnV8xXyu7Qh I/YrnhtjipH17DEVjLrDZBNN92kuhVbm/syWTShv37jcYOwdJ6fiwm7wQLDUw+nUOShS djpOzJ60SAh7IDHtp6oWHnh5mQfKPvVnnyw63WZ1OmSNI1JadDAGx7+F0nkOA2TahS4k rBQZAUu0ZsNk9gw2BQ/SvfUkDhZDt7ehwXUOwh+kBa3AbgVd7lWmguUiDUBEg0z3Oz1z Mo7Q== X-Received: by 10.112.106.40 with SMTP id gr8mr7855449lbb.0.1397063339981; Wed, 09 Apr 2014 10:08:59 -0700 (PDT) Received: from kloomba ([95.104.139.23]) by mx.google.com with ESMTPSA id wm1sm1530697lac.14.2014.04.09.10.08.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Apr 2014 10:08:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 9 Apr 2014 21:08:49 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: [SOLVED] Re: lazy bhyve doesn't boot until I conenct to console Message-ID: <20140409170847.GA1313@kloomba> References: <5310346B.4000001@monkeybrains.net> <531035FD.4030704@freebsd.org> <53103E8E.80709@monkeybrains.net> <53196617.4090706@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pWyiEgJYm5f9v55/" Content-Disposition: inline In-Reply-To: <53196617.4090706@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@freebsd.org, "Rudy \(bulk\)" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:09:02 -0000 --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > >>> I'm seeing something odd... I am running this command: > >>> > >>> bhyve -c 4 -m 2048M -H -P -A -s 0:0,hostbridge -s 1:0,lpc -s > >>> 2,virtio-net,tap1 -s 3,virtio-blk,/dev/zvol/data/vm1 -l com1,/dev/nmd= m1A > >>> vm1 > >>> but I the VM is being lazy... > >> > >> A number of people have reported this, and looking at it, I think > >> it's a bug in bhyve. >=20 > Fixed in r262884 Hi Peter, Do you plan to MFC that? The original commit set MFC to 3 weeks and now more than 4 weeks passed since. That would be a very useful fix to have in STABLE. Thanks, Roman Bogorodskiy --pWyiEgJYm5f9v55/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTRX6fAAoJEMltX/4IwiJqh8UIALh7TModVEd1wALeT1M0h+Mr uWbkQlCJWyeZdPP3nbRNjQ0ofAryXGaWLlZ6qbQBIc+Jr1w2KQ8FYF8kPryFdvzI 24JcReaBzmq3ffeyWvylD/MwSlxDJpkQoC6TYIIZ2kuSYT2jNBO+ReESt7YmwRvR X34XKTai++m1s3J/qAVXhZ9Zmr18zydZq5M1kq4sFW9CSkXKhEO4NlTCVSJ7HsoY M5ayEZZFg1vxQSxJp/NCt00GZFWoOnn/3L9t9EwJsRGSMy7ll1ph7PUKZkCHrPXr qf2r3Xqg93it4QHI4XGpJSMu52+V1sEKJlZywulJnB3TBAe/IRNa3vE2InBU41Y= =NhCP -----END PGP SIGNATURE----- --pWyiEgJYm5f9v55/-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 9 17:23:01 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD51A53F; Wed, 9 Apr 2014 17:23:01 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7D3561568; Wed, 9 Apr 2014 17:23:01 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 8E98612494; Thu, 10 Apr 2014 03:22:59 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([198.95.226.236]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTF94024 (AUTH peterg@ptree32.com.au); Thu, 10 Apr 2014 03:22:58 +1000 Message-ID: <534581D4.6060302@freebsd.org> Date: Wed, 09 Apr 2014 10:22:28 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: [SOLVED] Re: lazy bhyve doesn't boot until I conenct to console References: <5310346B.4000001@monkeybrains.net> <531035FD.4030704@freebsd.org> <53103E8E.80709@monkeybrains.net> <53196617.4090706@freebsd.org> <20140409170847.GA1313@kloomba> In-Reply-To: <20140409170847.GA1313@kloomba> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org, "Rudy \(bulk\)" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:23:01 -0000 Hi Roman, >> Fixed in r262884 ... > Do you plan to MFC that? The original commit set MFC to 3 weeks and now > more than 4 weeks passed since. That would be a very useful fix to have > in STABLE. Yes - there should be a large set of MFCs soon; just getting together the list of what will be done. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 9 17:27:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24177984; Wed, 9 Apr 2014 17:27:09 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FABA15A4; Wed, 9 Apr 2014 17:27:08 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so1412896lbj.31 for ; Wed, 09 Apr 2014 10:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=2/tioAsPxgwZjujwitsUQ0eMWpfzJbNv0hql95knwAk=; b=uLSK10xKtJXV77JtsZCx/9i5xKjMLCXf6BAapzTMyOjANpfsko7ngsQBxNUHYRzd5D ZBLwgDHGMrDnKN/fTeD1d4xAn00YiayoRxozv1C8Mb4iV2irIUBvhA1JAtNIcZKtbFb0 pMcng0yEm44PQUk7OCH0SRSVrgeD6DqE6906vBqnaIyn3IPiURw/LZhGlbHXarI4m58j EdXsGUMIc+0pkthyqUdjFlj7YmW99RZRxTOr9lMbZRK9B248Nrqb/nGUU7V43yH/PaF2 yzjsCiXDgFEfmmMvealFhQwoLeSdzxzLVRy6DBEYccbh62gy+zjlrXg9PqRykxfkCv+v sWhg== X-Received: by 10.112.163.69 with SMTP id yg5mr7862764lbb.14.1397064426323; Wed, 09 Apr 2014 10:27:06 -0700 (PDT) Received: from kloomba ([95.104.139.23]) by mx.google.com with ESMTPSA id d4sm1297444lbr.27.2014.04.09.10.27.04 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Apr 2014 10:27:05 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 9 Apr 2014 21:26:59 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: bhyve: allow specifiying tty by fd Message-ID: <20140409172657.GB1313@kloomba> References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f2QGlHpHGjS2mn6Y" Content-Disposition: inline In-Reply-To: <5318BF7F.4060306@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:27:09 -0000 --f2QGlHpHGjS2mn6Y Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > Hi Roman, > > Currently bhyve(8) allows to specify either stdio or device path as a > > TTY device. However, it could be useful to specify a TTY device by file > > descriptor, e.g. when bhyve is being executed in an automated way by > > other application, so a result of openpty(3) could be passed to it. > > > > Attached a poc patch for that. It allows to specify fd this way: > > > > bhyve -s 31,lpc -l com1,fd=3D19 vm0 >=20 > Yes, useful idea, and will become more relevant in the capsicum world= =20 > of passing fd's to a sandboxed bhyve. >=20 > Thanks: I'll get this submitted. Hi Peter! Any news on that? Thanks, Roman Bogorodskiy --f2QGlHpHGjS2mn6Y Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTRYLhAAoJEMltX/4IwiJq6AoIAK+OHJ8unohLc+5PW9fINKDK rfwMGYQnUR/fOTO68m05XGIIKkgap0EHvsNY0PS3DAtfi3646BTHC2vRV2LBX6Dx VOLZ4lZeubLcttzIZsZnXBDN8hPrNJAUY6kFPtAxKaDLCOetGplEVd3nM8CuTILj vPfnvlEW/5ULl+xnLCmGUMLco0qWepL8Vud2iskLEJ9vU8ZC7kUh3u8YgueNQDEC qZjUMW0sRdsibcjG/nwVyqzo+rIIwg+P+X0nWipryE4j3wxYaktWuJsyVUJJmQi9 9ZWznnCxhqELZ3GQN6ewkK4/mlgYIvbHdQQvT75q+ubC30WkMewzbkuKqyqeFtg= =kZIP -----END PGP SIGNATURE----- --f2QGlHpHGjS2mn6Y-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 9 17:36:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2727FE37; Wed, 9 Apr 2014 17:36:06 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id DC1B016AE; Wed, 9 Apr 2014 17:36:05 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 2873A12496; Thu, 10 Apr 2014 03:36:04 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([198.95.226.236]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTF94220 (AUTH peterg@ptree32.com.au); Thu, 10 Apr 2014 03:36:03 +1000 Message-ID: <534584FE.60807@freebsd.org> Date: Wed, 09 Apr 2014 10:35:58 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: bhyve: allow specifiying tty by fd References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> <20140409172657.GB1313@kloomba> In-Reply-To: <20140409172657.GB1313@kloomba> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:36:06 -0000 Hi Roman, >>> bhyve -s 31,lpc -l com1,fd=19 vm0 >> >> Yes, useful idea, and will become more relevant in the capsicum world >> of passing fd's to a sandboxed bhyve. >> >> Thanks: I'll get this submitted. > > Hi Peter! > > Any news on that? I started to do it, and then hesitated bit for two reasons. First, it's useful for all backends, but there still needs to be a type - file descriptors can't be treated as something opaque: there still needs to be knowledge of whether it's a socket, char device, file etc, so the option may need additional information to describe that. Also, given that Capsicum wants to pass fd's as parameters, I was wondering if there is a general syntax style for doing this. "fd=%d" seems Ok, though it would be useful if there was something in libutil to parse an option like this if it is the preferred way forward for FreeBSD executables. What do folk think ? Any useful precedents ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 10 18:09:43 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 817E8462 for ; Thu, 10 Apr 2014 18:09:43 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4311B1192 for ; Thu, 10 Apr 2014 18:09:40 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s3AI9U1v035389 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 10 Apr 2014 18:09:35 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s3AI9U7x035388 for virtualization@FreeBSD.org; Thu, 10 Apr 2014 18:09:30 GMT (envelope-from swills) Date: Thu, 10 Apr 2014 18:09:30 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: installing FreeBSD 8.4 in bhyve Message-ID: <20140410180929.GB34413@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Thu, 10 Apr 2014 18:09:35 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 18:09:43 -0000 Hi, I'm trying to install FreeBSD 8.4 in bhyve, but running into some issues. Sysinstall doesn't seem to like the disk. In the partition editor, I get: Disk name: vtbd1 FDISK Partition Editor DISK Geometry: 0 cyls/0 heads/0 sectors = 0 sectors (0MB) Offset Size(ST) End Name PType Desc Subtype Flags 0 209715200 209715199 - 12 unused 0 Any ideas what I'm doing wrong? This is using the 8.4 Release disc1 iso on 11-CURRENT r264056. I'm using virtio-blk for both the cd and hd. Should this be working? Steve From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 10 18:32:51 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F4F5DD2; Thu, 10 Apr 2014 18:32:51 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB312147D; Thu, 10 Apr 2014 18:32:50 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 06EEAB945; Thu, 10 Apr 2014 14:32:50 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: installing FreeBSD 8.4 in bhyve Date: Thu, 10 Apr 2014 14:31:59 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <20140410180929.GB34413@mouf.net> In-Reply-To: <20140410180929.GB34413@mouf.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201404101431.59749.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Apr 2014 14:32:50 -0400 (EDT) Cc: virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 18:32:51 -0000 On Thursday, April 10, 2014 2:09:30 pm Steve Wills wrote: > Hi, > > I'm trying to install FreeBSD 8.4 in bhyve, but running into some issues. > Sysinstall doesn't seem to like the disk. In the partition editor, I get: > > Disk name: vtbd1 FDISK Partition Editor > DISK Geometry: 0 cyls/0 heads/0 sectors = 0 sectors (0MB) > > Offset Size(ST) End Name PType Desc Subtype Flags > > 0 209715200 209715199 - 12 unused 0 > > Any ideas what I'm doing wrong? This is using the 8.4 Release disc1 iso on > 11-CURRENT r264056. I'm using virtio-blk for both the cd and hd. Should this be > working? I resorted to using the LiveCD to install by hand using the ZFSOnRoot instructions as a guideline. -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 10 19:07:29 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E883E91; Thu, 10 Apr 2014 19:07:29 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 124FD17FA; Thu, 10 Apr 2014 19:07:28 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 88406124C4; Fri, 11 Apr 2014 05:07:26 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTG72627 (AUTH peterg@ptree32.com.au); Fri, 11 Apr 2014 05:07:25 +1000 Message-ID: <5346EBEA.3070005@freebsd.org> Date: Thu, 10 Apr 2014 12:07:22 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steve Wills Subject: Re: installing FreeBSD 8.4 in bhyve References: <20140410180929.GB34413@mouf.net> In-Reply-To: <20140410180929.GB34413@mouf.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 19:07:29 -0000 Hi Steve, > I'm trying to install FreeBSD 8.4 in bhyve, but running into some issues. > Sysinstall doesn't seem to like the disk. In the partition editor, I get: > > Disk name: vtbd1 FDISK Partition Editor > DISK Geometry: 0 cyls/0 heads/0 sectors = 0 sectors (0MB) > > Offset Size(ST) End Name PType Desc Subtype Flags > > 0 209715200 209715199 - 12 unused 0 > > Any ideas what I'm doing wrong? The bhyve virtio-block backend doesn't report c/h/s geometry, which is required by the 8.4 installer. You can work around this by using the 'G' command to enter a fake geometry that will give the size of the device. > This is using the 8.4 Release disc1 iso on > 11-CURRENT r264056. I'm using virtio-blk for both the cd and hd. Should this be > working? Not cleanly :( I have an 8.4/i386 disk image so I managed to get through it at one point: can't remember what the workarounds were though. Think I did a network install with the minimal ISO. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 10 19:59:21 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26946B4E; Thu, 10 Apr 2014 19:59:21 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DA7BC1CD7; Thu, 10 Apr 2014 19:59:20 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s3AJxAxg037034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 10 Apr 2014 19:59:15 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s3AJxAh8037033; Thu, 10 Apr 2014 19:59:10 GMT (envelope-from swills) Date: Thu, 10 Apr 2014 19:59:10 +0000 From: Steve Wills To: Peter Grehan Subject: Re: installing FreeBSD 8.4 in bhyve Message-ID: <20140410195909.GC34413@mouf.net> References: <20140410180929.GB34413@mouf.net> <5346EBEA.3070005@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5346EBEA.3070005@freebsd.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Thu, 10 Apr 2014 19:59:15 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 19:59:21 -0000 On Thu, Apr 10, 2014 at 12:07:22PM -0700, Peter Grehan wrote: > Hi Steve, > > > I'm trying to install FreeBSD 8.4 in bhyve, but running into some issues. > > Sysinstall doesn't seem to like the disk. In the partition editor, I get: > > > > Disk name: vtbd1 FDISK Partition Editor > > DISK Geometry: 0 cyls/0 heads/0 sectors = 0 sectors (0MB) > > > > Offset Size(ST) End Name PType Desc Subtype Flags > > > > 0 209715200 209715199 - 12 unused 0 > > > > Any ideas what I'm doing wrong? > > The bhyve virtio-block backend doesn't report c/h/s geometry, which is > required by the 8.4 installer. You can work around this by using the 'G' > command to enter a fake geometry that will give the size of the device. > > > This is using the 8.4 Release disc1 iso on > > 11-CURRENT r264056. I'm using virtio-blk for both the cd and hd. Should this be > > working? > > Not cleanly :( I have an 8.4/i386 disk image so I managed to get > through it at one point: can't remember what the workarounds were > though. Think I did a network install with the minimal ISO. Thanks! Setting the geometry allowed me to partition and then after configuring the networking I was able to install via passive ftp. Steve From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 11 19:49:07 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 179CDADC for ; Fri, 11 Apr 2014 19:49:07 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CC72E15CA for ; Fri, 11 Apr 2014 19:49:06 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s3BJmu6d060929 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 11 Apr 2014 19:49:01 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s3BJmuaT060928 for virtualization@FreeBSD.org; Fri, 11 Apr 2014 19:48:56 GMT (envelope-from swills) Date: Fri, 11 Apr 2014 19:48:56 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: pinning CPUs in bhyve Message-ID: <20140411194855.GA59594@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Fri, 11 Apr 2014 19:49:01 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 19:49:07 -0000 Hi, Can someone help me understand the cpu pinning feature in bhyve? I'm afraid the docs have confused me: -p pinnedcpu Force guest virtual CPUs to be pinned to host CPUs. Virtual CPU n is pinned to host CPU pinnedcpu+n. What would the arg looks like? Something like -p 0,1,2,3 ? Also, is there like to be much or any benefit from it? Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 12 00:53:20 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43CEEDC7; Sat, 12 Apr 2014 00:53:20 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0764412DF; Sat, 12 Apr 2014 00:53:19 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 9B041125BC; Sat, 12 Apr 2014 10:53:17 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTH54461 (AUTH peterg@ptree32.com.au); Sat, 12 Apr 2014 10:53:16 +1000 Message-ID: <53488E7A.5070800@freebsd.org> Date: Fri, 11 Apr 2014 17:53:14 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steve Wills Subject: Re: pinning CPUs in bhyve References: <20140411194855.GA59594@mouf.net> In-Reply-To: <20140411194855.GA59594@mouf.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 00:53:20 -0000 Hi Steve, > Can someone help me understand the cpu pinning feature in bhyve? I'm afraid the > docs have confused me: > > -p pinnedcpu > Force guest virtual CPUs to be pinned to host CPUs. Virtual CPU n is pinned > to host CPU pinnedcpu+n. > > What would the arg looks like? Something like -p 0,1,2,3 ? No - currently the parameter can only select the starting host CPU e.g. "-p 3" for a quad vCPU guest will force the vCPU threads to be pinned to host CPUs 3,4,5 and 6. > Also, is there like to be much or any benefit from it? Not really. It's for very specialized use cases, and could actually hurt performance. Not too different in that respect than cpuset with user processes. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 13 15:08:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0967347; Sun, 13 Apr 2014 15:08:42 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1689C1717; Sun, 13 Apr 2014 15:08:41 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ec20so4963593lab.11 for ; Sun, 13 Apr 2014 08:08:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=SDf6FT3Uqtk1gsG0d/BCGTOKA7g+BkAKI5V+yeehPDg=; b=Z7a+kHDSEQOg72eQX1k1hiWuqUvuFmiftzDkMdYAMWiuzEbw8Wu7IZDyQ5KSjf0QLR oZ3/b5dt0YLy/FEQjAvMDamulsqqUqSRg8tmf/AJ8cUbUWHGKJlg15vriHcWvSAdeoLJ kC1T+JHJCZeQjzy4WK5fI9iehwxBr2o1HyhhbVYaalNPVmFgHyqYYMa43DBuNOPNi5qz adekZD7u5eDQ5C6LHA5LxINv1fjiO+m1M+gfjoy1FTFp+muv1bGFNH8WctjiN+Psp+th oaSvML0nZHris/bfV0LP+0eyrI2X1sfQsb8uXcqx/bC+jprWO+8j7IdUo+/OySm0WoZY 18RQ== X-Received: by 10.152.197.98 with SMTP id it2mr42359lac.53.1397401719974; Sun, 13 Apr 2014 08:08:39 -0700 (PDT) Received: from kloomba ([95.104.136.90]) by mx.google.com with ESMTPSA id kz7sm13136530lab.16.2014.04.13.08.08.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 13 Apr 2014 08:08:38 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 13 Apr 2014 19:08:24 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: bhyve: allow specifiying tty by fd Message-ID: <20140413150823.GB99234@kloomba> References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> <20140409172657.GB1313@kloomba> <534584FE.60807@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pvezYHf7grwyp3Bc" Content-Disposition: inline In-Reply-To: <534584FE.60807@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2014 15:08:42 -0000 --pvezYHf7grwyp3Bc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > Hi Roman, >=20 > >>> bhyve -s 31,lpc -l com1,fd=3D19 vm0 > >> > >> Yes, useful idea, and will become more relevant in the capsicum wor= ld > >> of passing fd's to a sandboxed bhyve. > >> > >> Thanks: I'll get this submitted. > > > > Hi Peter! > > > > Any news on that? >=20 > I started to do it, and then hesitated bit for two reasons. First,=20 > it's useful for all backends, but there still needs to be a type - file= =20 > descriptors can't be treated as something opaque: there still needs to=20 > be knowledge of whether it's a socket, char device, file etc, so the=20 > option may need additional information to describe that. Could it be an additional comma separated option? E.g.: -l com1,fd=3D42,sock or -l com1,fd=3D42,tty etc > Also, given that Capsicum wants to pass fd's as parameters, I was=20 > wondering if there is a general syntax style for doing this. "fd=3D%d"=20 > seems Ok, though it would be useful if there was something in libutil to= =20 > parse an option like this if it is the preferred way forward for FreeBSD= =20 > executables. >=20 > What do folk think ? Any useful precedents ? Off the top of my head I cannot recall such an example in the tree. Roman Bogorodskiy --pvezYHf7grwyp3Bc Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTSqhnAAoJEMltX/4IwiJqWe0H/Raut30Kmj1/JSCqOtDc/U7b 3sYIS0df9VfRhlAI4kpnhM+4A6AHZ8+EW5hZuDPB36xeDnmkm3OtJVvWpMlG26yz N9JZwcLuL61Sf3wLK5D8jLRqlvf6vdEdZd1YlcycumVLuS1/IPvURwN5WaER8xNT AoSnpOesrrIfWnWD3GNxmur+tfAjRXD6DFhZ+qddTFI/TMO8rtQCIgrsH09lXrMK dA6CwIurHNnr5ZG4j3kqf3mPyw6Agsqb9u5rib+YmvKqsn0wxDvAzP7xieZOHRnz nsBTQsBdRNwBpeZLTWpVOarrpPG4ED81YdOVlDmFBidgLd6vppHDO5R+oEe1sIA= =+Ll3 -----END PGP SIGNATURE----- --pvezYHf7grwyp3Bc-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 14 11:06:54 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70C5227A for ; Mon, 14 Apr 2014 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E35E167D for ; Mon, 14 Apr 2014 11:06:54 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3EB6s8c026061 for ; Mon, 14 Apr 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3EB6r3R026059 for freebsd-virtualization@FreeBSD.org; Mon, 14 Apr 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 14 Apr 2014 11:06:53 GMT Message-Id: <201404141106.s3EB6r3R026059@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p kern/183835 virtualizationKernel panic with VIMAGE on insertion of axe USB netwo o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 8 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 14 18:08:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36245BF7; Mon, 14 Apr 2014 18:08:33 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id EBC5417E3; Mon, 14 Apr 2014 18:08:32 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 1CA1B12328; Tue, 15 Apr 2014 04:08:25 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTK06776 (AUTH peterg@ptree32.com.au); Tue, 15 Apr 2014 04:08:24 +1000 Message-ID: <534C2411.5040208@freebsd.org> Date: Mon, 14 Apr 2014 11:08:17 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: bhyve: allow specifiying tty by fd References: <20140305130030.GA40560@dev.san.ru> <5318BF7F.4060306@freebsd.org> <20140409172657.GB1313@kloomba> <534584FE.60807@freebsd.org> <20140413150823.GB99234@kloomba> In-Reply-To: <20140413150823.GB99234@kloomba> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 18:08:33 -0000 Hi Roman, > Could it be an additional comma separated option? E.g.: > > -l com1,fd=42,sock > > or > > -l com1,fd=42,tty Yes, this is a possibility. I'll get your patch in for the tty backend as a start. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 17 01:01:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC4F37A9 for ; Thu, 17 Apr 2014 01:01:33 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0145.outbound.protection.outlook.com [207.46.163.145]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D1371CC4 for ; Thu, 17 Apr 2014 01:01:32 +0000 (UTC) Received: from BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) by BL2PR03MB210.namprd03.prod.outlook.com (10.255.230.144) with Microsoft SMTP Server (TLS) id 15.0.918.8; Thu, 17 Apr 2014 01:01:23 +0000 Received: from BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.186]) by BL2PR03MB210.namprd03.prod.outlook.com ([169.254.1.186]) with mapi id 15.00.0918.000; Thu, 17 Apr 2014 01:01:23 +0000 From: "Abhishek Gupta (LIS)" To: "freebsd-virtualization@freebsd.org" Subject: Duplicating code from NetBSD Thread-Topic: Duplicating code from NetBSD Thread-Index: Ac9Z17mSGu/tntFXQyOH+5QNusIlKQ== Date: Thu, 17 Apr 2014 01:01:23 +0000 Message-ID: <6a7255a1fdfc41c79e38532bf267a06f@BL2PR03MB210.namprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2001:4898:80e8:ee31::3] x-forefront-prvs: 01842C458A x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(164054003)(189002)(199002)(33646001)(81542001)(80022001)(20776003)(76576001)(15202345003)(16236675002)(31966008)(74662001)(74502001)(79102001)(4396001)(81342001)(77982001)(19300405004)(83322001)(83072002)(86612001)(19580395003)(15975445006)(74316001)(50986999)(87936001)(2656002)(85852003)(85806002)(54356999)(46102001)(76482001)(86362001)(80976001)(92566001)(99396002)(99286001)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR03MB210; H:BL2PR03MB210.namprd03.prod.outlook.com; FPR:7E92C04D.CF112B3A.4DE193F0.4CE129E0.20118; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (: microsoft.com does not designate permitted sender hosts) MIME-Version: 1.0 X-OriginatorOrg: microsoft.onmicrosoft.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 01:01:33 -0000 Hi folks, Newbie question so please excuse the naivety. If someone were to copy a few= functions from NetBSD in to their FreeBSD kernel mode drivers then is it s= till possible to submit the code in to the FreeBSD tree under the FreeBSD l= icense? If not then would the community accept the copied code cleanly sepa= rated out in a file and licensed under the NetBSD license? Please let me kn= ow if there is an acceptable procedure to do this or if this is a complete = NONO. Thanks, Abhishek From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 17 13:00:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72530F23 for ; Thu, 17 Apr 2014 13:00:06 +0000 (UTC) Received: from mail-ee0-x22c.google.com (mail-ee0-x22c.google.com [IPv6:2a00:1450:4013:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04DB218C1 for ; Thu, 17 Apr 2014 13:00:05 +0000 (UTC) Received: by mail-ee0-f44.google.com with SMTP id e49so645832eek.31 for ; Thu, 17 Apr 2014 06:00:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Dw8xzEmn6zWe696ivUVqU7bsQ9Su8WuRTz1naEDe5jk=; b=pT82A5kqG4OB6wNHr1Z785HGuWlpdgdnzFOUUyolWvOLqaq84h0yX0wv/k8QJqikdC JraeUgBB258Wx/NJANPw4h1F8Hrg69uZrOeGSoEJSX0MxzNnwPYRjou1ca/2h3SZC2gP EgosePSKfcUIDaD4WA3XYV7mNCjq+iffzSwqLYzVvkK42c2pucgn6PuKtUWDNps71KFj bOgeHSMdoZGnlLSmmKbpXM2yVDnTQhfaLbaxQP1HJAHZAHARGgBZNgcMDc7Lo9wAE0P0 u5kHZQ2xzbcs4LElH6p1kEfXfsfav02hPm61VSlEth5e6aCjSvREkBmdS9vjDbgIIu1v P+yg== X-Received: by 10.15.53.135 with SMTP id r7mr1874406eew.102.1397739603069; Thu, 17 Apr 2014 06:00:03 -0700 (PDT) Received: from strashydlo.home (adcx26.neoplus.adsl.tpnet.pl. [79.184.49.26]) by mx.google.com with ESMTPSA id s46sm67201634ees.3.2014.04.17.06.00.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 06:00:02 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Duplicating code from NetBSD Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= In-Reply-To: <6a7255a1fdfc41c79e38532bf267a06f@BL2PR03MB210.namprd03.prod.outlook.com> Date: Thu, 17 Apr 2014 15:00:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7A221B2C-6439-46E7-8343-51945B5626A7@FreeBSD.org> References: <6a7255a1fdfc41c79e38532bf267a06f@BL2PR03MB210.namprd03.prod.outlook.com> To: Abhishek Gupta (LIS) X-Mailer: Apple Mail (2.1283) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 13:00:06 -0000 Wiadomo=B6=E6 napisana przez Abhishek Gupta (LIS) w dniu 17 kwi 2014, o = godz. 03:01: > Hi folks, >=20 > Newbie question so please excuse the naivety. If someone were to copy = a few functions from NetBSD in to their FreeBSD kernel mode drivers then = is it still possible to submit the code in to the FreeBSD tree under the = FreeBSD license? If not then would the community accept the copied code = cleanly separated out in a file and licensed under the NetBSD license? = Please let me know if there is an acceptable procedure to do this or if = this is a complete NONO. It all depends on the license. If the code is licensed under standard = BSD license, then it's perfectly ok to commit it to FreeBSD. In fact, a lot = of things got ported from NetBSD, just grep for it in sources. From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 20 02:12:03 2014 Return-Path: Delivered-To: freebsd-virtualization@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABA26291; Sun, 20 Apr 2014 02:12:03 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8060F19BD; Sun, 20 Apr 2014 02:12:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3K2C3U3096764; Sun, 20 Apr 2014 02:12:03 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3K2C3k7096763; Sun, 20 Apr 2014 02:12:03 GMT (envelope-from linimon) Date: Sun, 20 Apr 2014 02:12:03 GMT Message-Id: <201404200212.s3K2C3k7096763@freefall.freebsd.org> To: elakin@infohell.net, linimon@FreeBSD.org, freebsd-virtualization@FreeBSD.org, freebsd-virtualization@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/183835: Kernel panic with VIMAGE on insertion of axe USB network interface X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 02:12:03 -0000 Synopsis: Kernel panic with VIMAGE on insertion of axe USB network interface State-Changed-From-To: patched->closed State-Changed-By: linimon State-Changed-When: Sun Apr 20 01:48:45 UTC 2014 State-Changed-Why: MFCed to appropriate branches. Responsible-Changed-From-To: freebsd-virtualization->freebsd-virtualization Responsible-Changed-By: linimon Responsible-Changed-When: Sun Apr 20 01:48:45 UTC 2014 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=183835 From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 20 23:35:48 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 712C0A01 for ; Sun, 20 Apr 2014 23:35:48 +0000 (UTC) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E62D610D2 for ; Sun, 20 Apr 2014 23:35:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id s3KNYHX1036073 for ; Mon, 21 Apr 2014 03:34:17 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Mon, 21 Apr 2014 03:34:17 +0400 (MSK) From: Dmitry Morozovsky To: freebsd-virtualization@FreeBSD.org Subject: HOWTO needed for installing Linux VM under bhyve Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Mon, 21 Apr 2014 03:34:17 +0400 (MSK) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 23:35:48 -0000 Dear colleagues, I've found some vague links to sysutils/grub2-bhyve, but my naive attempts to get things done failed. I tried to run debian7/i386, if this is important. Native freebsd VMs are fine. Could anybody please point me to the document how should we cook grub2? Or, maybe, collect it on wiki.freebsd.org before putting it into the Handbook? Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 21 04:52:37 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C022E4D4 for ; Mon, 21 Apr 2014 04:52:37 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8182F1A9F for ; Mon, 21 Apr 2014 04:52:37 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id C7108124FA; Mon, 21 Apr 2014 14:52:29 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-174-51-225-141.hsd1.co.comcast.net [174.51.225.141]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTN44545 (AUTH peterg@ptree32.com.au); Mon, 21 Apr 2014 14:52:28 +1000 Message-ID: <5354A40A.9050002@freebsd.org> Date: Sun, 20 Apr 2014 22:52:26 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Dmitry Morozovsky Subject: Re: HOWTO needed for installing Linux VM under bhyve References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 04:52:37 -0000 > I've found some vague links to sysutils/grub2-bhyve, but my naive attempts to > get things done failed. I tried to run debian7/i386, if this is important. > Native freebsd VMs are fine. Never tried a 32-bit Linux. 64-bit Debian7 works fine. > Could anybody please point me to the document how should we cook grub2? Or, > maybe, collect it on wiki.freebsd.org before putting it into the Handbook? I'll get something up on the wiki. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 21 10:22:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9E559E8 for ; Mon, 21 Apr 2014 10:22:22 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40A1F1618 for ; Mon, 21 Apr 2014 10:22:22 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id mc6so3020916lab.22 for ; Mon, 21 Apr 2014 03:22:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=1dRHUVbS9kZKyqrJW195lStirgND/JcaS9Mz6HdAaUM=; b=qfiM8bRW8tX4lyX2m9pxt8onfDxv434o4DygNEFIhLrcX/HQRyi6iU9S92KCuHCHtm o++TcUuc7DG9A6g9L6oI/6ahmT17DTnCfvAylnEtetS7hAjSFvr8U94ngBlya2i+jCkZ n190GZ26MSlzbby7s4lVUlSksGMIg0Zzl5HVeq49XdeaUJEqn4T31rw8rGViq/oG5bp5 mlnm/o5nh8E4BZs/hI7Kx1NM87WeilAI/qo1QxNvOK6H1ejJ4Ljwp8xKbNZp77qjkPns FKfFFovp49f91dfhA4wB/+0z4lAEekDys8L/QwknAj5t+pNJA6WeFzTe9WnT47/6X+Hk 2Egg== X-Received: by 10.112.126.7 with SMTP id mu7mr22965065lbb.17.1398075740148; Mon, 21 Apr 2014 03:22:20 -0700 (PDT) Received: from kloomba ([95.104.139.44]) by mx.google.com with ESMTPSA id q4sm36957440lbh.20.2014.04.21.03.22.19 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Apr 2014 03:22:19 -0700 (PDT) Sender: Roman Bogorodskiy Date: Mon, 21 Apr 2014 14:21:40 +0400 From: Roman Bogorodskiy To: freebsd-virtualization@FreeBSD.org Subject: Understanding CPU and memory usage in Bhyve Message-ID: <20140421102138.GA6157@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="45Z9DzgjV8m4Oswq" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 10:22:22 -0000 --45Z9DzgjV8m4Oswq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I'm trying to understand how to obtain resource usage information for a bhyve guest. For the CPU, there's some info provided by bhyvectl, e.g.: /usr/sbin/bhyvectl --vm=3Dbhyve --get-stats --cpu=3D1 in the field 'vcpu total runtime', for example: vcpu total runtime 8178870653 Two question about that: 1. What are the units used here? Looks like it's ticks, so if I want to convert this number, to, say, nanoseconds, would it be right to do something like: X * 1000**3 / kern.clockrate.stathz=20 (X beeing the value returned by bhyvectl and kern.clockrate is a sysctl name)? 2. This value shows only vcpu time, without hypervisor time? For the memory stats, it's less obvious. Does guest always use an amount of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could use less? What is the format of bhyvectl --get-{low,high}mem output? PS I found it a little confusing that bhyvectl displays vcpu0 stats by default if --cpu is not provided, expected it provide info of all vcpus in the guest by default. Also, didn't find a way to get a number of vcpus in a running guest. Roman Bogorodskiy --45Z9DzgjV8m4Oswq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTVPEyAAoJEMltX/4IwiJqvUcH/2cIUMrNCELYek49anhh4KNY IwUS7eJ0esJaBAvHVqAp5XpnUuxfJyxuKuJyMY+guwJ9u2exW0HWIIx8iB4IMRT2 qNpRTGwSAH6MuBGLdXuUTcK9q+nnSgB20KqfU4LrZbSsHirmTC0JV5hTFtIBhsDs eRIkkOBzxcW6VK7qQEye1jJo6jyqV9s7GTONTK3kDjLWYIDJNI+RkEUJnbaH36qI TPN5IFm2QeDXmSxuGSO5n4ZJTy33pkZN2JQCI0WQokhtTCD6TXEeniLo7bZywkI7 wi1eR1oyhzrMUmS3qcOPGe7+4jARdb8H0C8fUNSIVcJ7kZuouRlgm7a6s4OoREc= =BnP2 -----END PGP SIGNATURE----- --45Z9DzgjV8m4Oswq-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 21 11:06:56 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14407194 for ; Mon, 21 Apr 2014 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3A00197D for ; Mon, 21 Apr 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3LB6tqh085892 for ; Mon, 21 Apr 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3LB6trk085890 for freebsd-virtualization@FreeBSD.org; Mon, 21 Apr 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 21 Apr 2014 11:06:55 GMT Message-Id: <201404211106.s3LB6trk085890@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 21 17:39:16 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 781ABBCD; Mon, 21 Apr 2014 17:39:16 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3A97C1155; Mon, 21 Apr 2014 17:39:15 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D109312451; Tue, 22 Apr 2014 03:39:13 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-174-51-225-141.hsd1.co.comcast.net [174.51.225.141]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTN62243 (AUTH peterg@ptree32.com.au); Tue, 22 Apr 2014 03:39:11 +1000 Message-ID: <535557BC.8030300@freebsd.org> Date: Mon, 21 Apr 2014 11:39:08 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: Understanding CPU and memory usage in Bhyve References: <20140421102138.GA6157@kloomba> In-Reply-To: <20140421102138.GA6157@kloomba> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 17:39:16 -0000 Hi Roman, > For the CPU, there's some info provided by bhyvectl, e.g.: > > /usr/sbin/bhyvectl --vm=bhyve --get-stats --cpu=1 > > in the field 'vcpu total runtime', for example: > > vcpu total runtime 8178870653 > > Two question about that: > > 1. What are the units used here? Looks like it's ticks, so if I want to > convert this number, to, say, nanoseconds, would it be right to do > something like: > > X * 1000**3 / kern.clockrate.stathz > > (X beeing the value returned by bhyvectl and kern.clockrate is a > sysctl name)? It's in TSC units so you can convert to nsecs using the machdep.tsc_freq sysctl node. > 2. This value shows only vcpu time, without hypervisor time? That's correct. To get the hypervisor and vCPU time together, you can use existing CPU usage reporting (e.g. that used by top et al). > For the memory stats, it's less obvious. Does guest always use an amount > of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could > use less? That's the maximum it can use. We currently don't expose the incore amount from the vmspace representing the guest's physical - that should be added to bhyvectl soon. > What is the format of bhyvectl --get-{low,high}mem output? The number reported there isn't to do with guest usage, but how the memory is divided between < 4G and above 4G. For example, the default for bhyve is to use up to 3G below 4G, provide 1GB for PCI MMIO decode, and then put the remainder of guest memory above 4G. Here's an example for an 8G VM, with 3G of guest mem below 4G (starting address 0), and 5G above 4G (starting address 4G): lowmem 0x0000000000000000/3221225472 highmem 0x0000000100000000/5368709120 > PS I found it a little confusing that bhyvectl displays vcpu0 stats by > default if --cpu is not provided, expected it provide info of all vcpus > in the guest by default. Yes, that's a relic of the days when there was only a single vCPU supported :( > Also, didn't find a way to get a number of vcpus in a running guest. Until the vCPU state is exposed by bhyvectl, or we provide a sysctl, you can use heuristics: the number of vCPU threads for the bhyve process, or scan all vCPUs and only count those that have a non-zero RIP. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 22 21:48:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BADCF9F7 for ; Tue, 22 Apr 2014 21:48:27 +0000 (UTC) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A225811C2 for ; Tue, 22 Apr 2014 21:48:27 +0000 (UTC) Received: from [10.6.35.14] (199-188-192-168.PUBLIC.monkeybrains.net [199.188.192.168]) (authenticated bits=0) by mail.monkeybrains.net (8.14.7/8.14.7) with ESMTP id s3MLeCOG015439 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 22 Apr 2014 14:40:12 -0700 (PDT) (envelope-from crapsh@monkeybrains.net) X-Authentication-Warning: mail.monkeybrains.net: Host 199-188-192-168.PUBLIC.monkeybrains.net [199.188.192.168] claimed to be [10.6.35.14] Message-ID: <5356E1BC.7060802@monkeybrains.net> Date: Tue, 22 Apr 2014 14:40:12 -0700 From: Rudy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: HOW-TO resize a .img Ubuntu guest Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.98.1 at mail.monkeybrains.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 21:48:27 -0000 Cloning images and growing them. We have a 'standard install' image called, 'ubuntu14-clone-me.img'. To copy, enlarge, and then grow the guest image size, we use the process below. Let's take an Ubuntu image (ubuntu14-clone-me.img), clone it, grow the system, then log into the Ubuntu guest (vm12) and resize the / partition (all our guests are single partition). In FreeBSD: FreeBSD# gcp --sparse=always ubuntu14-clone-me.img vm12.img FreeBSD# truncate -s 300g vm12.img Launch guest with bhyve, get on console, and use these commands: Guest-VM# swapoff -a Guest-VM# fdisk /dev/vda hit 'p' to print existing table delete all the paritions recreate partition '1' with same stating block noted in the 'print' make end block near end but with enough room for your swap create an extended parition '5' and put the rest in there for guest swap Guest-VM# reboot Guest-VM# resize2fs /dev/vda1 Done! Pro tip: research sparse files, install gcp (pkg install coreutils). Rudy MonkeyBrains.net Local ISP From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 23 17:40:08 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABA27DB3 for ; Wed, 23 Apr 2014 17:40:08 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6594F1268 for ; Wed, 23 Apr 2014 17:40:08 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Wd19Q-0007IQ-Ig for freebsd-virtualization@freebsd.org; Wed, 23 Apr 2014 19:40:04 +0200 Received: from 77.202.37.188.rev.vodafone.pt ([188.37.202.77]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Apr 2014 19:40:04 +0200 Received: from miguelmclara by 77.202.37.188.rev.vodafone.pt with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Apr 2014 19:40:04 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-virtualization@freebsd.org From: Miguel Clara Subject: Re: PCI passthrough support for VGA devices in VirtualBox on FreeBSD Date: Wed, 23 Apr 2014 17:34:30 +0000 (UTC) Lines: 14 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 188.37.202.77 (Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 17:40:08 -0000 Looking to do the same... I have a laptop with dual graphics, Its one of those with Intel i7 CPU + Integrated Graphics but also ships with a AMD dedicated card! I cannot use the AMD card successfully with KMS anyway, and I would prefer to pass the card to a Winodows/Linux VM. Is there any work on this in FreeBSD's Vbox port? Thanks From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 23 18:53:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D837302 for ; Wed, 23 Apr 2014 18:53:27 +0000 (UTC) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 38AB419BA for ; Wed, 23 Apr 2014 18:53:26 +0000 (UTC) Received: from compute6.internal (compute6.nyi.mail.srv.osa [10.202.2.46]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id AD39720F83; Wed, 23 Apr 2014 14:53:25 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute6.internal (MEProxy); Wed, 23 Apr 2014 14:53:25 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mesmtp; bh=3wFvVNCPF4FKXDLPB2Meh8XYbqI=; b=nIGS2YkFiTnT86jD59cvMwbpPti9 71Bss2q0tUP8FOaGBE4gl32WYnBcHfRGhj3VVWCDMl41QGbSkdUsUU/X/l5e9eSZ OsOoBfVrBzEXv9Z9fy0nziMTGjLkRHynyjrC5SC42VKIMlZPHa/7s+u5RuvRCFOM CfpRg5iJGqWbFCQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:cc:subject:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; s=smtpout; bh=3wFvVNCPF4FKXDLPB2Meh8 XYbqI=; b=qasrpZrgyO1sW+5/3qfhc8Tou+7puhKf6ZiCKfzMsomtssthAN3VX6 KfLDI9dbeLphCuRyISyin0paPoNLrbTbDjXTjO2vpGrmS/r2n0GCqi8f4BOlssuQ 0s7wQLZceiD/KX1dk1LF5vbTKDeQLTf7EKZdTJUgidKcekppsVvSc= X-Sasl-enc: 7s3cVAMJ5vz5VQ0++6qT39DZdjqs6QOSipJwrFi+zcfb 1398279205 Received: from skysong.divine.net (unknown [109.192.195.65]) by mail.messagingengine.com (Postfix) with ESMTPA id F186A680190; Wed, 23 Apr 2014 14:53:24 -0400 (EDT) Date: Wed, 23 Apr 2014 20:53:27 +0200 From: Schaich Alonso To: miguelmclara@gmail.com Subject: Re: PCI passthrough support for VGA devices in VirtualBox on FreeBSD Message-Id: <20140423205327.80a2b5eead661ab832fbd10a@fastmail.fm> In-Reply-To: References: X-Mailer: Sylpheed 3.4.1 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 18:53:27 -0000 On Wed, 23 Apr 2014 17:34:30 +0000 (UTC) Miguel Clara wrote: > > > Looking to do the same... > > I have a laptop with dual graphics, Its one of those with Intel i7 CPU + > Integrated Graphics but also ships with a AMD dedicated card! > > I cannot use the AMD card successfully with KMS anyway, and I would prefer > to pass the card to a Winodows/Linux VM. > > Is there any work on this in FreeBSD's Vbox port? > > Thanks > > VirtualBox can't do that. If you need only GNU/Linux VMs you can use bhyve with PCI Forwarding ( see https://wiki.freebsd.org/bhyve/pci_passthru ). Otherwise, I think you'll have to resort to Xen or VMWare ESXi (or eventually Microsoft's VirtualPC). Alonso From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 23 19:05:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43927472 for ; Wed, 23 Apr 2014 19:05:29 +0000 (UTC) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC1531AAD for ; Wed, 23 Apr 2014 19:05:28 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id u56so1250979wes.23 for ; Wed, 23 Apr 2014 12:05:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Litbqtido3v8+tqQ4zoYuvXTWk4Zy62xH8zFwfBRoJk=; b=qhm6uxu0HXbE/tYIHTTv4ODBQk169omA+/aAzw4B5Vjg2Jli258/sXaKl977ooAKS5 2OPWZuxoi0CwAXRI5vUgTGGfEkaO0j01G3dl2vvXYMmK2TvL3RJO4Cuks1o30VjnXwwb Fr7oHEUTfLFNtac4j5SjWH7ETiYkfwtKZc4wP+eWyavsN39JDJWOJtoUJYoGErtDUELX vrfXu0kF5BJWbvnxJKHpg5/U2HlCrxLCTGop7yzLFwQIvPY3WGnK6U21PYyjjBtlkp3n hQfSTj1LxulJGm9xZGl1RyAqi6rzD1AOtd6e1bdOUNDpoDANRc2hTIVWJjR+jupZbmcN +96Q== X-Received: by 10.194.78.4 with SMTP id x4mr2988149wjw.58.1398279927184; Wed, 23 Apr 2014 12:05:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.217.55.138 with HTTP; Wed, 23 Apr 2014 12:05:07 -0700 (PDT) In-Reply-To: <20140423205327.80a2b5eead661ab832fbd10a@fastmail.fm> References: <20140423205327.80a2b5eead661ab832fbd10a@fastmail.fm> From: Miguel Clara Date: Wed, 23 Apr 2014 20:05:07 +0100 Message-ID: Subject: Re: PCI passthrough support for VGA devices in VirtualBox on FreeBSD To: Schaich Alonso Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 19:05:29 -0000 VirtualBox can... I'm assuming you mean the FreeBSD port can't. In this case I would need a Windows Guest, but its a laptop and I still wan't to use the OS. I'm aware of the other options, and there's also KVM but I want the host to be FreeBSD. Xen could be an option, since its a laptop and I still want to use the host os, maybe its the most suited! I would go the bhyve path but has you said and very well only works for Linux, not windows! Anyway the main question is is if FreeBSD port of Vbox supports this, or if there's any work done or planned for this. Thanks On Wed, Apr 23, 2014 at 7:53 PM, Schaich Alonso wrote: > On Wed, 23 Apr 2014 17:34:30 +0000 (UTC) > Miguel Clara wrote: > > > > > > > Looking to do the same... > > > > I have a laptop with dual graphics, Its one of those with Intel i7 CPU + > > Integrated Graphics but also ships with a AMD dedicated card! > > > > I cannot use the AMD card successfully with KMS anyway, and I would > prefer > > to pass the card to a Winodows/Linux VM. > > > > Is there any work on this in FreeBSD's Vbox port? > > > > Thanks > > > > > > VirtualBox can't do that. > If you need only GNU/Linux VMs you can use bhyve with PCI Forwarding ( see > https://wiki.freebsd.org/bhyve/pci_passthru ). Otherwise, I think you'll > have > to resort to Xen or VMWare ESXi (or eventually Microsoft's VirtualPC). > > Alonso > From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 23 19:07:31 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66CA94D2 for ; Wed, 23 Apr 2014 19:07:31 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF3A51AC1 for ; Wed, 23 Apr 2014 19:07:30 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id t61so1294247wes.31 for ; Wed, 23 Apr 2014 12:07:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=hCFtYQZZeBTZ6ubhn3iX9NuwvP1vB/HVSyi1P832Sp4=; b=SKGNmhZsMmEMsmj8GwH9IBnyMKD5ThV55raOBjGiTOdRip8mH8SC5NljfccoC/FU73 lKjk2sAuyRctpkFQMVY6APvb4sKZNM0X8tsYSnkVhy9tHmDEmOS36pmkDoq9sy07qq0p aJdlXXsT0oYJzFxinXpPz4r6yYUZzu0C+l/1n693EHYhl1UOXU7zlu4eFs8AeTbciUrC QkRrk/tjZX8/FouQUuYWGfw09vpPPkP6t6b9sTbLc2KAgWYla+0Nx277xpfpzRIo+R6C 9lZ+pauaqE3doi3yhAVmCsHnWmcAvtzSTE6kpfPbL9d6dINiyw524nco9okQK34XUohi GvSw== X-Received: by 10.180.7.227 with SMTP id m3mr2939392wia.59.1398280049156; Wed, 23 Apr 2014 12:07:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.217.55.138 with HTTP; Wed, 23 Apr 2014 12:07:09 -0700 (PDT) In-Reply-To: References: <20140423205327.80a2b5eead661ab832fbd10a@fastmail.fm> From: Miguel Clara Date: Wed, 23 Apr 2014 20:07:09 +0100 Message-ID: Subject: Re: PCI passthrough support for VGA devices in VirtualBox on FreeBSD To: Schaich Alonso Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 19:07:31 -0000 Actually Xen is not really a suitable option if I want to keep FreeBSD has host, AFAIK dom0 support is just being added and its not really usable yet! On Wed, Apr 23, 2014 at 8:05 PM, Miguel Clara wrote: > VirtualBox can... I'm assuming you mean the FreeBSD port can't. > > In this case I would need a Windows Guest, but its a laptop and I still > wan't to use the OS. > > I'm aware of the other options, and there's also KVM but I want the host > to be FreeBSD. > > Xen could be an option, since its a laptop and I still want to use the > host os, maybe its the most suited! > > I would go the bhyve path but has you said and very well only works for > Linux, not windows! > > Anyway the main question is is if FreeBSD port of Vbox supports this, or > if there's any work done or planned for this. > > Thanks > > > > On Wed, Apr 23, 2014 at 7:53 PM, Schaich Alonso > wrote: > >> On Wed, 23 Apr 2014 17:34:30 +0000 (UTC) >> Miguel Clara wrote: >> >> > >> > >> > Looking to do the same... >> > >> > I have a laptop with dual graphics, Its one of those with Intel i7 CPU + >> > Integrated Graphics but also ships with a AMD dedicated card! >> > >> > I cannot use the AMD card successfully with KMS anyway, and I would >> prefer >> > to pass the card to a Winodows/Linux VM. >> > >> > Is there any work on this in FreeBSD's Vbox port? >> > >> > Thanks >> > >> > >> >> VirtualBox can't do that. >> If you need only GNU/Linux VMs you can use bhyve with PCI Forwarding ( see >> https://wiki.freebsd.org/bhyve/pci_passthru ). Otherwise, I think you'll >> have >> to resort to Xen or VMWare ESXi (or eventually Microsoft's VirtualPC). >> >> Alonso >> > > From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 24 16:09:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED3556F9; Thu, 24 Apr 2014 16:09:00 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 447451CD6; Thu, 24 Apr 2014 16:09:00 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id q8so2155761lbi.38 for ; Thu, 24 Apr 2014 09:08:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=zj+moZUYwbHlp6VG+wvcR5fnPg1NRDoZF2AUKHJxpJY=; b=Nfkr+d3t/wr37RquNEXFoqfuzJw2YkpH4GcFUICPF1Cbjd2odB381DtapfeJK4jHQh ZsIwpJEptfDe2SKi7O58h5gzkE/PYmZJuXBjHG8jZ68B0+nru0ZLI1WqU+EsOQKCtBfL l0hCMS04S0pFL/+0LXNFkBUfAERv3v/pJoH8+/BfOdYtUA2cIrE0MqM/SxdQo9h/e5fD xlgJNWBz7RdLlfQrXEngjn0lFYP5yFtS3QAmO9Gr+Ih5ZE0+5y7K3yTNNJ5kDCJFu2xf PCO/13jYRkMbONqJxPNP5QypYk5clZ1PgcP5R1n0B2KAvKVODCv7ufFaqInWHRvjIehe flKA== X-Received: by 10.112.92.109 with SMTP id cl13mr1708220lbb.50.1398355736318; Thu, 24 Apr 2014 09:08:56 -0700 (PDT) Received: from kloomba ([217.65.211.142]) by mx.google.com with ESMTPSA id t3sm4334866laj.15.2014.04.24.09.08.53 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Apr 2014 09:08:53 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 24 Apr 2014 20:08:17 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: Understanding CPU and memory usage in Bhyve Message-ID: <20140424160816.GB3494@kloomba> References: <20140421102138.GA6157@kloomba> <535557BC.8030300@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lEGEL1/lMxI0MVQ2" Content-Disposition: inline In-Reply-To: <535557BC.8030300@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 16:09:01 -0000 --lEGEL1/lMxI0MVQ2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > Hi Roman, >=20 > > For the CPU, there's some info provided by bhyvectl, e.g.: > > > > /usr/sbin/bhyvectl --vm=3Dbhyve --get-stats --cpu=3D1 > > > > in the field 'vcpu total runtime', for example: > > > > vcpu total runtime 8178870653 > > > > Two question about that: > > > > 1. What are the units used here? Looks like it's ticks, so if I want = to > > convert this number, to, say, nanoseconds, would it be right to do > > something like: > > > > X * 1000**3 / kern.clockrate.stathz > > > > (X beeing the value returned by bhyvectl and kern.clockrate is a > > sysctl name)? >=20 > It's in TSC units so you can convert to nsecs using the=20 > machdep.tsc_freq sysctl node. >=20 > > 2. This value shows only vcpu time, without hypervisor time? >=20 > That's correct. To get the hypervisor and vCPU time together, you can= =20 > use existing CPU usage reporting (e.g. that used by top et al). Thanks for the clarification, it's very helpful. However, I need some more help with that. I'm trying to implement some domain stats reporting for the bhyve driver in libvirt and it looks like I choose a wrong approach initially.=20 It's described here: https://www.redhat.com/archives/libvir-list/2014-April/msg00912.html But the summary is: is there a way to figure out how much CPU time bhyve and the guest spends on host CPUs N (N =3D 0, 1, ...)? > > For the memory stats, it's less obvious. Does guest always use an amount > > of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could > > use less? >=20 > That's the maximum it can use. We currently don't expose the incore=20 > amount from the vmspace representing the guest's physical - that should= =20 > be added to bhyvectl soon. Looking forward to that. > > What is the format of bhyvectl --get-{low,high}mem output? >=20 > The number reported there isn't to do with guest usage, but how the=20 > memory is divided between < 4G and above 4G. For example, the default=20 > for bhyve is to use up to 3G below 4G, provide 1GB for PCI MMIO decode,= =20 > and then put the remainder of guest memory above 4G. Here's an example=20 > for an 8G VM, with 3G of guest mem below 4G (starting address 0), and 5G= =20 > above 4G (starting address 4G): >=20 > lowmem 0x0000000000000000/3221225472 > highmem 0x0000000100000000/5368709120 >=20 > > PS I found it a little confusing that bhyvectl displays vcpu0 stats by > > default if --cpu is not provided, expected it provide info of all vcpus > > in the guest by default. >=20 > Yes, that's a relic of the days when there was only a single vCPU=20 > supported :( >=20 > > Also, didn't find a way to get a number of vcpus in a running guest. >=20 > Until the vCPU state is exposed by bhyvectl, or we provide a sysctl,=20 > you can use heuristics: the number of vCPU threads for the bhyve=20 > process, or scan all vCPUs and only count those that have a non-zero RIP. >=20 > later, >=20 > Peter. >=20 Thanks, Roman Bogorodskiy --lEGEL1/lMxI0MVQ2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTWTbwAAoJEMltX/4IwiJqEygH/ippVMN8384c2RA8G8q2/jE1 LU2f4tyJaNQ/SMQtl3Ed0RTr8KThal2wXXGnujEw3TJaYTFDFaZ31rHAFT1/cPTG szuMgZ6mO47anNInoOBTQu0jOxgeUv/1z0V/zd+WDidFPRSI5YQFHeQe47/TjFqQ fJEUo6dExYeQl1tMkxi1oC1xvi04zbZgMuNPpeGUQBhpJ8MZkSILIEaiHWwFAOex g8hii/zHF9bEgNG/NnNPgrc56dIqn5kv736nLuWTHHog1pleTsW0qJL+X3IBIVZu CSKQ5twHMgWw57xXmpoMZwsIORkHb1AUHBHMZKbYA3WFKBNivrKB4x+6D5SR2S4= =ERSu -----END PGP SIGNATURE----- --lEGEL1/lMxI0MVQ2-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 25 19:05:13 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F38D074B for ; Fri, 25 Apr 2014 19:05:12 +0000 (UTC) Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com [209.85.217.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B9B41A63 for ; Fri, 25 Apr 2014 19:05:11 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id s7so3433487lbd.9 for ; Fri, 25 Apr 2014 12:05:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version:content-transfer-encoding:content-type; bh=oEmQpoTNxMM1URru2VRADzFrh7bxvBCxKAgnsp4xZQ4=; b=XH1e7IOOXYH6nfbT4djN/W19yGOF1s8P5p5SPJXj6NICmQ4liOtw8sZIhXbRE/J8xZ WYw/aQqHFPOJU0alr8ZgIBg4nviSX7kdxKN1kRhMMizmR7QtxUk9AyiBMs1PwrSeUrTh xDpFCIju/u4aXfA1KoRbeVJ5pESSvb38LsPwfVlHWTJdFxYqjk5u7CaOl+A3QZCxf1Hw aM0idEAGM0IzrDn3kPV+0gu0Vui8M5bWAvuM5HmTZ1nT08TLZIhPquc4g3Du5YvwKDUe HbQxAY2JivxF2Zii9wzEnJRcrY0Y9ST7PoN+T9KuKETvIk4HlRsDdW6ig4YqP7awXb+X vPmA== X-Gm-Message-State: ALoCoQm78DgjAJ6M17gvV9hp9VMI/CcCCFlVpPGekyerj1ieeXf8WYc6EUg31w03X/lA4CtkSyHa X-Received: by 10.112.46.225 with SMTP id y1mr6466537lbm.12.1398452704607; Fri, 25 Apr 2014 12:05:04 -0700 (PDT) Received: from kde4.my.domain (nat-server-217.15.27.35.static.futures.ru. [217.15.27.35]) by mx.google.com with ESMTPSA id q8sm9028680lbr.3.2014.04.25.12.05.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 25 Apr 2014 12:05:03 -0700 (PDT) From: Oleg Ginzburg To: freebsd-virtualization@freebsd.org Subject: bhyve behavior under cpuset_setaffinity Date: Fri, 25 Apr 2014 23:05:06 +0400 Message-ID: <11465971.5E2sZFzEpK@kde4.my.domain> User-Agent: KMail/4.12.4 (FreeBSD/11.0-CURRENT; KDE/4.12.4; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 19:05:13 -0000 Hi, I planned to look at the bhyve through PMC and found bad behavior if you apply cpuset mask for the bhyve process: guest process on the host begins to consume up to 40-50% without any load within guest. For example by executing follow command on bhyve pid: % cpuset -l 2 -p 3476 % top PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 3082 root 3 75 0 2075M 5016K CPU2 2 0:01 10.06% bhyve % ktrace -p 3476 % ktrace -C % kdump at this point shows only those entries: ... 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 1 RET ioctl 0 3476 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 3476 vcpu 1 RET ioctl 0 3476 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 0 RET ioctl 0 3476 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 3476 vcpu 1 RET ioctl 0 3476 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 3476 vcpu 1 RET ioctl 0 3476 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 3476 vcpu 0 RET ioctl 0 .. All process in the guest is very slow and any action adjust the load on the CPU>= 100% Without cpuset_setaffinity in the same conditions ktrace output is similar: .. 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 22506 vcpu 0 RET ioctl 0 22506 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 22506 vcpu 0 RET ioctl 0 22506 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 22506 vcpu 0 RET ioctl 0 22506 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 22506 vcpu 0 RET ioctl 0 22506 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 22506 vcpu 1 RET ioctl 0 22506 vcpu 0 RET ioctl 0 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 22506 vcpu 0 CALL ioctl(0x3,0xc0787601,0x7fffff9fce40) 22506 vcpu 1 RET ioctl 0 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 22506 vcpu 1 RET ioctl 0 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 22506 vcpu 1 RET ioctl 0 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) 22506 vcpu 1 RET ioctl 0 22506 vcpu 1 CALL ioctl(0x3,0xc0787601,0x7fffff7fbe40) .. but the cpu usage on the core is close to 5-10% wherein guest processes are very responsive. From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 25 19:11:48 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C122940; Fri, 25 Apr 2014 19:11:48 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id BE9F41B29; Fri, 25 Apr 2014 19:11:47 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 0957E12581; Sat, 26 Apr 2014 05:11:40 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTQ22979 (AUTH peterg@ptree32.com.au); Sat, 26 Apr 2014 05:11:38 +1000 Message-ID: <535AB368.9060101@freebsd.org> Date: Fri, 25 Apr 2014 12:11:36 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Roman Bogorodskiy Subject: Re: Understanding CPU and memory usage in Bhyve References: <20140421102138.GA6157@kloomba> <535557BC.8030300@freebsd.org> <20140424160816.GB3494@kloomba> In-Reply-To: <20140424160816.GB3494@kloomba> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 19:11:48 -0000 Hi Roman, > But the summary is: is there a way to figure out how much CPU time > bhyve and the guest spends on host CPUs N (N = 0, 1, ...)? I don't think FreeBSD records that for a process, and bhyve doesn't record the guest vCPU time on individual host CPUs (it's an aggregate over all CPUs). Do you know if Linux supports that ? later, From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 27 10:46:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 731A97EC for ; Sun, 27 Apr 2014 10:46:02 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E8F867C7 for ; Sun, 27 Apr 2014 10:46:01 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id ec20so4199886lab.40 for ; Sun, 27 Apr 2014 03:45:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=Bo5R78XCsmRM/abulXQe5pzmmSSnhI+zOl5GEdyq7TM=; b=OGII31cynY1hpYfyKq5q8hfS2hS5peC1MpBTwPGf5GcHHWpkpCid+/K3rRbFDqLGUR i1aqSg5/O4M/6hK9Gskja8J1a3NWAOVUXjBKx2JrZ4H+exInjpSxyDw8kk05jek5whj9 JAvSjw5gqAkhe5K7CCTpQw9eIL5BpYYYG5dFPDXZ8m/CJVt/SgsuD9SOBA3qw8ZruI0L 32UgQ5weC09pMBBGAZ7wPv5Sf4InqN/Muvlyy9q2Swhacl6lZn3BlhAETdcKB88mFE2D kIHx3894Pl7HdKlWH2cGwjyOeRlO5xQ99hfbukd2FkGxLtubkg/zyJeCmj5M/zvXf6sF iTXQ== X-Received: by 10.152.170.137 with SMTP id am9mr14255307lac.15.1398595559824; Sun, 27 Apr 2014 03:45:59 -0700 (PDT) Received: from kloomba ([95.104.135.172]) by mx.google.com with ESMTPSA id bm3sm14871756lbb.12.2014.04.27.03.45.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Apr 2014 03:45:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 27 Apr 2014 14:45:13 +0400 From: Roman Bogorodskiy To: freebsd-virtualization@FreeBSD.org Subject: [PATCH] Flexible vcpu pinning configuration Message-ID: <20140427104511.GA7804@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Pd0ReVV5GZGQvF3a" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2014 10:46:02 -0000 --Pd0ReVV5GZGQvF3a Content-Type: multipart/mixed; boundary="6c2NcOVqGQ03X4Wi" Content-Disposition: inline --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've created an initial version of the patch which allows more flexible vcpu pinning configuration. Current schema is: bhyve -p N=20 pins vcpu i to hostcpu N + i. The propsed extension is: bhyve -p N:M .... -p 0:1 -p 3:5 which pins vcpu N to host pcpu M. Option needs to be specified individually for each vcpu. So it works like that for me: sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... # sudo cpuset -g -t 100262 tid 100262 mask: 0 # sudo cpuset -g -t 100264 tid 100264 mask: 3 PS I used cpumat_t* array to store these values instead of int, because if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1) supports, e.g.: "1:2-5". The questions are: - Is it OK to chance '-p' arg syntax or it's better to introduce a new one? - Is the syntax OK (currently: 'vcpu:pcpu', later 'vcpu:pcpuN-pcpuM,pcpuX")? Roman Bogorodskiy --6c2NcOVqGQ03X4Wi Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="bhyve_vcpupin.diff" Content-Transfer-Encoding: quoted-printable Index: bhyverun.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- bhyverun.c (revision 264921) +++ bhyverun.c (working copy) @@ -83,7 +83,6 @@ =20 int guest_ncpus; =20 -static int pincpu =3D -1; static int guest_vmexit_on_hlt, guest_vmexit_on_pause, disable_x2apic; static int virtio_msix =3D 1; =20 @@ -119,6 +118,8 @@ int mt_vcpu;=09 } mt_vmm_info[VM_MAXCPU]; =20 +cpuset_t *vcpumap[VM_MAXCPU] =3D { NULL }; + static void usage(int code) { @@ -146,6 +147,56 @@ exit(code); } =20 +static int +pincpu_parse(const char *cpupin) +{ + char *token, *string; + size_t i =3D 0; + int vcpu =3D -1, pcpu =3D -1, ret =3D -1; + cpuset_t *mask; + + if ((string =3D strdup(cpupin)) =3D=3D NULL) { + fprintf(stderr, "strdup failed: %d\n", errno); + return -1; + } + + while ((token =3D strsep(&string, ":")) !=3D NULL) { + switch (i) { + case 0: + vcpu =3D atoi(token); + break; + case 1: + pcpu =3D atoi(token); + break; + default: + fprintf(stderr, "invalid format: %s\n", token); + goto cleanup; + } + i++; + } + + if (vcpu =3D=3D -1 || pcpu =3D=3D -1) { + fprintf(stderr, "invalid format: %s\n", cpupin); + goto cleanup; + } + + if ((mask =3D malloc(sizeof(cpuset_t))) =3D=3D NULL) { + fprintf(stderr, "malloc failed: %d\n", errno); + goto cleanup; + } + CPU_ZERO(mask); + CPU_SET(pcpu, mask); + + vcpumap[vcpu] =3D mask; + + ret =3D 0; + +cleanup: + free(string); + + return ret; +} + void * paddr_guest2host(struct vmctx *ctx, uintptr_t gaddr, size_t len) { @@ -479,15 +530,13 @@ static void vm_loop(struct vmctx *ctx, int vcpu, uint64_t rip) { - cpuset_t mask; int error, rc, prevcpu; enum vm_exitcode exitcode; =20 - if (pincpu >=3D 0) { - CPU_ZERO(&mask); - CPU_SET(pincpu + vcpu, &mask); + if (vcpumap[vcpu] !=3D NULL) { error =3D pthread_setaffinity_np(pthread_self(), - sizeof(mask), &mask); + sizeof(cpuset_t), + vcpumap[vcpu]); assert(error =3D=3D 0); } =20 @@ -622,7 +671,10 @@ bvmcons =3D 1; break; case 'p': - pincpu =3D atoi(optarg); + if (pincpu_parse(optarg) < 0) { + errx(EX_USAGE, "invalid cpu pin " + "configuration '%s'", optarg); + } break; case 'c': guest_ncpus =3D atoi(optarg); --6c2NcOVqGQ03X4Wi-- --Pd0ReVV5GZGQvF3a Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTXN+3AAoJEMltX/4IwiJqNHUH/3AdX0YTgLg45PFDyrwV04Gr VfQIwTEJ1FneeY2FCJ27cTYuJRGI6T/uYuZO5y0r1XPKXOc8tvbF4kT7dOr/nx80 3Ipoh0m+sgQE92zfuKpn6FTSzNkEv8Qo3y3a1CDueX+f+8SfzhwqPNpRTBzCekNW XIHmqQHyan6XrQ8lvWx1cs+502ZSsQXd5+s5DqM527u1pHUJ8IVrFzidFVSPgwO+ medP6M4GEpTvjBTCp75LhgL4q3o9hxu1t9f0MMateQha+H7h8s33e378dBuip+Nw 212/bg3VU2W4cVZ528DfQz9KlnjPZrzTTLZhBTfq5X21Bz1dSuaydnDoPjoUWfA= =1fNw -----END PGP SIGNATURE----- --Pd0ReVV5GZGQvF3a-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 27 10:55:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA0C9884; Sun, 27 Apr 2014 10:55:48 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 301D9877; Sun, 27 Apr 2014 10:55:48 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id gl10so677490lab.38 for ; Sun, 27 Apr 2014 03:55:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=lGzqBz0hT6BldItqxi4CbtDYKy5sspK+ccDgTEGZN8k=; b=V28zrsjAntd4J/NyULlE3VfPcShKjGN46CdO64W6EGITsFRUaQIcHhu6fl8Epy2N3Y DR9TKEsu6qUvU6D8JSZJwNEsOfsEuee2NFKif//vgmF948byrebJN6OoeKNjq09RzRzb Jydb6EQOVH0hCV+Qk2tN5RecrpN2iLJrqXoiWu7iHM/oTT0ViIIhEBMWhEKTo8xkq9oZ SSH+R92VJWUQclbGhYVLaeGghGzooDAPzu4xYzIqc5b8X0oCsdtbnRADb3P1j1L3CpUS WWBoE9yWIKUdd7gry4qFpA9dWHedQchAnwmbJc0K9aQ43+pjj38JU2RwPCDsY44W2V4n tquA== X-Received: by 10.112.85.6 with SMTP id d6mr13361331lbz.8.1398596145851; Sun, 27 Apr 2014 03:55:45 -0700 (PDT) Received: from kloomba ([95.104.135.172]) by mx.google.com with ESMTPSA id jm3sm14883484lbc.29.2014.04.27.03.55.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Apr 2014 03:55:44 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 27 Apr 2014 14:55:03 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: Understanding CPU and memory usage in Bhyve Message-ID: <20140427105502.GB7804@kloomba> References: <20140421102138.GA6157@kloomba> <535557BC.8030300@freebsd.org> <20140424160816.GB3494@kloomba> <535AB368.9060101@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H8ygTp4AXg6deix2" Content-Disposition: inline In-Reply-To: <535AB368.9060101@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2014 10:55:48 -0000 --H8ygTp4AXg6deix2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > Hi Roman, >=20 > > But the summary is: is there a way to figure out how much CPU time > > bhyve and the guest spends on host CPUs N (N =3D 0, 1, ...)? >=20 > I don't think FreeBSD records that for a process, and bhyve doesn't=20 > record the guest vCPU time on individual host CPUs (it's an aggregate=20 > over all CPUs). >=20 > Do you know if Linux supports that ? As far as I understand, Linux supports that through Cgroups, specifically, quoting: http://docs.fedoraproject.org/en-US/Fedora/15/html/Resource_Management_Guid= e/sec-cpuacct.html cpuacct.usage_percpu reports the CPU time (in nanoseconds) consumed on each CPU by all tasks in this cgroup (including tasks lower in the hierarchy).=20 Roman Bogorodskiy --H8ygTp4AXg6deix2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTXOIGAAoJEMltX/4IwiJqpOsIAKxrX44phEjSKybFd6F+j/Lm GBUT4PHzDZPBJOKl/5+TRvxTZvfjNt/zIxjAZUr38QTzw5kxsRK9ekl4pWmE2IZj FDFOTxYGGPOpvaGtrUuzroJ4q9JUyUw3m9KYHCuLMcWYkvGulf3muGmtUNOIywJd KfmXqTrsLxFlXwq3UYEFF6pIprcTctyxS3BoL3zgRDCS4npJCLI1Z/bnEpZDhFY6 g/p3pFzSr6x9KVMsfXggGxxG628kKLDwC4GI2rfvUgnlMQ4neEzGzNURBJ9r/Ci4 XQG90ZYYHLAdOprPjSwKKlQlleZo/uh7mfm1sg2oAKo+NTDwcj4NOunesj4UroI= =+Me1 -----END PGP SIGNATURE----- --H8ygTp4AXg6deix2-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 27 15:21:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2551B8F for ; Sun, 27 Apr 2014 15:21:06 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8582E1DC2 for ; Sun, 27 Apr 2014 15:21:06 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 32CE012579; Mon, 28 Apr 2014 01:20:59 +1000 (EST) Received: from Peters-MacBook-Pro.local (m208-191.dsl.rawbw.com [198.144.208.191]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTQ84030 (AUTH peterg@ptree32.com.au); Mon, 28 Apr 2014 01:20:57 +1000 Message-ID: <535D2055.6060107@freebsd.org> Date: Sun, 27 Apr 2014 08:20:53 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Oleg Ginzburg Subject: Re: bhyve behavior under cpuset_setaffinity References: <11465971.5E2sZFzEpK@kde4.my.domain> In-Reply-To: <11465971.5E2sZFzEpK@kde4.my.domain> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2014 15:21:06 -0000 Hi Oleg, > For example by executing follow command on bhyve pid: > > % cpuset -l 2 -p 3476 > 3476 vcpu 0 RET ioctl 0 ... > 3476 vcpu 1 RET ioctl 0 Looks like you have a 2 vCPU guest, but the cpuset only contains a single host CPU. What happens here under load is that you will start hitting "lock-holder preemption", where a vCPU can be preempted while holding a lock, and the other vCPU will spin trying to acquire it. Using the "-P" option with bhyve will help this somewhat, though I'd recommend using >= 2 CPUs with the cpuset command. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 28 11:06:56 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0C6D60D for ; Mon, 28 Apr 2014 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8DB9E1AB9 for ; Mon, 28 Apr 2014 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3SB6uVN086316 for ; Mon, 28 Apr 2014 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3SB6ufw086314 for freebsd-virtualization@FreeBSD.org; Mon, 28 Apr 2014 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 28 Apr 2014 11:06:56 GMT Message-Id: <201404281106.s3SB6ufw086314@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 29 16:21:49 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4AF26F2; Tue, 29 Apr 2014 16:21:49 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5633519EC; Tue, 29 Apr 2014 16:21:49 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so499804qcz.33 for ; Tue, 29 Apr 2014 09:21:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=5bAQ6sW1u4tExIq9dvabmJifrMdhE8b/12Zg4JDCEY8=; b=k+63GJrLAlsYOFPbKCb8Cb+gX3cGzHgnmVn+nW+uO3KmfqURWNv0CM3yM1w2og/o83 QND1DVKxGQm0RIHfliFPGoB0Rkcg7ookm3/k2zsjHd4vd6c6b3iDbPdanhTqAVR1eUOV ccGBbGQBTUp5Ox3IGTmj3w8jPFyfbkI0dj7mHRjrgaMtA6jaFwj2uG80j8FjqDAsK013 rBC7Zggyjkd+hw8avkjdLeSZej+KOMTd7Gu1OdVw1TYxmRdo/pcmZKQxYOPi3saiMgqQ H+7T3v+eVqWewYFKKwiwpQQPR6PvFehqG8fPZq0FaLhmdIpkk0q3kLhOkUuMsH1v6/LW w96Q== MIME-Version: 1.0 X-Received: by 10.140.101.201 with SMTP id u67mr196278qge.107.1398788508476; Tue, 29 Apr 2014 09:21:48 -0700 (PDT) Received: by 10.140.47.201 with HTTP; Tue, 29 Apr 2014 09:21:48 -0700 (PDT) In-Reply-To: <20140427104511.GA7804@kloomba> References: <20140427104511.GA7804@kloomba> Date: Tue, 29 Apr 2014 09:21:48 -0700 Message-ID: Subject: Re: [PATCH] Flexible vcpu pinning configuration From: Neel Natu To: Roman Bogorodskiy Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:21:49 -0000 Hi Roman, On Sun, Apr 27, 2014 at 3:45 AM, Roman Bogorodskiy wrote: > I've created an initial version of the patch which allows more flexible > vcpu pinning configuration. > > Current schema is: > > bhyve -p N > > pins vcpu i to hostcpu N + i. > > The propsed extension is: > > bhyve -p N:M .... -p 0:1 -p 3:5 > > which pins vcpu N to host pcpu M. Option needs to be specified > individually for each vcpu. > > So it works like that for me: > > sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... > > # sudo cpuset -g -t 100262 > tid 100262 mask: 0 > # sudo cpuset -g -t 100264 > tid 100264 mask: 3 > > PS I used cpumat_t* array to store these values instead of int, because > if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1) > supports, e.g.: "1:2-5". > > The questions are: > > - Is it OK to chance '-p' arg syntax or it's better to introduce a new > one? > I think we can reuse the "-p" option unless anybody objects vociferously. > - Is the syntax OK (currently: 'vcpu:pcpu', later > 'vcpu:pcpuN-pcpuM,pcpuX")? Yup, I think that works fine. The patch looks good in general but I have a few comments: - Scope of 'vcpupmap[]' should be restricted to 'static'. - usage() and man page need to be updated. - pincpu_parse(): The option parsing can be made much easier by using: if (sscanf(str, "%d:%d", &vcpu, &pcpu) == 2) { /* success */ } else { return (-1); } If the same vcpu is specified multiple times then we should malloc(sizeof(cpuset_t)) only the first time: if (vcpumap[vcpu] != NULL) mask = vcpumap[vcpu]; else mask = malloc(sizeof(cpuset_t)); We need to range-check 'vcpu' before using it as an index into the 'vcpumap[]' array. best Neel > > Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 29 16:57:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8FFA37D for ; Tue, 29 Apr 2014 16:57:48 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3BA5D1D0A for ; Tue, 29 Apr 2014 16:57:48 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id 10so364280lbg.37 for ; Tue, 29 Apr 2014 09:57:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=/L5PveGb7c0ymYPd8CRaWCpzy+qRK/8vHlIfvNfrjo8=; b=HokS5k/TH8+6K8Ce7vj5VpUzbOXMQIqRH2dFso07b0O/9xMcEJTbaB9a7QQzLJ4SRp PT347uStspuMhySBZUOgXBgvcQJc9z+2l7kPeNL82xFPgqAkDrVwPF/Yv0br6DYx+VRI nk+UPl7+nrG4TymzAmPmjI+arxsJ48MgS+XSD3iY6kpQCiWb/pYdfWCaovZJZ/gR4mi7 1iUyOJ+r307zSmG9pZfKljGwOSNi24OfR0rAVIWeIFNihgiEkIBBjM+WMaxQjcPTTttt kJYP4GeiO1t1h9UMRhyoyOhZq9c0K+eCUw8VODDmCtnobEy5fEbHzit+a9mZeFkK9xFb ThwA== X-Received: by 10.152.19.65 with SMTP id c1mr175675lae.35.1398790666086; Tue, 29 Apr 2014 09:57:46 -0700 (PDT) Received: from kloomba ([95.104.141.159]) by mx.google.com with ESMTPSA id lc3sm23200677lbb.21.2014.04.29.09.57.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Apr 2014 09:57:45 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 29 Apr 2014 20:57:00 +0400 From: Roman Bogorodskiy To: Neel Natu Subject: Re: [PATCH] Flexible vcpu pinning configuration Message-ID: <20140429165659.GB2170@kloomba> References: <20140427104511.GA7804@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tsOsTdHNUZQcU9Ye" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:57:48 -0000 --tsOsTdHNUZQcU9Ye Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Neel Natu wrote: > Hi Roman, >=20 > On Sun, Apr 27, 2014 at 3:45 AM, Roman Bogorodskiy wr= ote: > > I've created an initial version of the patch which allows more flexible > > vcpu pinning configuration. > > > > Current schema is: > > > > bhyve -p N > > > > pins vcpu i to hostcpu N + i. > > > > The propsed extension is: > > > > bhyve -p N:M .... -p 0:1 -p 3:5 > > > > which pins vcpu N to host pcpu M. Option needs to be specified > > individually for each vcpu. > > > > So it works like that for me: > > > > sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... > > > > # sudo cpuset -g -t 100262 > > tid 100262 mask: 0 > > # sudo cpuset -g -t 100264 > > tid 100264 mask: 3 > > > > PS I used cpumat_t* array to store these values instead of int, because > > if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1) > > supports, e.g.: "1:2-5". > > > > The questions are: > > > > - Is it OK to chance '-p' arg syntax or it's better to introduce a new > > one? > > >=20 > I think we can reuse the "-p" option unless anybody objects vociferously. >=20 > > - Is the syntax OK (currently: 'vcpu:pcpu', later > > 'vcpu:pcpuN-pcpuM,pcpuX")? >=20 > Yup, I think that works fine. >=20 > The patch looks good in general but I have a few comments: >=20 > - Scope of 'vcpupmap[]' should be restricted to 'static'. >=20 > - usage() and man page need to be updated. >=20 > - pincpu_parse(): > The option parsing can be made much easier by using: >=20 > if (sscanf(str, "%d:%d", &vcpu, &pcpu) =3D=3D 2) { > /* success */ > } else { > return (-1); > } >=20 > If the same vcpu is specified multiple times then we should > malloc(sizeof(cpuset_t)) only the first time: >=20 > if (vcpumap[vcpu] !=3D NULL) > mask =3D vcpumap[vcpu]; > else > mask =3D malloc(sizeof(cpuset_t)); >=20 > We need to range-check 'vcpu' before using it as an index into the > 'vcpumap[]' array. Thanks for the comments, I'll make the fixes you pointed and re-send the patch. Roman Bogorodskiy --tsOsTdHNUZQcU9Ye Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTX9nbAAoJEMltX/4IwiJqJmsH/0/KxxVBn8stW50t2RBgUEgV g7ojE9yKGICuuOr/iFvjf+wRdvzqGbniU+jMA2/KtlRRSRdodgwxuOkjiLHagENn IF82sT6tWqaqx0QqclnHeW0Bj4YUYpP6n0sm719Nk5BPIIsAmip8jtRE6eYTy2md 8kC4FRTXFIGIR9eLQWvxzPLhAOM3o9b2IzdqwcBdQWZ9cSZlruXcL2vUPJ0Rftjm PYSKqodYa0RYBzdilBJ0HsfHy7ONPqx4EdDURUvTu5Ynp38Q+ks7OrzYRsSd36uQ mO230JeUTpNMukYd4mRkrtpxuTaC3yr3taj8xQNlKZpR5MUCxets+NRmyIfq5uY= =ZTHI -----END PGP SIGNATURE----- --tsOsTdHNUZQcU9Ye-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 29 21:12:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF0AB9F4 for ; Tue, 29 Apr 2014 21:12:42 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CF121A66 for ; Tue, 29 Apr 2014 21:12:42 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id hu19so1060244vcb.37 for ; Tue, 29 Apr 2014 14:12:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=eLjhHL/jZvsNp55eBsgyqdoxuO6NsiKsMlGUpCM4J38=; b=ITgbUzGBVtUHpW8+y66Rly6cm+6gFAEbyWB1ILwZ/utlGyYM5TKYuGV6Cpaa9z23Tb yDZ/cc26l4tVRa/aYSvbK4gw1oP5FaXMwXplL1ulO/xc5YHI585jcQwZxhcRvASaLglh glSmoKNSigk3EyAl/btivAbrKjcLgx5H0U8BA+IS2GR1xw08nuXqdWnQL3vOZDL7Cu61 EPMf/Y8+Y3nC8rFKMLQ+CYDBe7yDHGmTl5e7S1ZVSJ7QpgKvO1tcnQCTakjHFN64Uux3 A4KiOqNNWBEld2+pylJOdmXre0P+k7MQk9SfaykST2vNws0wJ6P86A0513JaCubbVHCK ojtw== X-Received: by 10.58.146.5 with SMTP id sy5mr4086veb.43.1398805961650; Tue, 29 Apr 2014 14:12:41 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.2.225 with HTTP; Tue, 29 Apr 2014 14:12:21 -0700 (PDT) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Tue, 29 Apr 2014 23:12:21 +0200 X-Google-Sender-Auth: fZehJdbUqLM-g0CvrHqKUgW1TJo Message-ID: Subject: bhyve and serial port emulation problem To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 21:12:43 -0000 Hi, I'm using the same nanobsd image on Virtualbox and bhyve (-CURRENT r265101). The nanobsd image I'm using is configured for serial port as default console: [root@nanobsd]~# cat /boot.config -S38400 -h [root@nanobsd]~# grep ttyu0 /etc/ttys ttyu0 "/usr/libexec/getty std.38400" vt100 on secure The Virtualbox VM is configured for creating a serial pipe: VBoxManage modifyvm nanobsd --uart1 0x3F8 4 --uartmode1 server /tmp/nanobsd.serial and I connect to this guest using "socat unix-connect:/tmp/nanobsd.serial stdio,raw,echo=0,icanon=0" => And it's works without problem. The bhyve is configured with a com1 redirecting to a nmdm device: "-l com1,/dev/nmdm1" and I connect to this guest using "cu -l /dev/nmdm1B" => But this bhyve console didn't works correctly: I can't use tool like less/more (even ssh-keygen can't ask for a passphrase). If I run truss with these tools, It seems they hang for opening /dev/tty. Some differences between virtualbox and bhyve (don't know if it can help): [root@vbox_guest]~# kenv | grep console comconsole_pcidev="" comconsole_port="1016" comconsole_speed="9600" console="comconsole" [root@bhyve_guest]~# kenv | grep console console="userboot" Regards, Olivier From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 30 17:33:12 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5D639F0 for ; Wed, 30 Apr 2014 17:33:12 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A750B1D4C for ; Wed, 30 Apr 2014 17:33:12 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id j5so2171496qga.22 for ; Wed, 30 Apr 2014 10:33:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=9TxDIu2xV0mJlOEyp73G0O/9wRPq0uWoOI2gXxmYp+g=; b=W5B5hBEHHjxfnQjO9/K0phXP0uULcgzcjzWdCOU1avkZyNa1VWCHxq/XOYE7h5bXWG OtLaqwAf/DEoU3K03JHjqJMQGAbsUovAjTJxEV0wTan9H/HqouEZ7db3PblHcw1fMkUT HvlujZMKHNLJUEluGOJsHGvZ/hgZDTsptstk8kdhahVf0tsZjPXESFd3EZZfFIH94iZu uFpv/vAPFRkev+jHm5/7eE/3JcBIFT0BhEkgeJ1iio8xhRGQnqvi6tmI2UgwmeXwQkwx jU+cWLSCsH5JLd0tzvldSd3KKiFY+bRZOPz//CozCQQGFEgC5bSljLUUIwPYfkogTv2w 9GSA== MIME-Version: 1.0 X-Received: by 10.224.72.12 with SMTP id k12mr7230406qaj.81.1398879191699; Wed, 30 Apr 2014 10:33:11 -0700 (PDT) Received: by 10.140.47.201 with HTTP; Wed, 30 Apr 2014 10:33:11 -0700 (PDT) In-Reply-To: References: Date: Wed, 30 Apr 2014 10:33:11 -0700 Message-ID: Subject: Re: bhyve and serial port emulation problem From: Neel Natu To: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 17:33:13 -0000 Hi Olivier, On Tue, Apr 29, 2014 at 2:12 PM, Olivier Cochard-Labb=C3=A9 wrote: > Hi, > > I'm using the same nanobsd image on Virtualbox and bhyve (-CURRENT r26510= 1). > > The nanobsd image I'm using is configured for serial port as default > console: > > [root@nanobsd]~# cat /boot.config > -S38400 -h > [root@nanobsd]~# grep ttyu0 /etc/ttys > ttyu0 "/usr/libexec/getty std.38400" vt100 on secure > > The Virtualbox VM is configured for creating a serial pipe: > VBoxManage modifyvm nanobsd --uart1 0x3F8 4 --uartmode1 server > /tmp/nanobsd.serial > and I connect to this guest using "socat unix-connect:/tmp/nanobsd.serial > stdio,raw,echo=3D0,icanon=3D0" > =3D> And it's works without problem. > > The bhyve is configured with a com1 redirecting to a nmdm device: "-l > com1,/dev/nmdm1" > and I connect to this guest using "cu -l /dev/nmdm1B" > > =3D> But this bhyve console didn't works correctly: I can't use tool like > less/more (even ssh-keygen can't ask for a passphrase). > If I run truss with these tools, It seems they hang for opening /dev/tty. > > Some differences between virtualbox and bhyve (don't know if it can help)= : > > [root@vbox_guest]~# kenv | grep console > comconsole_pcidev=3D"" > comconsole_port=3D"1016" > comconsole_speed=3D"9600" > console=3D"comconsole" > > [root@bhyve_guest]~# kenv | grep console > console=3D"userboot" > Can you share the command line options you pass the bhyve(8)? best Neel > Regards, > > Olivier > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 30 19:13:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 231143D9; Wed, 30 Apr 2014 19:13:32 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C76041A36; Wed, 30 Apr 2014 19:13:31 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id x13so2424980qcv.29 for ; Wed, 30 Apr 2014 12:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=CS74NVYD3jNmnrRpYdQqnImbBVuzkNCzufQm1jjjPP4=; b=FlJ52sqHN3RytlaHep+Hm9rDuXYT2l63tzvbhwX2MSWQiua6Y7vNhtUySqimVP+Tia lxYaVx1bqkPxJAMbhn8EJx2mk5aGdmxFODP0SUOhmk89DBFFhCQxRbi+wBS7hQCmBCuH Q93bV2+seL4ewS93k42T/Ycpi5k0hhpXmSuTZLhaDI03kf1kfJMeI7YGf9TLW5ZBtsrF V3O1BzCUPn8yKtIrpE242byZk9W2y1u0wTASZocG2goFUdaUOFuivLZPpaPmcr7nWQlg TTYcUp0Pr1hQr7zWlyDGL788O5/FAxGEqFEm0Gr3NCod12YywAbWNX9oKVicZ4HR5V6K mKUw== MIME-Version: 1.0 X-Received: by 10.140.98.116 with SMTP id n107mr7444853qge.93.1398885210780; Wed, 30 Apr 2014 12:13:30 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.49.239 with HTTP; Wed, 30 Apr 2014 12:13:30 -0700 (PDT) In-Reply-To: <7A221B2C-6439-46E7-8343-51945B5626A7@FreeBSD.org> References: <6a7255a1fdfc41c79e38532bf267a06f@BL2PR03MB210.namprd03.prod.outlook.com> <7A221B2C-6439-46E7-8343-51945B5626A7@FreeBSD.org> Date: Wed, 30 Apr 2014 15:13:30 -0400 X-Google-Sender-Auth: uL-DRMxEtOmIYjc-nXlyVg6_e-U Message-ID: Subject: Re: Duplicating code from NetBSD From: Ed Maste To: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 19:13:32 -0000 On 17 April 2014 09:00, Edward Tomasz Napiera=C5=82a wr= ote: > Wiadomo=C5=9B=C4=87 napisana przez Abhishek Gupta (LIS) w dniu 17 kwi 201= 4, o godz. 03:01: >> Hi folks, >> >> Newbie question so please excuse the naivety. If someone were to copy a = few functions from NetBSD in to their FreeBSD kernel mode drivers then is i= t still possible to submit the code in to the FreeBSD tree under the FreeBS= D license? If not then would the community accept the copied code cleanly s= eparated out in a file and licensed under the NetBSD license? Please let me= know if there is an acceptable procedure to do this or if this is a comple= te NONO. > > It all depends on the license. If the code is licensed under standard BS= D > license, then it's perfectly ok to commit it to FreeBSD. In fact, a lot = of things > got ported from NetBSD, just grep for it in sources. It sounds to me as if Abhishek is proposing copying functions from NetBSD into a FreeBSD source file for their driver. While there's probably an acceptable way to do this (while maintaining the copyright / license text), it seems like it would be more awkward and confusing, and also make future updates more difficult. I'd recommend leaving the code in separate files. Abhishek, do you have a pointer to the functions you're interested in using= ? From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 30 22:28:24 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 292338DB for ; Wed, 30 Apr 2014 22:28:24 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9E7D1FAA for ; Wed, 30 Apr 2014 22:28:23 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id sa20so3067324veb.1 for ; Wed, 30 Apr 2014 15:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=nc+VIshVW6y9YTqxSyXOMRzaJyrxsSvsKhn5ApepSCs=; b=KHuKKUC+qmVGoP6wcJFsnohWVdxY2TAhjFfvQGTFy2DRhvEfSkEZk65dCBLQvlCgaX lESDxMviOBBCsdLEEQiNgkVUYSLMC0dOBYTfl2Lf+oxSMCvFAYV1IZXQ1Eyy9XtNzSfY AM8zes/kfXALWDpEql1S6R0p98o0McifGhIj5rWAhvj/JOgJLaOBOlfmfN3acKGONBzY uKwbPOlCgtxJMN27NMfyNJ8yrTjpGUiADQXPLvVE3B0B7jHk1OO6X50Mj1qGKusDEnLl rK7IDK2GjDgeMrHi9YPgJcBwlp9MK/5KPN46f+C3YmM2UHb6lNOViaMlR13/kPBjLSIb oLKg== X-Received: by 10.52.123.39 with SMTP id lx7mr4563556vdb.22.1398896902898; Wed, 30 Apr 2014 15:28:22 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.2.225 with HTTP; Wed, 30 Apr 2014 15:28:02 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Thu, 1 May 2014 00:28:02 +0200 X-Google-Sender-Auth: yaerNLq-KHhCcDyKhA0NQUmIfgo Message-ID: Subject: Re: bhyve and serial port emulation problem To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 22:28:24 -0000 On Wed, Apr 30, 2014 at 7:33 PM, Neel Natu wrote: > Hi Olivier, > > Can you share the command line options you pass the bhyve(8)? > > Hi Neel, sure, here it is: bhyveload -m 256M -d /tmp/BSDRP/BSDRP_1 -c /dev/nmdm1A BSDRP_1 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc -s 2:0,virtio-net,tap0,mac=58:9c:fc:01:00:01 -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_1 -l com1,/dev/nmdm1A BSDRP_1 Regards, Olivier From owner-freebsd-virtualization@FreeBSD.ORG Thu May 1 14:31:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BA4267E for ; Thu, 1 May 2014 14:31:02 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4B461203 for ; Thu, 1 May 2014 14:31:01 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id pn19so2222929lab.34 for ; Thu, 01 May 2014 07:30:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ffIQzEbPFZY5Pxw+UH6fGp5Q4a4W82siJGKwgDiP5/U=; b=naA+ZcUd0YqlZ/eInUrniOF8uDuyG5/6hQbLuSLqu+DTyFzqx8R0RoaMDpYLyiklkH tbDU/a/a8yVTOUyu2HXGqYg5CR2dbu4CTAbRS1HVbf1gG/NSBZMcsg/JdeC7Tp2PFure vzP6fiiRhUa/R/gtMcy3wpep2LLBCXKpBeSvMHcmSfM658Dz3jH+5SemFHBHyZA5oclc /8gag3G1OcGQ0lvVDRrHngfVVJDeQNzorqDJYrOrjdrcaaqZkf7maXSEvGXEoGHgiM/C uTTrMRljm0utZXQoSp9XoohVoKz+0hGAEmVITl/TtoH3epXVTF5WT4jPLxHsYTOo/CiM mVvQ== X-Received: by 10.152.1.8 with SMTP id 8mr7790666lai.1.1398954659735; Thu, 01 May 2014 07:30:59 -0700 (PDT) Received: from kloomba ([95.104.133.236]) by mx.google.com with ESMTPSA id jp6sm30073349lbc.15.2014.05.01.07.30.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 May 2014 07:30:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 1 May 2014 18:30:06 +0400 From: Roman Bogorodskiy To: Neel Natu Subject: Re: [PATCH] Flexible vcpu pinning configuration Message-ID: <20140501143005.GA91029@kloomba> References: <20140427104511.GA7804@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tjCHc7DPkfUGtrlw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 14:31:02 -0000 --tjCHc7DPkfUGtrlw Content-Type: multipart/mixed; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Neel Natu wrote: > Hi Roman, >=20 > On Sun, Apr 27, 2014 at 3:45 AM, Roman Bogorodskiy wr= ote: > > I've created an initial version of the patch which allows more flexible > > vcpu pinning configuration. > > > > Current schema is: > > > > bhyve -p N > > > > pins vcpu i to hostcpu N + i. > > > > The propsed extension is: > > > > bhyve -p N:M .... -p 0:1 -p 3:5 > > > > which pins vcpu N to host pcpu M. Option needs to be specified > > individually for each vcpu. > > > > So it works like that for me: > > > > sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... > > > > # sudo cpuset -g -t 100262 > > tid 100262 mask: 0 > > # sudo cpuset -g -t 100264 > > tid 100264 mask: 3 > > > > PS I used cpumat_t* array to store these values instead of int, because > > if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1) > > supports, e.g.: "1:2-5". > > > > The questions are: > > > > - Is it OK to chance '-p' arg syntax or it's better to introduce a new > > one? > > >=20 > I think we can reuse the "-p" option unless anybody objects vociferously. >=20 > > - Is the syntax OK (currently: 'vcpu:pcpu', later > > 'vcpu:pcpuN-pcpuM,pcpuX")? >=20 > Yup, I think that works fine. >=20 > The patch looks good in general but I have a few comments: >=20 > - Scope of 'vcpupmap[]' should be restricted to 'static'. >=20 > - usage() and man page need to be updated. >=20 > - pincpu_parse(): > The option parsing can be made much easier by using: >=20 > if (sscanf(str, "%d:%d", &vcpu, &pcpu) =3D=3D 2) { > /* success */ > } else { > return (-1); > } >=20 > If the same vcpu is specified multiple times then we should > malloc(sizeof(cpuset_t)) only the first time: >=20 > if (vcpumap[vcpu] !=3D NULL) > mask =3D vcpumap[vcpu]; > else > mask =3D malloc(sizeof(cpuset_t)); >=20 > We need to range-check 'vcpu' before using it as an index into the > 'vcpumap[]' array. >=20 > best > Neel Attached an updated patch. I'm still inclined to use something like parselist() from usr.bin/cpuset/cpuset.c, but I don't want to copy/paste and I don't know where it'd make sense to move it so it was usable outside of cpuset? PS While reading bhyverun.c, I think I spotted a typo: in fbsdrun_deletecpu= () error message says fprintf(stderr, "addcpu: .... Should it be "deletecpu:" = instead? Roman Bogorodskiy --YiEDa0DAkWCtVeE4 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="bhyve_vcpupin2.diff" Content-Transfer-Encoding: quoted-printable Index: bhyve.8 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- bhyve.8 (revision 264921) +++ bhyve.8 (working copy) @@ -78,12 +78,14 @@ allow a remote kernel kgdb to be relayed to the guest kernel gdb stub via a local IPv4 address and this port. This option will be deprecated in a future version. -.It Fl p Ar pinnedcpu +.It Fl p Ar n:m Force guest virtual CPUs to be pinned to host CPUs. Virtual CPU .Em n is pinned to host CPU -.Em pinnedcpu+n . +.Em m . +This option could be specified multiple times to set pinning for each +Virtual CPU and to pin Virtual CPU to more than one host CPU. .It Fl P Force the guest virtual CPU to exit when a PAUSE instruction is detected. .It Fl W Index: bhyverun.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- bhyverun.c (revision 264921) +++ bhyverun.c (working copy) @@ -83,7 +83,6 @@ =20 int guest_ncpus; =20 -static int pincpu =3D -1; static int guest_vmexit_on_hlt, guest_vmexit_on_pause, disable_x2apic; static int virtio_msix =3D 1; =20 @@ -119,6 +118,8 @@ int mt_vcpu;=09 } mt_vmm_info[VM_MAXCPU]; =20 +static cpuset_t *vcpumap[VM_MAXCPU] =3D { NULL }; + static void usage(int code) { @@ -125,12 +126,12 @@ =20 fprintf(stderr, "Usage: %s [-aehwAHIPW] [-g ] [-s ] [-S ]\n" - " %*s [-c vcpus] [-p pincpu] [-m mem] [-l ] \n" + " %*s [-c vcpus] [-p vcpu:hostcpu] [-m mem] [-l ] \n" " -a: local apic is in XAPIC mode (default is X2APIC)\n" " -A: create an ACPI table\n" " -g: gdb port\n" " -c: # cpus (default 1)\n" - " -p: pin vcpu 'n' to host cpu 'pincpu + n'\n" + " -p: pin vcpu 'n' to host cpu 'm'\n" " -H: vmexit from the guest on hlt\n" " -P: vmexit from the guest on pause\n" " -W: force virtio to use single-vector MSI\n" @@ -146,6 +147,53 @@ exit(code); } =20 +static int +pincpu_parse(const char *cpupin) +{ + char *string; + int vcpu =3D -1, pcpu =3D -1, ret =3D -1; + + if ((string =3D strdup(cpupin)) =3D=3D NULL) { + fprintf(stderr, "strdup failed: %d\n", errno); + return -1; + } + + if (sscanf(string, "%d:%d", &vcpu, &pcpu) !=3D 2) { + fprintf(stderr, "invalid format: %s\n", string); + goto cleanup; + } + + if (vcpu < 0 || vcpu > VM_MAXCPU - 1) { + fprintf(stderr, "invalid vcpu value '%d', " + "should be from 0 to %d\n", + vcpu, VM_MAXCPU - 1); + goto cleanup; + } + + if (pcpu < 0 || pcpu > CPU_SETSIZE) { + fprintf(stderr, "invalid pcpu value '%d', " + "should be from 0 to %d\n", + pcpu, CPU_SETSIZE); + goto cleanup; + } + + if (vcpumap[vcpu] =3D=3D NULL) { + if ((vcpumap[vcpu] =3D malloc(sizeof(cpuset_t))) =3D=3D NULL) { + fprintf(stderr, "malloc failed: %d\n", errno); + goto cleanup; + } + CPU_ZERO(vcpumap[vcpu]); + } + CPU_SET(pcpu, vcpumap[vcpu]); + + ret =3D 0; + +cleanup: + free(string); + + return ret; +} + void * paddr_guest2host(struct vmctx *ctx, uintptr_t gaddr, size_t len) { @@ -479,15 +527,13 @@ static void vm_loop(struct vmctx *ctx, int vcpu, uint64_t rip) { - cpuset_t mask; int error, rc, prevcpu; enum vm_exitcode exitcode; =20 - if (pincpu >=3D 0) { - CPU_ZERO(&mask); - CPU_SET(pincpu + vcpu, &mask); + if (vcpumap[vcpu] !=3D NULL) { error =3D pthread_setaffinity_np(pthread_self(), - sizeof(mask), &mask); + sizeof(cpuset_t), + vcpumap[vcpu]); assert(error =3D=3D 0); } =20 @@ -622,7 +668,10 @@ bvmcons =3D 1; break; case 'p': - pincpu =3D atoi(optarg); + if (pincpu_parse(optarg) < 0) { + errx(EX_USAGE, "invalid cpu pin " + "configuration '%s'", optarg); + } break; case 'c': guest_ncpus =3D atoi(optarg); --YiEDa0DAkWCtVeE4-- --tjCHc7DPkfUGtrlw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTYlptAAoJEMltX/4IwiJqEkoH/ixfHJaNREQrMECNKv+B7R1i YJSS3TesSLCiCzKUA887+i+DMgXfbvKu/xMciS6Vllj6ejIONMlrEQgRF9tUO23J p1GRcQRzNXsmhSLlBhZL/HA2LoZN8UStj2vScKYMtAYotFyoXwc024mVy8t6q4wU vllCl21Pa3TAjVYPpzlrKQAadhZXRTIr2U0BDmUWFeMl553U/ZOlT/d/TBDhSeYG 4VVS4eTK7oDkK8/XRPq4bQvswx5KmFGp1Sy0uoVSbUCVYQ6oNg2TEWJtpa+spPl8 SCgiK15s2lHaB/UeAQkSre/sJ/I0Gy5lZ5R/Ncn12tfu0OrCO4YSvZuGPKwKFAA= =Ix2V -----END PGP SIGNATURE----- --tjCHc7DPkfUGtrlw-- From owner-freebsd-virtualization@FreeBSD.ORG Thu May 1 15:10:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D2657EF for ; Thu, 1 May 2014 15:10:22 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1219B160D for ; Thu, 1 May 2014 15:10:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id 8E9AF3AE36 for ; Thu, 1 May 2014 11:00:55 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 95549-08 for ; Thu, 1 May 2014 11:00:55 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 6431C3ADEC; Thu, 1 May 2014 11:00:55 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 01 May 2014 11:00:55 -0400 From: Andre Goree Message-ID: X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 15:10:22 -0000 Hello, I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS host. Prior to a recent upgrade, I've been able to run FreeBSD guests (both 10.0 and 9.2) perfectly fine without issue. Now, this the aforementioned versions, I'm running into a kernel panic almost immediately after booting, same message with both 10.0 & 9.2: "Fatal trap 12: page fault while in kernel mode". To ensure that it had nothing to do with my configuration [of the guest OS] I've tried installing the guest from guest using virt-install, however I still run into the same kernel panic. It'll be hard for me to post the output of the panic, so I'll try to show as much info as possible via images[1]. Upon researching, I've come across a few threads (e.g., this one[2]) that appear to be related, but none have helped. Any information that you can provide to help me get to the root of this issue would be greatly appreciated! [1]http://www.drenet.net/images/panic1.png http://www.drenet.net/images/panic2.png http://www.drenet.net/images/panic_noacpi1.png http://www.drenet.net/images/panic_noacpi2.png [2]http://forums.freebsd.org/viewtopic.php?t=36761 -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Thu May 1 15:34:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 762C25D7 for ; Thu, 1 May 2014 15:34:42 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5851218D0 for ; Thu, 1 May 2014 15:34:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id 88EB13AE36 for ; Thu, 1 May 2014 11:34:41 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 06039-02 for ; Thu, 1 May 2014 11:34:41 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 607D03ADEC; Thu, 1 May 2014 11:34:41 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 01 May 2014 11:34:41 -0400 From: Andre Goree In-Reply-To: References: Message-ID: X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 15:34:42 -0000 On 05/01/2014 11:00 am, Andre Goree wrote: > Hello, > > I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS host. > Prior to a recent upgrade, I've been able to run FreeBSD guests (both > 10.0 and 9.2) perfectly fine without issue. Now, this the > aforementioned versions, I'm running into a kernel panic almost > immediately after booting, same message with both 10.0 & 9.2: "Fatal > trap 12: page fault while in kernel mode". > > To ensure that it had nothing to do with my configuration [of the > guest OS] I've tried installing the guest from guest using > virt-install, however I still run into the same kernel panic. It'll > be hard for me to post the output of the panic, so I'll try to show as > much info as possible via images[1]. > > Upon researching, I've come across a few threads (e.g., this one[2]) > that appear to be related, but none have helped. Any information that > you can provide to help me get to the root of this issue would be > greatly appreciated! > > [1]http://www.drenet.net/images/panic1.png > http://www.drenet.net/images/panic2.png > http://www.drenet.net/images/panic_noacpi1.png > http://www.drenet.net/images/panic_noacpi2.png > > [2]http://forums.freebsd.org/viewtopic.php?t=36761 After using my noggin a bit, I've determine the culprit to be atkbd. If I disable it via the bootloader with "set hint.atkbd.0.disabled=1", the vm will boot fine. HOWEVER, my problem then is that I cannot use the keyboard via VNC. I'm trying to figure out ways around this...any help would be much appreciated. I've already tried setting different flags for atkbd, but no matter which flag I set, I get the kernel panic -- ostensibly because atkbd is enabled (as it must be for a flag to be set for it). -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Thu May 1 17:29:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C2DFFEC for ; Thu, 1 May 2014 17:29:36 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18ED9162A for ; Thu, 1 May 2014 17:29:36 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id w7so2085145qcr.20 for ; Thu, 01 May 2014 10:29:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=5e0/JFj6ZBzOtz3uPXCl94wI0RaSwd2/CmIDs/nQ7Y8=; b=A1n39tpl0GioFFUSVGZMzF2DGMez8wmFN8f2OXsqOwJwsCPm9raU1z4lRdf1fg5AoM 5YhRB8kpVpHOLkF2Ymjtru6vanKMO4VNBvwqbXkfepuUXkTVANZwY7ft6Q1w1BAfLA0H tbJAapBvdEDUGH9lCXczx+xZbvustZzNky3UTAmA9JFcwx79/cRAcvuDZUJH0qnUxW5E QLUbGw5Gx+J+6Lws2WLRZNRQvLte+Qh2C7LPfcqW0oESIK5Ykxc+I4f7CTDzxeli8ymV l+neVA+mvTbIIEdIOyL2HVvnqEd8eSOiWDjMDFOGZKYwIWhSkj/H/uBDWaqqaKe8bHpz sxdA== MIME-Version: 1.0 X-Received: by 10.224.72.12 with SMTP id k12mr15693293qaj.81.1398965374233; Thu, 01 May 2014 10:29:34 -0700 (PDT) Received: by 10.140.47.201 with HTTP; Thu, 1 May 2014 10:29:34 -0700 (PDT) In-Reply-To: References: Date: Thu, 1 May 2014 10:29:34 -0700 Message-ID: Subject: Re: bhyve and serial port emulation problem From: Neel Natu To: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 17:29:36 -0000 Hi Olivier, On Wed, Apr 30, 2014 at 3:28 PM, Olivier Cochard-Labb=C3=A9 wrote: > On Wed, Apr 30, 2014 at 7:33 PM, Neel Natu wrote: >> >> Hi Olivier, >> >> >> Can you share the command line options you pass the bhyve(8)? >> > > Hi Neel, > > sure, here it is: > > bhyveload -m 256M -d /tmp/BSDRP/BSDRP_1 -c /dev/nmdm1A BSDRP_1 > bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc -s > 2:0,virtio-net,tap0,mac=3D58:9c:fc:01:00:01 -s > 1:0,virtio-blk,/tmp/BSDRP/BSDRP_1 -l com1,/dev/nmdm1A BSDRP_1 > The UPDATING entry in r264175 describes how to fix this: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D264175 Briefly, you will now need to change your /etc/ttys to look like this: ttyu0 "/usr/libexec/getty 3wire" vt100 on secure If your virtual machine has an older world then your /etc/gettytab will not have the entry for "3wire" and you may want to use "3wire.9600" instead. best Neel > Regards, > > Olivier From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 06:27:51 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0009C6A6 for ; Fri, 2 May 2014 06:27:50 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AEE271FB7 for ; Fri, 2 May 2014 06:27:50 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id lg15so2174265vcb.7 for ; Thu, 01 May 2014 23:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=usXETkHrRhtJbE8wciax+KPbvI0bEIpSZqBwaUBWCl8=; b=xeV9+UL57r83d+Pb1WnC5HkrlRAylk2EPpgPazcQVfPpZkUDuYQrtw2Ej7kILXK83t Cs6u8qorDCdHbvFZYmoXn3DBTACL6qt6wpVs+QVnUf12rwPrigv3nHB5OYqzuVCkXLHX Z55FB/NMqTogG+mFbj+zBDgOANIt5vyAsYwcXGZoU/QTLrNpJkGklnJsd+RYkZ1uo0z3 2BaQKkpKsSde1oCWrvcM4mnpw/REzV8vL3lt2UG9dkbh1SHGFVeY4JK/8w3tIdhhzqGD Vbf2gJC8wIJlpJAjkpY3Xy3dSem/97p6RM84nW4UCPk7MO2d/HUGTtsmjUS9kS5SQCYk 2Byg== X-Received: by 10.58.13.104 with SMTP id g8mr11989378vec.16.1399012069115; Thu, 01 May 2014 23:27:49 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.2.225 with HTTP; Thu, 1 May 2014 23:27:28 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Fri, 2 May 2014 08:27:28 +0200 X-Google-Sender-Auth: _0UH2D8wNe5lnpcwM3l109nxs2E Message-ID: Subject: Re: bhyve and serial port emulation problem To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 06:27:51 -0000 On Thu, May 1, 2014 at 7:29 PM, Neel Natu wrote: > Hi Olivier, > > The UPDATING entry in r264175 describes how to fix this: > http://svnweb.freebsd.org/base?view=revision&revision=264175 > > Briefly, you will now need to change your /etc/ttys to look like this: > ttyu0 "/usr/libexec/getty 3wire" vt100 on secure > > If your virtual machine has an older world then your /etc/gettytab > will not have the entry for "3wire" and you may want to use > "3wire.9600" instead. > Hi Neel, I've just discovered that on an headless server I had the same problem too, then concluded that I need to check some serial port behavior change and not bhyve bug. I believe the default templates for /etc/ttys (like src/etc/etc.amd64/ttys) need to be updated for replacing "std.9600" by "3wire" for avoiding this regression on new installation. Sorry for the noise regarding bhyve, Olivier From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 18:02:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D401FD90 for ; Fri, 2 May 2014 18:02:04 +0000 (UTC) Received: from smtp.kn-bremen.de (gruenbaer.kn-bremen.de [148.251.8.79]) by mx1.freebsd.org (Postfix) with ESMTP id 92849189C for ; Fri, 2 May 2014 18:02:03 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 754113DE0DDC; Fri, 2 May 2014 20:01:55 +0200 (CEST) Received: from enceladus10.kn-bremen.de (noident@localhost [127.0.0.1]) by enceladus10.kn-bremen.de (8.14.5/8.14.5) with ESMTP id s42I0TMi094543; Fri, 2 May 2014 20:00:29 +0200 (CEST) (envelope-from nox@enceladus10.kn-bremen.de) Received: (from nox@localhost) by enceladus10.kn-bremen.de (8.14.5/8.14.5/Submit) id s42I0TiE094542; Fri, 2 May 2014 20:00:29 +0200 (CEST) (envelope-from nox) Date: Fri, 2 May 2014 20:00:29 +0200 (CEST) From: Juergen Lock Message-Id: <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> To: andre-62Z4ZxnHd7CsTnJN9+BGXg@public.gmane.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-Newsgroups: gmane.os.freebsd.devel.virtualization In-Reply-To: References: Organization: home Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 18:02:04 -0000 In article you write: >On 05/01/2014 11:00 am, Andre Goree wrote: >> Hello, >> >> I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS host. >> Prior to a recent upgrade, I've been able to run FreeBSD guests (both >> 10.0 and 9.2) perfectly fine without issue. Now, this the >> aforementioned versions, I'm running into a kernel panic almost >> immediately after booting, same message with both 10.0 & 9.2: "Fatal >> trap 12: page fault while in kernel mode". >> >> To ensure that it had nothing to do with my configuration [of the >> guest OS] I've tried installing the guest from guest using >> virt-install, however I still run into the same kernel panic. It'll >> be hard for me to post the output of the panic, so I'll try to show as >> much info as possible via images[1]. >> >> Upon researching, I've come across a few threads (e.g., this one[2]) >> that appear to be related, but none have helped. Any information that >> you can provide to help me get to the root of this issue would be >> greatly appreciated! >> >> [1]http://www.drenet.net/images/panic1.png >> http://www.drenet.net/images/panic2.png >> http://www.drenet.net/images/panic_noacpi1.png >> http://www.drenet.net/images/panic_noacpi2.png >> >> [2]http://forums.freebsd.org/viewtopic.php?t=36761 > > >After using my noggin a bit, I've determine the culprit to be atkbd. If >I disable it via the bootloader with "set hint.atkbd.0.disabled=1", the >vm will boot fine. HOWEVER, my problem then is that I cannot use the >keyboard via VNC. I'm trying to figure out ways around this...any help >would be much appreciated. I've already tried setting different flags >for atkbd, but no matter which flag I set, I get the kernel panic -- >ostensibly because atkbd is enabled (as it must be for a flag to be set >for it). > I think you have hit this issue for which the workaround is a patch to seabios: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug http://git.qemu.org/?p=seabios.git;a=commit;h=8032b8a0fec550de5cb2f7d37aa031cebc2200c3 References: http://article.gmane.org/gmane.comp.emulators.qemu/254414/ https://lists.gnu.org/archive/html/qemu-stable/2013-03/msg00037.html HTH, :) Juergen From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 18:19:31 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FD7455D for ; Fri, 2 May 2014 18:19:31 +0000 (UTC) Received: from smtp.kn-bremen.de (gruenbaer.kn-bremen.de [148.251.8.79]) by mx1.freebsd.org (Postfix) with ESMTP id 2DB721A11 for ; Fri, 2 May 2014 18:19:30 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id A8DC43DE0DE3; Fri, 2 May 2014 20:19:29 +0200 (CEST) Received: from enceladus10.kn-bremen.de (noident@localhost [127.0.0.1]) by enceladus10.kn-bremen.de (8.14.5/8.14.5) with ESMTP id s42IIJFD095512; Fri, 2 May 2014 20:18:19 +0200 (CEST) (envelope-from nox@enceladus10.kn-bremen.de) Received: (from nox@localhost) by enceladus10.kn-bremen.de (8.14.5/8.14.5/Submit) id s42IIJnu095511; Fri, 2 May 2014 20:18:19 +0200 (CEST) (envelope-from nox) Date: Fri, 2 May 2014 20:18:19 +0200 (CEST) From: Juergen Lock Message-Id: <201405021818.s42IIJnu095511@enceladus10.kn-bremen.de> To: freebsd-virtualization@freebsd.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-Newsgroups: gmane.os.freebsd.devel.virtualization In-Reply-To: <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> References: Organization: home Cc: Andre Goree X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 18:19:31 -0000 In article <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> you write: >In article you write: >>On 05/01/2014 11:00 am, Andre Goree wrote: >>> Hello, >>> >>> I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS host. >>> Prior to a recent upgrade, I've been able to run FreeBSD guests (both >>> 10.0 and 9.2) perfectly fine without issue. Now, this the >>> aforementioned versions, I'm running into a kernel panic almost >>> immediately after booting, same message with both 10.0 & 9.2: "Fatal >>> trap 12: page fault while in kernel mode". >>> >>> To ensure that it had nothing to do with my configuration [of the >>> guest OS] I've tried installing the guest from guest using >>> virt-install, however I still run into the same kernel panic. It'll >>> be hard for me to post the output of the panic, so I'll try to show as >>> much info as possible via images[1]. >>> >>> Upon researching, I've come across a few threads (e.g., this one[2]) >>> that appear to be related, but none have helped. Any information that >>> you can provide to help me get to the root of this issue would be >>> greatly appreciated! >>> >>> [1]http://www.drenet.net/images/panic1.png >>> http://www.drenet.net/images/panic2.png >>> http://www.drenet.net/images/panic_noacpi1.png >>> http://www.drenet.net/images/panic_noacpi2.png >>> >>> [2]http://forums.freebsd.org/viewtopic.php?t=36761 >> >> >>After using my noggin a bit, I've determine the culprit to be atkbd. If >>I disable it via the bootloader with "set hint.atkbd.0.disabled=1", the >>vm will boot fine. HOWEVER, my problem then is that I cannot use the >>keyboard via VNC. I'm trying to figure out ways around this...any help >>would be much appreciated. I've already tried setting different flags >>for atkbd, but no matter which flag I set, I get the kernel panic -- >>ostensibly because atkbd is enabled (as it must be for a flag to be set >>for it). >> >I think you have hit this issue for which the workaround is a patch >to seabios: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug > Hm for some reason that link got truncated, try this one: https://bugs.debian.org/737142 > http://git.qemu.org/?p=seabios.git;a=commit;h=8032b8a0fec550de5cb2f7d37aa031cebc2200c3 > > References: > > http://article.gmane.org/gmane.comp.emulators.qemu/254414/ > > https://lists.gnu.org/archive/html/qemu-stable/2013-03/msg00037.html > Juergen From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 19:35:54 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C65FF2EF for ; Fri, 2 May 2014 19:35:54 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A0BE512A9 for ; Fri, 2 May 2014 19:35:54 +0000 (UTC) Received: from torb.pix.net (torb.pix.net [IPv6:2001:470:e254:10:12dd:b1ff:febf:eca9]) (authenticated bits=0) by hydra.pix.net (8.14.5/8.14.5) with ESMTP id s42JZrDd015830; Fri, 2 May 2014 15:35:53 -0400 (EDT) (envelope-from lidl@pix.net) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98 at mail.pix.net Message-ID: <5363F399.40709@pix.net> Date: Fri, 02 May 2014 15:35:53 -0400 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Problems with bhyve's kgdb support and loadable modules Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 19:35:54 -0000 Is there any reasonable tutorial for using kgdb with the bvmdebug kernel option? A couple of folks I know have run into issues trying to debug a FreeBSD stable/9 kernel from their bhyve hosting machine (running stable/10). In particular, the loadable modules that are in use in the stable/9 kernel are being "troublesome" to get to the point where source-level debugging actually works. Even a pointer to a couple of "worked" examples might be useful. I've read this: http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html but not all the techniques in there appear to work properly. -Kurt From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 19:42:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0274A452 for ; Fri, 2 May 2014 19:42:28 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC5DB1373 for ; Fri, 2 May 2014 19:42:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id D27BE3ADA0 for ; Fri, 2 May 2014 15:42:19 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 12519-10 for ; Fri, 2 May 2014 15:42:19 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 927CC3ACD4; Fri, 2 May 2014 15:42:19 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 02 May 2014 15:42:19 -0400 From: Andre Goree In-Reply-To: <201405021818.s42IIJnu095511@enceladus10.kn-bremen.de> References: <201405021818.s42IIJnu095511@enceladus10.kn-bremen.de> Message-ID: X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 19:42:28 -0000 On 05/02/2014 2:18 pm, Juergen Lock wrote: > In article <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> you > write: >> In article >> >> you write: >>> On 05/01/2014 11:00 am, Andre Goree wrote: >>>> Hello, >>>> >>>> I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS >>>> host. >>>> Prior to a recent upgrade, I've been able to run FreeBSD guests >>>> (both >>>> 10.0 and 9.2) perfectly fine without issue. Now, this the >>>> aforementioned versions, I'm running into a kernel panic almost >>>> immediately after booting, same message with both 10.0 & 9.2: >>>> "Fatal >>>> trap 12: page fault while in kernel mode". >>>> >>>> To ensure that it had nothing to do with my configuration [of the >>>> guest OS] I've tried installing the guest from guest using >>>> virt-install, however I still run into the same kernel panic. It'll >>>> be hard for me to post the output of the panic, so I'll try to show >>>> as >>>> much info as possible via images[1]. >>>> >>>> Upon researching, I've come across a few threads (e.g., this one[2]) >>>> that appear to be related, but none have helped. Any information >>>> that >>>> you can provide to help me get to the root of this issue would be >>>> greatly appreciated! >>>> >>>> [1]http://www.drenet.net/images/panic1.png >>>> http://www.drenet.net/images/panic2.png >>>> http://www.drenet.net/images/panic_noacpi1.png >>>> http://www.drenet.net/images/panic_noacpi2.png >>>> >>>> [2]http://forums.freebsd.org/viewtopic.php?t=36761 >>> >>> >>> After using my noggin a bit, I've determine the culprit to be atkbd. >>> If >>> I disable it via the bootloader with "set hint.atkbd.0.disabled=1", >>> the >>> vm will boot fine. HOWEVER, my problem then is that I cannot use the >>> keyboard via VNC. I'm trying to figure out ways around this...any >>> help >>> would be much appreciated. I've already tried setting different >>> flags >>> for atkbd, but no matter which flag I set, I get the kernel panic -- >>> ostensibly because atkbd is enabled (as it must be for a flag to be >>> set >>> for it). >>> >> I think you have hit this issue for which the workaround is a patch >> to seabios: >> >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug >> > Hm for some reason that link got truncated, try this one: > > https://bugs.debian.org/737142 > >> http://git.qemu.org/?p=seabios.git;a=commit;h=8032b8a0fec550de5cb2f7d37aa031cebc2200c3 >> >> References: >> >> http://article.gmane.org/gmane.comp.emulators.qemu/254414/ >> >> https://lists.gnu.org/archive/html/qemu-stable/2013-03/msg00037.html >> > > Juergen > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" Lol, yeah, I was gonna say...but thanks! Not sure if I'm into implementing a patch vs. finding a way around this, and I'm wondering why it reared it's head after an update to libvirt & qemu vs. earlier in my testing -- makes me think it may be an unrelated, albeit similar bug. Thanks again Juergen. -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Fri May 2 19:51:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B4D1871 for ; Fri, 2 May 2014 19:51:14 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DB0E1446 for ; Fri, 2 May 2014 19:51:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id 119043ADA0 for ; Fri, 2 May 2014 15:51:13 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 13265-01 for ; Fri, 2 May 2014 15:51:11 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id F15CB3ACD4; Fri, 2 May 2014 15:51:10 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 02 May 2014 15:51:10 -0400 From: Andre Goree In-Reply-To: References: <201405021818.s42IIJnu095511@enceladus10.kn-bremen.de> Message-ID: <566708d82c9575d14789b91544ebe0fa@drenet.net> X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 19:51:14 -0000 On 05/02/2014 3:42 pm, Andre Goree wrote: > On 05/02/2014 2:18 pm, Juergen Lock wrote: >> In article <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> you >> write: >>> In article >>> >>> you write: >>>> On 05/01/2014 11:00 am, Andre Goree wrote: >>>>> Hello, >>>>> >>>>> I'm running libvirt 1.2.2 and QEMU 1.7.91 on an Ubuntu 12.04LTS >>>>> host. >>>>> Prior to a recent upgrade, I've been able to run FreeBSD guests >>>>> (both >>>>> 10.0 and 9.2) perfectly fine without issue. Now, this the >>>>> aforementioned versions, I'm running into a kernel panic almost >>>>> immediately after booting, same message with both 10.0 & 9.2: >>>>> "Fatal >>>>> trap 12: page fault while in kernel mode". >>>>> >>>>> To ensure that it had nothing to do with my configuration [of the >>>>> guest OS] I've tried installing the guest from guest using >>>>> virt-install, however I still run into the same kernel panic. >>>>> It'll >>>>> be hard for me to post the output of the panic, so I'll try to show >>>>> as >>>>> much info as possible via images[1]. >>>>> >>>>> Upon researching, I've come across a few threads (e.g., this >>>>> one[2]) >>>>> that appear to be related, but none have helped. Any information >>>>> that >>>>> you can provide to help me get to the root of this issue would be >>>>> greatly appreciated! >>>>> >>>>> [1]http://www.drenet.net/images/panic1.png >>>>> http://www.drenet.net/images/panic2.png >>>>> http://www.drenet.net/images/panic_noacpi1.png >>>>> http://www.drenet.net/images/panic_noacpi2.png >>>>> >>>>> [2]http://forums.freebsd.org/viewtopic.php?t=36761 >>>> >>>> >>>> After using my noggin a bit, I've determine the culprit to be atkbd. >>>> If >>>> I disable it via the bootloader with "set hint.atkbd.0.disabled=1", >>>> the >>>> vm will boot fine. HOWEVER, my problem then is that I cannot use >>>> the >>>> keyboard via VNC. I'm trying to figure out ways around this...any >>>> help >>>> would be much appreciated. I've already tried setting different >>>> flags >>>> for atkbd, but no matter which flag I set, I get the kernel panic -- >>>> ostensibly because atkbd is enabled (as it must be for a flag to be >>>> set >>>> for it). >>>> >>> I think you have hit this issue for which the workaround is a patch >>> to seabios: >>> >>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug >>> >> Hm for some reason that link got truncated, try this one: >> >> https://bugs.debian.org/737142 >> >>> http://git.qemu.org/?p=seabios.git;a=commit;h=8032b8a0fec550de5cb2f7d37aa031cebc2200c3 >>> >>> References: >>> >>> http://article.gmane.org/gmane.comp.emulators.qemu/254414/ >>> >>> https://lists.gnu.org/archive/html/qemu-stable/2013-03/msg00037.html >>> >> >> Juergen >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > Lol, yeah, I was gonna say...but thanks! Not sure if I'm into > implementing a patch vs. finding a way around this, and I'm wondering > why it reared it's head after an update to libvirt & qemu vs. earlier > in my testing -- makes me think it may be an unrelated, albeit similar > bug. Thanks again Juergen. Btw, from the link you posted: "From: Michael Tokarev To: 737142-close@bugs.debian.org Subject: Bug#737142: fixed in seabios 1.7.4-4 Source: seabios Source-Version: 1.7.4-4 We believe that the bug you reported is fixed in the latest version of seabios, which is due to be installed in the Debian FTP archive." and on my host server having the issue: "# dpkg -l |grep seabio ii seabios 1.7.4-4~cloud0 Legacy BIOS implementation" I'll try to determine which version of seabios I had running initially before the issue, however from what I can tell, an upgrade to the version that _should_ have fixed this appears be what caused my issue in the first place, heh. -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Sun May 4 03:01:12 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0DD4B51 for ; Sun, 4 May 2014 03:01:12 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 55DEF1306 for ; Sun, 4 May 2014 03:01:11 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 0875212429; Sun, 4 May 2014 13:01:04 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTX64641 (AUTH peterg@ptree32.com.au); Sun, 4 May 2014 13:01:02 +1000 Message-ID: <5365AD6C.5060107@freebsd.org> Date: Sat, 03 May 2014 20:01:00 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Kurt Lidl Subject: Re: Problems with bhyve's kgdb support and loadable modules References: <5363F399.40709@pix.net> In-Reply-To: <5363F399.40709@pix.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 03:01:12 -0000 Hi Kurt, > Is there any reasonable tutorial for using kgdb with the > bvmdebug kernel option? Not really (https://wiki.freebsd.org/BHyVe/gdb) In any event, 9.2 doesn't have bvmdebug, though it would be a simple backport. It's also not strictly required - you can use the serial port same as on h/w; see below. > A couple of folks I know have run into issues trying to > debug a FreeBSD stable/9 kernel from their bhyve > hosting machine (running stable/10). > > In particular, the loadable modules that are in use in > the stable/9 kernel are being "troublesome" to get to > the point where source-level debugging actually works. > > Even a pointer to a couple of "worked" examples might be > useful. > > I've read this: > http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html > but not all the techniques in there appear to work properly. I tried to repro this with some success from a host running CURRENT. Firstly, I installed a 9.2 VM, with source. I edited GENERIC and added options DDB and GDB, and reinstalled the kernel. The disk was then copied, and mdconfig'd/mounted on the host to provide access to the just-buit 9.2 kernel syms and sources. com2 was set up as a debug port by dropping to the bhyveload prompt and hint.uart.1.flags="0x80" (this could also have been done in the guest's /boot/loader.conf) com2 was then set up in the bhyve command line to point to an nmdm device -l com2,/dev/nmdm0A The guest probed uart1 as a debug port: uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x80 on acpi0 ioapic0: routing intpin 3 (ISA IRQ 3) to lapic 0 vector 64 uart1: fast interrupt uart1: debug port (9600,n,8,1) After booting, I loaded the the tap device in the guest to provide a kld for kgdb to examine: root@fbsd9-2:~ # kldload if_tap root@fbsd9-2:~ # kldstat Id Refs Address Size Name 1 3 0xffffffff80200000 15f92d8 kernel 2 1 0xffffffff81a12000 59e9 if_tap.ko Now time to try kgdb: root@fbsd9-2:~ # sysctl debug.kdb.enter=1 debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter [ thread pid 577 tid 100054 ] Stopped at kdb_enter+0x3b: movq $0,0xaf0362(%rip) db> gdb (ctrl-c will return control to ddb) Switching to gdb back-end Switching to gdb back-end In another window, cd'd to the mounted copy of the 9.2 disk: kgdb -r /dev/nmdm0B kernel.debug ... This GDB was configured as "amd64-marcel-freebsd"...Switching to remote protocol kdb_enter (why=0xffffffff80fbf671 "sysctl", msg=0x80
) at ../../../kern/subr_kdb.c:441 441 kdb_why = KDB_WHY_UNSET; (kgdb) After some experimentation, I found the way to get the correct symbols for the kld was to manually specify it: (kgdb) add-kld /mnt/boot/kernel/if_tap.ko add symbol table from file "/mnt/boot/kernel/if_tap.ko.symbols" at .text_addr = 0xffffffff81a12000 .rodata.str1.8_addr = 0xffffffff81a13b50 .rodata.str1.1_addr = 0xffffffff81a13ddb set_sysinit_set_addr = 0xffffffff81a13f68 set_modmetadata_set_addr = 0xffffffff81a13f80 set_sysctl_set_addr = 0xffffffff81a13f90 set_sysuninit_set_addr = 0xffffffff81a13fc0 .data_addr = 0xffffffff81a13fe0 .bss_addr = 0xffffffff81a14620 (y or n) y However, 'info sharedlibrary' didn't seem to reflect this: (kgdb) info sharedlibrary From To Syms Read Shared Object Library 0xffffffff81a12000 0xffffffff81a13c04 No /boot/kernel/if_tap.ko.symbols Might have been a bug there, since I was able to successfully set breakpoints in if_tap routines and have them trigger. I did have some trouble getting the source path set up correctly, but never fully investigated that: seems like gdb is quite rich in that area and it should be possible to get sorted. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun May 4 12:57:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBF914BE for ; Sun, 4 May 2014 12:57:58 +0000 (UTC) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A85D123D for ; Sun, 4 May 2014 12:57:58 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 5F6482842F for ; Sun, 4 May 2014 14:57:49 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 786E328426 for ; Sun, 4 May 2014 14:57:48 +0200 (CEST) Message-ID: <5366394B.6040500@quip.cz> Date: Sun, 04 May 2014 14:57:47 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Best practices with network settings for virtualization Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 12:57:58 -0000 I would like to ask some really experienced person - what is the best way to run virtual guests connected to network with public IPs? I think many people run unsecure setup with guests with simple bridged network. I know there are many options with tun, bridge, epair, VDE, Open vSwitch etc., my main concern is the setup of network where each guest can use only predefined MAC and predifined IP(s). If some malicious user or malware in guest OS tried to change MAC od IP, I would like to disallow that or do not allow any offending traffic to reach outside network or any other guest running on the same machine. Guests can be VirtualBox, Bhyve or anything else. I really appreciate any help or ideas. -- Miroslav Lachman PS: I don't know if this is the best lsit to ask, maybe freebsd-net@ is better place? From owner-freebsd-virtualization@FreeBSD.ORG Sun May 4 16:01:47 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5BDDDED for ; Sun, 4 May 2014 16:01:47 +0000 (UTC) Received: from smtp149.ord.emailsrvr.com (smtp149.ord.emailsrvr.com [173.203.6.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 69EC01271 for ; Sun, 4 May 2014 16:01:47 +0000 (UTC) Received: from smtp19.relay.ord1a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp19.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 83C693B0392 for ; Sun, 4 May 2014 11:53:19 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp19.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 800C43B03BC for ; Sun, 4 May 2014 11:53:19 -0400 (EDT) X-Virus-Scanned: OK Received: from smtp192.mex05.mlsrvr.com (unknown [184.106.31.85]) by smtp19.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTPS id 6D0F73B0392 for ; Sun, 4 May 2014 11:53:19 -0400 (EDT) Received: from ORD2MBX05C.mex05.mlsrvr.com ([fe80::90e2:baff:fe30:7498]) by ORD2HUB21.mex05.mlsrvr.com ([fe80::be30:5bff:feee:18f4%15]) with mapi id 14.03.0169.001; Sun, 4 May 2014 10:53:19 -0500 From: Michael Berman To: "freebsd-virtualization@freebsd.org" Subject: Re: Problems with bhyve's kgdb support and loadable modules Thread-Topic: Problems with bhyve's kgdb support and loadable modules Thread-Index: AQHPZj3AozzOshyFXESNtt0V99BgVZswEX0AgABibAA= Date: Sun, 4 May 2014 15:53:18 +0000 Message-ID: References: <5363F399.40709@pix.net> <5365AD6C.5060107@freebsd.org> In-Reply-To: <5365AD6C.5060107@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [71.198.217.229] Content-Type: text/plain; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 16:01:47 -0000 On 5/3/14, 8:01 PM, "Peter Grehan" wrote: >Hi Kurt, > >> Is there any reasonable tutorial for using kgdb with the >> bvmdebug kernel option? > > Not really (https://wiki.freebsd.org/BHyVe/gdb) > > In any event, 9.2 doesn't have bvmdebug, though it would be a simple >backport. It's also not strictly required - you can use the serial port >same as on h/w; see below. > >> A couple of folks I know have run into issues trying to >> debug a FreeBSD stable/9 kernel from their bhyve >> hosting machine (running stable/10). >> >> In particular, the loadable modules that are in use in >> the stable/9 kernel are being "troublesome" to get to >> the point where source-level debugging actually works. >> >> Even a pointer to a couple of "worked" examples might be >> useful. >> >> I've read this: >> http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html >> but not all the techniques in there appear to work properly. > > I tried to repro this with some success from a host running CURRENT. > > Firstly, I installed a 9.2 VM, with source. I edited GENERIC and added >options DDB and GDB, and reinstalled the kernel. > > The disk was then copied, and mdconfig'd/mounted on the host to >provide access to the just-buit 9.2 kernel syms and sources. > > com2 was set up as a debug port by dropping to the bhyveload prompt and > > hint.uart.1.flags=3D"0x80" > > (this could also have been done in the guest's /boot/loader.conf) > > com2 was then set up in the bhyve command line to point to an nmdm >device > > -l com2,/dev/nmdm0A > > The guest probed uart1 as a debug port: > >uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x80 on acpi0 >ioapic0: routing intpin 3 (ISA IRQ 3) to lapic 0 vector 64 >uart1: fast interrupt >uart1: debug port (9600,n,8,1) > > After booting, I loaded the the tap device in the guest to provide a >kld for kgdb to examine: > >root@fbsd9-2:~ # kldload if_tap >root@fbsd9-2:~ # kldstat >Id Refs Address Size Name > 1 3 0xffffffff80200000 15f92d8 kernel > 2 1 0xffffffff81a12000 59e9 if_tap.ko > > Now time to try kgdb: > >root@fbsd9-2:~ # sysctl debug.kdb.enter=3D1 >debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter >[ thread pid 577 tid 100054 ] >Stopped at kdb_enter+0x3b: movq $0,0xaf0362(%rip) >db> gdb >(ctrl-c will return control to ddb) >Switching to gdb back-end >Switching to gdb back-end > > In another window, cd'd to the mounted copy of the 9.2 disk: > >kgdb -r /dev/nmdm0B kernel.debug >... >This GDB was configured as "amd64-marcel-freebsd"...Switching to remote >protocol >kdb_enter (why=3D0xffffffff80fbf671 "sysctl", msg=3D0x80
of bounds>) at ../../../kern/subr_kdb.c:441 >441 kdb_why =3D KDB_WHY_UNSET; > >(kgdb) > > After some experimentation, I found the way to get the correct symbols >for the kld was to manually specify it: > >(kgdb) add-kld /mnt/boot/kernel/if_tap.ko >add symbol table from file "/mnt/boot/kernel/if_tap.ko.symbols" at > .text_addr =3D 0xffffffff81a12000 > .rodata.str1.8_addr =3D 0xffffffff81a13b50 > .rodata.str1.1_addr =3D 0xffffffff81a13ddb > set_sysinit_set_addr =3D 0xffffffff81a13f68 > set_modmetadata_set_addr =3D 0xffffffff81a13f80 > set_sysctl_set_addr =3D 0xffffffff81a13f90 > set_sysuninit_set_addr =3D 0xffffffff81a13fc0 > .data_addr =3D 0xffffffff81a13fe0 > .bss_addr =3D 0xffffffff81a14620 >(y or n) y > > However, 'info sharedlibrary' didn't seem to reflect this: > >(kgdb) info sharedlibrary > From To Syms Read Shared Object Library >0xffffffff81a12000 0xffffffff81a13c04 No >/boot/kernel/if_tap.ko.symbols > > Might have been a bug there, since I was able to successfully set >breakpoints in if_tap routines and have them trigger. > > I did have some trouble getting the source path set up correctly, but >never fully investigated that: seems like gdb is quite rich in that area >and it should be possible to get sorted. > >later, > >Peter. I=B9ve had good results with gdb a few of different ways, first as Peter described, but also using a .gdbinit file to set the substitute-path. For example: set substitute-path /usr/home/michael/src /usr/src Another nice thing about the bhyve diskdev is that you can mount it, here=B9s a Bourne shell script snippet: md=3D`sudo mdconfig -a -t vnode -f $diskdev` sudo mkdir -p /mnt/ts_guest sudo fsck -y -t ufs /dev/${md}p2 sudo mount -t ufs /dev/${md}p2 /mnt/ts_guest This means that with some careful gimmicks one can mount the guest such that its source tree is at a useful place. Depending on how/when you do this, I recommend a read only mount ;-) Michael From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 11:06:53 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C2F5F53 for ; Mon, 5 May 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18E221D07 for ; Mon, 5 May 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s45B6qOA083297 for ; Mon, 5 May 2014 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s45B6q3G083295 for freebsd-virtualization@FreeBSD.org; Mon, 5 May 2014 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 5 May 2014 11:06:52 GMT Message-Id: <201405051106.s45B6q3G083295@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 18:14:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A698129; Mon, 5 May 2014 18:14:02 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0135CFF; Mon, 5 May 2014 18:14:01 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id 63so2291131qgz.2 for ; Mon, 05 May 2014 11:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=QyuEsIr7mVZdPViRGLygGC9rJGUFzPZksDcO3DXPCoU=; b=FRnv9poQxZn2O6UDRB3cSe+e4deZm2rXVDjPKS9fkD4pLKDjgBQAV0ABC68FeUU1IM Jgu/2SKTG7Ohj4h42c4mjHY89pFhC/3azECZRlgy44Oa0pH/usKdGu1mtxjptp/t8ykG Y4USrvFn3qWLG3uvDYAQ2Dl0d2hLBjijG2Zx8gXypVqTd3fPO6yXyYLS78cq47RnkAkv dg/eqAS8amFvPJLMhjbKCiTZEBBhZnseCSa5u/kZZQji7e3fm5F8scLTlUrEfdUPHpyR wRNmr1BkRYi9l48qejJBvSkjIDAm3HefuEzGpeUa4LFJHYgQzbe8S8ZtAChB6qVaRIka ydhg== MIME-Version: 1.0 X-Received: by 10.224.72.12 with SMTP id k12mr48232900qaj.81.1399313640659; Mon, 05 May 2014 11:14:00 -0700 (PDT) Received: by 10.140.50.235 with HTTP; Mon, 5 May 2014 11:14:00 -0700 (PDT) In-Reply-To: <20140501143005.GA91029@kloomba> References: <20140427104511.GA7804@kloomba> <20140501143005.GA91029@kloomba> Date: Mon, 5 May 2014 11:14:00 -0700 Message-ID: Subject: Re: [PATCH] Flexible vcpu pinning configuration From: Neel Natu To: Roman Bogorodskiy Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 18:14:02 -0000 Hi Roman, On Thu, May 1, 2014 at 7:30 AM, Roman Bogorodskiy wrote= : > Neel Natu wrote: > >> Hi Roman, >> >> On Sun, Apr 27, 2014 at 3:45 AM, Roman Bogorodskiy w= rote: >> > I've created an initial version of the patch which allows more flexibl= e >> > vcpu pinning configuration. >> > >> > Current schema is: >> > >> > bhyve -p N >> > >> > pins vcpu i to hostcpu N + i. >> > >> > The propsed extension is: >> > >> > bhyve -p N:M .... -p 0:1 -p 3:5 >> > >> > which pins vcpu N to host pcpu M. Option needs to be specified >> > individually for each vcpu. >> > >> > So it works like that for me: >> > >> > sudo /usr/sbin/bhyve -p 0:0 -p 1:3 -c 2 ... >> > >> > # sudo cpuset -g -t 100262 >> > tid 100262 mask: 0 >> > # sudo cpuset -g -t 100264 >> > tid 100264 mask: 3 >> > >> > PS I used cpumat_t* array to store these values instead of int, becaus= e >> > if the idea is OK, I'll extend it to support ranges like e.g. cpuset(1= ) >> > supports, e.g.: "1:2-5". >> > >> > The questions are: >> > >> > - Is it OK to chance '-p' arg syntax or it's better to introduce a ne= w >> > one? >> > >> >> I think we can reuse the "-p" option unless anybody objects vociferously= . >> >> > - Is the syntax OK (currently: 'vcpu:pcpu', later >> > 'vcpu:pcpuN-pcpuM,pcpuX")? >> >> Yup, I think that works fine. >> >> The patch looks good in general but I have a few comments: >> >> - Scope of 'vcpupmap[]' should be restricted to 'static'. >> >> - usage() and man page need to be updated. >> >> - pincpu_parse(): >> The option parsing can be made much easier by using: >> >> if (sscanf(str, "%d:%d", &vcpu, &pcpu) =3D=3D 2) { >> /* success */ >> } else { >> return (-1); >> } >> >> If the same vcpu is specified multiple times then we should >> malloc(sizeof(cpuset_t)) only the first time: >> >> if (vcpumap[vcpu] !=3D NULL) >> mask =3D vcpumap[vcpu]; >> else >> mask =3D malloc(sizeof(cpuset_t)); >> >> We need to range-check 'vcpu' before using it as an index into the >> 'vcpumap[]' array. >> >> best >> Neel > > Attached an updated patch. A slightly modified version was submitted: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265376 > > I'm still inclined to use something like parselist() from > usr.bin/cpuset/cpuset.c, but I don't want to copy/paste and I don't know > where it'd make sense to move it so it was usable outside of cpuset? > Hmm, not sure really =E2=80=A6 but you can get started by copying it into bhyve and if there is a better place to put it we can do that too. > PS While reading bhyverun.c, I think I spotted a typo: in fbsdrun_deletec= pu() > error message says fprintf(stderr, "addcpu: .... Should it be "deletecpu:= " instead? Thanks. This is fixed now: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D265366 best Neel > > Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 18:29:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91FB781F; Mon, 5 May 2014 18:29:16 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 685ACE38; Mon, 5 May 2014 18:29:16 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 1C135B949; Mon, 5 May 2014 14:29:14 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: Problems with bhyve's kgdb support and loadable modules Date: Mon, 5 May 2014 13:33:49 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <5363F399.40709@pix.net> <5365AD6C.5060107@freebsd.org> In-Reply-To: <5365AD6C.5060107@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405051333.49758.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 05 May 2014 14:29:14 -0400 (EDT) Cc: Kurt Lidl X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 18:29:16 -0000 On Saturday, May 03, 2014 11:01:00 pm Peter Grehan wrote: > Hi Kurt, > > > Is there any reasonable tutorial for using kgdb with the > > bvmdebug kernel option? > > Not really (https://wiki.freebsd.org/BHyVe/gdb) > > In any event, 9.2 doesn't have bvmdebug, though it would be a simple > backport. It's also not strictly required - you can use the serial port > same as on h/w; see below. > > > A couple of folks I know have run into issues trying to > > debug a FreeBSD stable/9 kernel from their bhyve > > hosting machine (running stable/10). > > > > In particular, the loadable modules that are in use in > > the stable/9 kernel are being "troublesome" to get to > > the point where source-level debugging actually works. > > > > Even a pointer to a couple of "worked" examples might be > > useful. > > > > I've read this: > > http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html > > but not all the techniques in there appear to work properly. > > I tried to repro this with some success from a host running CURRENT. > > Firstly, I installed a 9.2 VM, with source. I edited GENERIC and added > options DDB and GDB, and reinstalled the kernel. > > The disk was then copied, and mdconfig'd/mounted on the host to > provide access to the just-buit 9.2 kernel syms and sources. > > com2 was set up as a debug port by dropping to the bhyveload prompt and > > hint.uart.1.flags="0x80" > > (this could also have been done in the guest's /boot/loader.conf) > > com2 was then set up in the bhyve command line to point to an nmdm device > > -l com2,/dev/nmdm0A > > The guest probed uart1 as a debug port: > > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x80 on acpi0 > ioapic0: routing intpin 3 (ISA IRQ 3) to lapic 0 vector 64 > uart1: fast interrupt > uart1: debug port (9600,n,8,1) > > After booting, I loaded the the tap device in the guest to provide a > kld for kgdb to examine: > > root@fbsd9-2:~ # kldload if_tap > root@fbsd9-2:~ # kldstat > Id Refs Address Size Name > 1 3 0xffffffff80200000 15f92d8 kernel > 2 1 0xffffffff81a12000 59e9 if_tap.ko > > Now time to try kgdb: > > root@fbsd9-2:~ # sysctl debug.kdb.enter=1 > debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter > [ thread pid 577 tid 100054 ] > Stopped at kdb_enter+0x3b: movq $0,0xaf0362(%rip) > db> gdb > (ctrl-c will return control to ddb) > Switching to gdb back-end > Switching to gdb back-end > > In another window, cd'd to the mounted copy of the 9.2 disk: > > kgdb -r /dev/nmdm0B kernel.debug > ... > This GDB was configured as "amd64-marcel-freebsd"...Switching to remote > protocol > kdb_enter (why=0xffffffff80fbf671 "sysctl", msg=0x80
of bounds>) at ../../../kern/subr_kdb.c:441 > 441 kdb_why = KDB_WHY_UNSET; > > (kgdb) > > After some experimentation, I found the way to get the correct symbols > for the kld was to manually specify it: > > (kgdb) add-kld /mnt/boot/kernel/if_tap.ko > add symbol table from file "/mnt/boot/kernel/if_tap.ko.symbols" at > .text_addr = 0xffffffff81a12000 > .rodata.str1.8_addr = 0xffffffff81a13b50 > .rodata.str1.1_addr = 0xffffffff81a13ddb > set_sysinit_set_addr = 0xffffffff81a13f68 > set_modmetadata_set_addr = 0xffffffff81a13f80 > set_sysctl_set_addr = 0xffffffff81a13f90 > set_sysuninit_set_addr = 0xffffffff81a13fc0 > .data_addr = 0xffffffff81a13fe0 > .bss_addr = 0xffffffff81a14620 > (y or n) y > > However, 'info sharedlibrary' didn't seem to reflect this: > > (kgdb) info sharedlibrary > From To Syms Read Shared Object Library > 0xffffffff81a12000 0xffffffff81a13c04 No > /boot/kernel/if_tap.ko.symbols > > Might have been a bug there, since I was able to successfully set > breakpoints in if_tap routines and have them trigger. > > I did have some trouble getting the source path set up correctly, but > never fully investigated that: seems like gdb is quite rich in that area > and it should be possible to get sorted. I think you want to set solib-absolute-prefix to '/mnt'. That will cause kgdb to prepend '/mnt' to the paths it finds for each kld. Mounting /mnt directly on the host is a bit risky though. What I've done with bhyve is to build a test kernel on the host and install it to a directory on the host, then use the '-h' flag to bhyveload to export that directory to the loader and explicitly load the test kernel from the 'host0:' filesystem. I've even been able to build a cross kgdb-i386 to debug a cross-built kernel for an i386 guest and have this work ok. -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 21:45:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF09D7C3; Mon, 5 May 2014 21:45:25 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 72EFE9F0; Mon, 5 May 2014 21:45:25 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 72B60122F7; Tue, 6 May 2014 07:44:52 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BTY95335 (AUTH peterg@ptree32.com.au); Tue, 6 May 2014 07:44:50 +1000 Message-ID: <53680650.3090702@freebsd.org> Date: Mon, 05 May 2014 14:44:48 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: John Baldwin Subject: Re: Problems with bhyve's kgdb support and loadable modules References: <5363F399.40709@pix.net> <5365AD6C.5060107@freebsd.org> <201405051333.49758.jhb@freebsd.org> In-Reply-To: <201405051333.49758.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kurt Lidl , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 21:45:25 -0000 > Mounting /mnt directly on the host is a bit risky though. Absolutely - I should have been clearer; I copied the guest disk image to another file before doing that. Even a read-only mount of the image on one system, as Michael mentioned, can still result in a crash. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon May 5 21:47:55 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 603B0999 for ; Mon, 5 May 2014 21:47:55 +0000 (UTC) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1C7CA0D for ; Mon, 5 May 2014 21:47:54 +0000 (UTC) Received: by mail-wi0-f176.google.com with SMTP id n15so3550970wiw.15 for ; Mon, 05 May 2014 14:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6WJ0w1sOQqtKjtKchZEb9uoKYmKWMAO5LG2iTTj/9dU=; b=UfCM3+vFnLlkHWGB+yv4hD7qQSiihhJ62PsyQypXsyfagxm1cHEBWHM1FCXAKJOewC lo9LLy4QJ59rAjZu45JKVuVUhTHXlUeanINjMAsLt2YDiaXs7T92UmEes3wZdrAZ7ij8 tKCrgZkQ1TK+bg0NBE0h3V/zgnI8lCd7c/vkGdlQo9HqCI5SHgDehKhuamnVSL2t8JPP GcVIeeFRIwGDJY3V9Ld2z/BXEGMtzM9pkrZEUcpdIT3DxPltlCdZCpir+4gh1PwkhoXg PN3TqMcRjI6YAP9RzHrOZXQXpz4ywj+MQ19t9P+UI2IlffROC3Tb/Fl7VBs+OKgHE6YD b0Rg== MIME-Version: 1.0 X-Received: by 10.180.212.107 with SMTP id nj11mr17962612wic.40.1399326472161; Mon, 05 May 2014 14:47:52 -0700 (PDT) Received: by 10.194.237.103 with HTTP; Mon, 5 May 2014 14:47:52 -0700 (PDT) Date: Mon, 5 May 2014 14:47:52 -0700 Message-ID: Subject: virt-manager bhyve From: Alexandar Narayan To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 21:47:55 -0000 Has anyone built virt-manager with bhyve support? I read that it was supported just turned off by default. I installed virt-manager via pkg install virt-manager and can get it to run but I can't get it to see bhyve. Thanks, Alex From owner-freebsd-virtualization@FreeBSD.ORG Tue May 6 00:00:59 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DB1AAB2 for ; Tue, 6 May 2014 00:00:59 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BA6035D for ; Tue, 6 May 2014 00:00:58 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id 10so666963lbg.30 for ; Mon, 05 May 2014 17:00:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pd3079k34NP6+u8OjrF47ApLRpKCxBc04iK1xmzt1a4=; b=fXuVM94/t8I4z2JNAOn/abJdsx0eolro4kn9eQ406upEGOhf2pURVVIlHQPRTFYpYd yFAYsBCj2E2IaYhulFcv082NwVcVMd3avqtHwBpzxkgK146wjOFGzx9Q1tI6muDcr+kE Q2jjvGsAG9cAsD8LoRPVStqDI5Hvpb6wH+pI4lxYKD+jofrM7s7urX96qj2aRu2rHS9X IvSy72OOA8klKB9oM2evIfj/DrXDnXLukKRJj7PdISYeaYJohttBL+JkUhKI9pEdHBzU wLvQg+NbyNkBmzBcp8BUdOAp8ZC3X6Da8F6p3lCViTHuEtBBhWdHiZAhet6mpk6MT0xo hNpg== MIME-Version: 1.0 X-Received: by 10.112.157.162 with SMTP id wn2mr1290821lbb.38.1399334456356; Mon, 05 May 2014 17:00:56 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Mon, 5 May 2014 17:00:56 -0700 (PDT) In-Reply-To: References: Date: Mon, 5 May 2014 17:00:56 -0700 X-Google-Sender-Auth: 2DVfnXSAtW43uowr0ppYXH6AfuA Message-ID: Subject: Re: virt-manager bhyve From: Craig Rodrigues To: Alexandar Narayan Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 00:00:59 -0000 On Mon, May 5, 2014 at 2:47 PM, Alexandar Narayan wrote: > Has anyone built virt-manager with bhyve support? I haven't tried any of this stuff, but you may want to read the blog postings mentioned at this link: http://www.freebsd.org/news/status/report-2014-01-2014-03.html#libvirt/bhyve-Support Those blog postings were written by Roman Bogorodskiy. Roman has done a lot of work to get libvirt working with bhyve, but there seems to be some additional setup steps required. These steps are mentioned in his blog postings. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue May 6 04:45:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADC698C4; Tue, 6 May 2014 04:45:42 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67011F3B; Tue, 6 May 2014 04:45:42 +0000 (UTC) Received: from torb.pix.net (torb.pix.net [IPv6:2001:470:e254:10:12dd:b1ff:febf:eca9]) (authenticated bits=0) by hydra.pix.net (8.14.5/8.14.5) with ESMTP id s464jeqw061729; Tue, 6 May 2014 00:45:40 -0400 (EDT) (envelope-from lidl@pix.net) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98 at mail.pix.net Message-ID: <536868F4.7010006@pix.net> Date: Tue, 06 May 2014 00:45:40 -0400 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Problems with bhyve's kgdb support and loadable modules References: <5363F399.40709@pix.net> <5365AD6C.5060107@freebsd.org> In-Reply-To: <5365AD6C.5060107@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 04:45:42 -0000 On 5/3/14 11:01 PM, Peter Grehan wrote: > Hi Kurt, > >> Is there any reasonable tutorial for using kgdb with the >> bvmdebug kernel option? > > Not really (https://wiki.freebsd.org/BHyVe/gdb) > > In any event, 9.2 doesn't have bvmdebug, though it would be a simple > backport. It's also not strictly required - you can use the serial port > same as on h/w; see below. > >> A couple of folks I know have run into issues trying to >> debug a FreeBSD stable/9 kernel from their bhyve >> hosting machine (running stable/10). >> >> In particular, the loadable modules that are in use in >> the stable/9 kernel are being "troublesome" to get to >> the point where source-level debugging actually works. >> >> Even a pointer to a couple of "worked" examples might be >> useful. >> >> I've read this: >> http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html >> but not all the techniques in there appear to work properly. > > I tried to repro this with some success from a host running CURRENT. > > Firstly, I installed a 9.2 VM, with source. I edited GENERIC and added > options DDB and GDB, and reinstalled the kernel. > > The disk was then copied, and mdconfig'd/mounted on the host to > provide access to the just-buit 9.2 kernel syms and sources. > > com2 was set up as a debug port by dropping to the bhyveload prompt and > > hint.uart.1.flags="0x80" > > (this could also have been done in the guest's /boot/loader.conf) > > com2 was then set up in the bhyve command line to point to an nmdm device > > -l com2,/dev/nmdm0A > > The guest probed uart1 as a debug port: > > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 flags 0x80 on acpi0 > ioapic0: routing intpin 3 (ISA IRQ 3) to lapic 0 vector 64 > uart1: fast interrupt > uart1: debug port (9600,n,8,1) > > After booting, I loaded the the tap device in the guest to provide a > kld for kgdb to examine: > > root@fbsd9-2:~ # kldload if_tap > root@fbsd9-2:~ # kldstat > Id Refs Address Size Name > 1 3 0xffffffff80200000 15f92d8 kernel > 2 1 0xffffffff81a12000 59e9 if_tap.ko > > Now time to try kgdb: > > root@fbsd9-2:~ # sysctl debug.kdb.enter=1 > debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter > [ thread pid 577 tid 100054 ] > Stopped at kdb_enter+0x3b: movq $0,0xaf0362(%rip) > db> gdb > (ctrl-c will return control to ddb) > Switching to gdb back-end > Switching to gdb back-end > > In another window, cd'd to the mounted copy of the 9.2 disk: > > kgdb -r /dev/nmdm0B kernel.debug > ... > This GDB was configured as "amd64-marcel-freebsd"...Switching to remote > protocol > kdb_enter (why=0xffffffff80fbf671 "sysctl", msg=0x80
of bounds>) at ../../../kern/subr_kdb.c:441 > 441 kdb_why = KDB_WHY_UNSET; > > (kgdb) > > After some experimentation, I found the way to get the correct symbols > for the kld was to manually specify it: > > (kgdb) add-kld /mnt/boot/kernel/if_tap.ko > add symbol table from file "/mnt/boot/kernel/if_tap.ko.symbols" at > .text_addr = 0xffffffff81a12000 > .rodata.str1.8_addr = 0xffffffff81a13b50 > .rodata.str1.1_addr = 0xffffffff81a13ddb > set_sysinit_set_addr = 0xffffffff81a13f68 > set_modmetadata_set_addr = 0xffffffff81a13f80 > set_sysctl_set_addr = 0xffffffff81a13f90 > set_sysuninit_set_addr = 0xffffffff81a13fc0 > .data_addr = 0xffffffff81a13fe0 > .bss_addr = 0xffffffff81a14620 > (y or n) y > > However, 'info sharedlibrary' didn't seem to reflect this: > > (kgdb) info sharedlibrary > From To Syms Read Shared Object Library > 0xffffffff81a12000 0xffffffff81a13c04 No /boot/kernel/if_tap.ko.symbols > > Might have been a bug there, since I was able to successfully set > breakpoints in if_tap routines and have them trigger. > > I did have some trouble getting the source path set up correctly, but > never fully investigated that: seems like gdb is quite rich in that area > and it should be possible to get sorted. > > later, > > Peter. > > > Thanks for your help. It showed me one or two things that I didn't know before, and confirms some of the shortcomings of the current kgdb that others have noticed in trying to get this work. Namely, that the 'info sharedlibrary' command doesn't seem to work properly. -Kurt From owner-freebsd-virtualization@FreeBSD.ORG Tue May 6 06:55:37 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 358ADFE; Tue, 6 May 2014 06:55:37 +0000 (UTC) Received: from mail-oa0-x229.google.com (mail-oa0-x229.google.com [IPv6:2607:f8b0:4003:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6533B7B; Tue, 6 May 2014 06:55:36 +0000 (UTC) Received: by mail-oa0-f41.google.com with SMTP id m1so7232090oag.0 for ; Mon, 05 May 2014 23:55:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ex/nLBq9hXUwU7UBnfFwza9TcQZrVME+hbPQ0s8uC/Q=; b=Dbk3p4OfVU7j54PIUYyPDBR0BZWvL+tjqfri+DnZxQEKKzqbahG9ZojmK6Rv6bbDIN qVdGL6sg4WsSnoCQLjfcp3z8pAOiRlx9GteORkuEYtEDhdgVvNgc9N6NtAoByotRY40g +WDVVcupaLkyIsEwrRV9ZNhC1YGHvjFbIDmHmGA9jkfg+/HIo0yVgxScN+RfZxteOa8x tQuQFc8gUYqfTGfpTep2Kg/4F9sopjYAq5RFJrIuZuKOcVVUZyf2NammYPpRAqV3wuXy 3Wo6/BngfXEUF6QLe6hA1/bDPW+48keJZffFmiikH0HYCeDKHR04q0hpEzeIKjo3OBQK 9Sbw== MIME-Version: 1.0 X-Received: by 10.60.159.5 with SMTP id wy5mr710423oeb.58.1399359336200; Mon, 05 May 2014 23:55:36 -0700 (PDT) Received: by 10.76.173.229 with HTTP; Mon, 5 May 2014 23:55:36 -0700 (PDT) In-Reply-To: References: Date: Tue, 6 May 2014 08:55:36 +0200 Message-ID: Subject: Re: virt-manager bhyve From: Andreas Nilsson To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 06:55:37 -0000 On Tue, May 6, 2014 at 2:00 AM, Craig Rodrigues wrote: > On Mon, May 5, 2014 at 2:47 PM, Alexandar Narayan > wrote: > > Has anyone built virt-manager with bhyve support? > > I haven't tried any of this stuff, but you may want to read the blog > postings mentioned at this link: > > > http://www.freebsd.org/news/status/report-2014-01-2014-03.html#libvirt/bhyve-Support > > Those blog postings were written by Roman Bogorodskiy. Roman > has done a lot of work to get libvirt working with bhyve, but there seems > to be > some additional setup steps required. These steps are mentioned in > his blog postings. > > -- > Craig > I for one would like the libvirt package provided with pkgng to have have support for it. At least for 10-RELEASE. Best regards Andreas From owner-freebsd-virtualization@FreeBSD.ORG Tue May 6 15:21:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13AA2FA2 for ; Tue, 6 May 2014 15:21:22 +0000 (UTC) Received: from nqhost.drenet.net (nqhost.drenet.net [184.95.47.164]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E957DDB8 for ; Tue, 6 May 2014 15:21:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nqhost.drenet.net (Postfix) with ESMTP id C7D993ADD8 for ; Tue, 6 May 2014 11:21:19 -0400 (EDT) Received: from nqhost.drenet.net ([127.0.0.1]) by localhost (nqhost.drenet.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 55571-03 for ; Tue, 6 May 2014 11:21:19 -0400 (EDT) Received: by nqhost.drenet.net (Postfix, from userid 80) id 6CB683AD11; Tue, 6 May 2014 11:21:19 -0400 (EDT) To: freebsd-virtualization@freebsd.org Subject: Re: Kernel Panic on FreeBSD guests with libvirt 1.2.2 and qemu 1.7.91 X-PHP-Originating-Script: 80:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 06 May 2014 11:21:19 -0400 From: Andre Goree In-Reply-To: <566708d82c9575d14789b91544ebe0fa@drenet.net> References: <201405021818.s42IIJnu095511@enceladus10.kn-bremen.de> <566708d82c9575d14789b91544ebe0fa@drenet.net> Message-ID: <046432f6833ebfe0f71dada0a3fa779c@drenet.net> X-Sender: andre@drenet.net User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 15:21:22 -0000 On 05/02/2014 3:51 pm, Andre Goree wrote: > On 05/02/2014 3:42 pm, Andre Goree wrote: >> On 05/02/2014 2:18 pm, Juergen Lock wrote: >>> In article <201405021800.s42I0TiE094542@enceladus10.kn-bremen.de> you >>> write: >>>> In article >>>> >>>> you write: >>>> I think you have hit this issue for which the workaround is a patch >>>> to seabios: >>>> >>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug >>>> >>> Hm for some reason that link got truncated, try this one: >>> >>> https://bugs.debian.org/737142 >>> >>> >>> Juergen >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to >>> "freebsd-virtualization-unsubscribe@freebsd.org" >> >> Lol, yeah, I was gonna say...but thanks! Not sure if I'm into >> implementing a patch vs. finding a way around this, and I'm wondering >> why it reared it's head after an update to libvirt & qemu vs. earlier >> in my testing -- makes me think it may be an unrelated, albeit similar >> bug. Thanks again Juergen. > > Btw, from the link you posted: > > "From: Michael Tokarev > To: 737142-close@bugs.debian.org > Subject: Bug#737142: fixed in seabios 1.7.4-4 > Source: seabios > Source-Version: 1.7.4-4 > > We believe that the bug you reported is fixed in the latest version of > seabios, which is due to be installed in the Debian FTP archive." > > and on my host server having the issue: > "# dpkg -l |grep seabio > ii seabios 1.7.4-4~cloud0 > Legacy BIOS implementation" > > I'll try to determine which version of seabios I had running initially > before the issue, however from what I can tell, an upgrade to the > version that _should_ have fixed this appears be what caused my issue > in the first place, heh. Oddly enough, after downgrading seabios to version 1.7.3, I'm able to boot FreeBSD on the Ubuntu hosts without issue. I downgraded like so: apt-get install seabios=1.7.3-1ubuntu0.1~cloud0 No other packages (namely, libvirt/qemu/etc.) were downgraded as a result and I do not see any dependency issues, so it looks like this will be the workaround for now. I suppose I should make my situation know in that bug report, eh? Thanks again Juergen for pointing me in the right direction! -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://www.drenet.net PGP key - http://www.drenet.net/pubkey.txt -=-=-=-=-=- From owner-freebsd-virtualization@FreeBSD.ORG Wed May 7 05:49:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FA452C0; Wed, 7 May 2014 05:49:36 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B03A4622; Wed, 7 May 2014 05:49:35 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id s7so611900lbd.8 for ; Tue, 06 May 2014 22:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=x8Aq+1W7Q3m+e+nOxVdeCDws8ZsQmfZgMHtHn5ZzjPA=; b=S7fJ6Y7H5AWjqzGq2dL6S7+Jqu/Y1Vuj6oEZ56/93hbFzYZXQWZjzmim/D0EW/7zor ien36gzUl2uWrkMPZxmEF5n+0vOkbeRASOaTja/Mc+Ky6AWxJ4axsbHtynQgZ9M13hSW M3HZM9A+4PSOWRh7TfVhdfBMzJIz8jTiDAqnQRCvV35K7YPF5tUV9wWU5zCCvJUzwdey jYLOuJmX06yNvjX8P+FPlUQ+1vQlyC8mqe5ZzcaGvut1PuqcUa1OQlrXvXyny+wvNwie Nb+n257FwjJmkYIsH5zF/6Aa/7WYLUFiiF+cfI2hvCX25hGg6vYx9o+EQtlKS1Dtiu0S JeyA== X-Received: by 10.112.13.72 with SMTP id f8mr2317467lbc.40.1399441773430; Tue, 06 May 2014 22:49:33 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id ui5sm814028lbb.32.2014.05.06.22.49.02 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 May 2014 22:49:02 -0700 (PDT) Date: Wed, 7 May 2014 09:48:55 +0400 From: Roman Bogorodskiy To: Alexandar Narayan Subject: Re: virt-manager bhyve Message-ID: <20140507054854.GA53331@dev.san.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: jgh@FreeBSD.org, freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 05:49:36 -0000 Alexandar Narayan wrote: > Has anyone built virt-manager with bhyve support? I read that it was > supported just turned off by default. I installed virt-manager via >=20 > pkg install virt-manager >=20 > and can get it to run but I can't get it to see bhyve. >=20 > Thanks, Indeed, bhyve support was added to virt-manager some time ago. But there were no release with that support so far. Also, deskutils/virt-manager port is a little outdated, and, AFAIK, its updated is blocked by some outdated dependencies. CCed jgh@, he knows better about that. Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Wed May 7 21:55:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2245C78C for ; Wed, 7 May 2014 21:55:22 +0000 (UTC) Received: from mail.52pokupki.com (103-98-248-162-static.reverse.queryfoundry.net [162.248.98.103]) by mx1.freebsd.org (Postfix) with ESMTP id E9367304 for ; Wed, 7 May 2014 21:55:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=52pokupki.com; h=Date:From:To:Subject:MIME-Version:Content-Type:Message-ID; i=joint.health@52pokupki.com; bh=DYtcfDzytwCaySN3NU0VESjZ6lg=; b=pZ4IC3GEDuNuxe9XF35HSHzc7nxVEZVLR1KZB59CwFW1K9Z+yy8kQypX3KFWA1KDV97qnCDpEFQV 20xzYwR4F7O6odhe+Q839ncIRXhJrfe1Y29FMiVFoxkn0SCDNiDjarT9zV7PjrLsBlnzj1C1k8HW heXob4Wp038Z5RDeMps= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=dkim; d=52pokupki.com; b=cdUzH6S+Zrgzxl1e9By2/KuiSPTV3NnIGYI9bLxYfF2IP7hh4Rrh4b0q4LBTQrOjkThEu7kDoqtw /KiJd5O49bxTZZ1LEiVnPL8dtB9FUEoWG+10UtiN5aparhagzoYqVTdHCJICaUbRXMMpPHZe2+db Sa0tqQNviir5w5l50mQ=; Received: by mail.52pokupki.com id hdalsi0001gs for ; Wed, 7 May 2014 16:51:21 -0500 (envelope-from ) Date: Wed, 7 May 2014 16:51:21 -0500 From: "Joint Health" To: Subject: This Years Best Stocking Stuffer, Now At GNC Message-ID: <0.0.0.1D3.1CF6A3E7BCA2E26.40C9BA@mail.52pokupki.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 21:55:22 -0000 [1]Do You Suffer From Joint Pain? [2]Are You One of the 46 Million Americans Who Suffer from Joint Pain? New Study: Glucosamine, Chondroitin Offer No Aid for Joint Pain Glucosamine Under Fire After Proven Ineffective: If you're one of the millions of Americans who buy glucosamine or chondroitin supplements to easy joint pain, you may want to think twice. The America Academy of Orthapaedic Surgeons said this week that there's no reason for people to take the supplements, citing a number of recent studies that failed to show any benefit from them. The new analysis of 10 studies, comprising 3,803 people, reinforces the negative findings of GAIT trial. Glucosamine, chondroitin, or their combination are no better than placebo (dummy pills) when it comes to joint pain and joint space narrowing, the new study shows... [3]Read Full Article Here This is an advertisement. All trademarks, service marks, logos and/or domain names (including the names of products or retailers) are the property of their respective owners. The manufacturers, retailers or providers of the items offered may not have endorsed, approved of or otherwise sponsored this promotion. Restrictions apply. Void where prohibited by law. To opt-put/unsubscribe please [4]click here or write to: TFX Health, LLC PO Box 900164 Sandy, UT 84090-0164 [482BA1818BED17361EE90B86D12654FF3142C4EF067460A0336638518D0B2AF68CCCB3 5409F8EDF7CC9CA35445258DA1] References 1. http://www.52pokupki.com/u/oy1yCB7FVc/482BA1818BED17361EE90B86D12654FF3142C4EF067460A0336638518D0B2AF68CCCB35409F8EDF7CC9CA35445258DA1 2. http://www.52pokupki.com/u/oy1yCB7FVc/482BA1818BED17361EE90B86D12654FF3142C4EF067460A0336638518D0B2AF68CCCB35409F8EDF7CC9CA35445258DA1 3. http://www.52pokupki.com/u/oy1yCB7FVc/482BA1818BED17361EE90B86D12654FF3142C4EF067460A0336638518D0B2AF68CCCB35409F8EDF7CC9CA35445258DA1 4. http://www.52pokupki.com/u/E1pggw7dmt/482BA1818BED17361EE90B86D12654FF3142C4EF067460A0336638518D0B2AF68CCCB35409F8EDF7CC9CA35445258DA1 From owner-freebsd-virtualization@FreeBSD.ORG Wed May 7 23:15:52 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 859D4956; Wed, 7 May 2014 23:15:52 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50978BDC; Wed, 7 May 2014 23:15:51 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s47NFfkm021464 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 7 May 2014 19:15:41 -0400 (EDT) (envelope-from lists@jnielsen.net) From: John Nielsen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: consistent VM hang during reboot Date: Wed, 7 May 2014 17:15:43 -0600 Message-Id: To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-DCC-x.dcc-servers-Metrics: ns1.jnielsen.net 104; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 23:15:52 -0000 I am trying to solve a problem with amd64 FreeBSD virtual machines = running on a Linux+KVM hypervisor. To be honest I'm not sure if the = problem is in FreeBSD or the hypervisor, but I'm trying to rule out the = OS first. The _second_ time FreeBSD boots in a virtual machine with more than one = core, the boot hangs just before the kernel would normally print e.g. = "SMP: AP CPU #1 Launched!" (The last line on the console is "usbus0: = 12Mbps Full Speed USB v1.0", but the problem persists even without USB). = The VM will boot fine a first time, but running either "shutdown -r now" = OR "reboot" will lead to a hung second boot. Stopping and starting the = host qemu-kvm process is the only way to continue. The problem seems to be triggered by something in the SMP portion of = cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual = "reset" button the next boot is fine. If I have 'kern.smp.disabled=3D"1"' = set for the initial boot then subsequent boots are fine (but I can only = use one CPU core, of course). However, if I boot normally the first time = then set 'kern.smp.disabled=3D"1"' for the second (re)boot, the problem = is triggered. Apparently something in the shutdown code is "poisoning = the well" for the next boot. The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as of = yesterday. This (heavy-handed and wrong) patch (to HEAD) lets me avoid the issue: --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 13:19:07.400981580 = -0600 +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 = -0600 @@ -593,7 +593,7 @@ void cpu_reset() { -#ifdef SMP +#if 0 cpuset_t map; u_int cnt; I've tried skipping or disabling smaller chunks of code within the #if = block but haven't found a consistent winner yet. I'm hoping the list will have suggestions on how I can further narrow = down the problem, or theories on what might be going on. Thanks! JN From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 15:57:45 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E5B41F6 for ; Thu, 8 May 2014 15:57:45 +0000 (UTC) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0CD29B5 for ; Thu, 8 May 2014 15:57:44 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id BEEE028422 for ; Thu, 8 May 2014 17:57:35 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 9A5F628423 for ; Thu, 8 May 2014 17:57:34 +0200 (CEST) Message-ID: <536BA96E.5090800@quip.cz> Date: Thu, 08 May 2014 17:57:34 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Disk IO throttling for VM guests? Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 15:57:45 -0000 Is there any possibilities to limit disk IO for virtualization guest on FreeBSD? I would like to know, if it is possible to limit IOps for jails, or Bhyve guest, or VirtualBox quests. There are ways to limit CPU or RAM for them, but CPU and RAM are really huge these days. On the other hand, HDDs are still very IO limited and if one guest runs disk IO hungy task, then all other guest are affected / slow. I read about plugable GEOM scheduler few years ago (GEOM_SCHED), but it seems that it is dead project and there is no module for it to allow some scheduling according to PID, JID or something like this. So do we have anything like this for Jails or Bhyve? http://wiki.qemu.org/Features/DiskIOLimits http://wiki.smartos.org/display/DOC/Tuning+the+IO+Throttle Miroslav Lachman From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 16:20:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDA75922 for ; Thu, 8 May 2014 16:20:15 +0000 (UTC) Received: from mail-oa0-x22f.google.com (mail-oa0-x22f.google.com [IPv6:2607:f8b0:4003:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7CCA33E for ; Thu, 8 May 2014 16:20:15 +0000 (UTC) Received: by mail-oa0-f47.google.com with SMTP id i7so3365952oag.6 for ; Thu, 08 May 2014 09:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pmkhf2x4mR4oPPz6yn7plfdRPyYWk2n7EoAyKJImYmo=; b=KQ5fIQY/QPWCrTpd6xrV3dRSEB9PxgAsMnftQsaIJYIEBu6zpb732d/SHPEN1u6yPu z0hk9huyDOcrWVSY4QA7vNAqpdUQWVwFm5aMQvBjD9t8cHOaXl8SFfUVWbpOycuQ7fP+ me1ZbTnKM/Z3J0vEln0Vt5DC8Cy+LYrPyC0m62/Z2+puE8rfVZB5/Y2RFSONkU3BCRlG AehO2iHGEOm89+jFT/NjkG/u/gtXRqrHtE0Db9w0mbtmofSDytwziOC4Nm+Asdm4kudJ dt02tqzkvyl71yAwucXUDg3QRP9BsTexkw/mGwm0sBHANoWRJBWL/qRoGq1ZBmTAXsd9 96TQ== MIME-Version: 1.0 X-Received: by 10.60.161.101 with SMTP id xr5mr5032397oeb.71.1399566015096; Thu, 08 May 2014 09:20:15 -0700 (PDT) Received: by 10.76.170.39 with HTTP; Thu, 8 May 2014 09:20:14 -0700 (PDT) In-Reply-To: <536BA96E.5090800@quip.cz> References: <536BA96E.5090800@quip.cz> Date: Thu, 8 May 2014 18:20:14 +0200 Message-ID: Subject: Re: Disk IO throttling for VM guests? From: Andreas Nilsson To: Miroslav Lachman <000.fbsd@quip.cz> Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 16:20:16 -0000 On Thu, May 8, 2014 at 5:57 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: > Is there any possibilities to limit disk IO for virtualization guest on > FreeBSD? > I would like to know, if it is possible to limit IOps for jails, or Bhyve > guest, or VirtualBox quests. There are ways to limit CPU or RAM for them, > but CPU and RAM are really huge these days. On the other hand, HDDs are > still very IO limited and if one guest runs disk IO hungy task, then all > other guest are affected / slow. > > I read about plugable GEOM scheduler few years ago (GEOM_SCHED), but it > seems that it is dead project and there is no module for it to allow some > scheduling according to PID, JID or something like this. > > So do we have anything like this for Jails or Bhyve? > http://wiki.qemu.org/Features/DiskIOLimits > http://wiki.smartos.org/display/DOC/Tuning+the+IO+Throttle > > Miroslav Lachman > Well, there is rctl. I haven't tried it yet though. Best regards Andreas From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 17:26:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1318B80E; Thu, 8 May 2014 17:26:10 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFFA7BE3; Thu, 8 May 2014 17:26:09 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 69D71B97C; Thu, 8 May 2014 13:26:08 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: consistent VM hang during reboot Date: Thu, 8 May 2014 13:03:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405081303.17079.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 08 May 2014 13:26:08 -0400 (EDT) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 17:26:10 -0000 On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: > I am trying to solve a problem with amd64 FreeBSD virtual machines running on a Linux+KVM hypervisor. To be honest I'm not sure if the problem is in FreeBSD or the hypervisor, but I'm trying to rule out the OS first. > > The _second_ time FreeBSD boots in a virtual machine with more than one core, the boot hangs just before the kernel would normally print e.g. "SMP: AP CPU #1 Launched!" (The last line on the console is "usbus0: 12Mbps Full Speed USB v1.0", but the problem persists even without USB). The VM will boot fine a first time, but running either "shutdown -r now" OR "reboot" will lead to a hung second boot. Stopping and starting the host qemu-kvm process is the only way to continue. > > The problem seems to be triggered by something in the SMP portion of cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual "reset" button the next boot is fine. If I have 'kern.smp.disabled="1"' set for the initial boot then subsequent boots are fine (but I can only use one CPU core, of course). However, if I boot normally the first time then set 'kern.smp.disabled="1"' for the second (re)boot, the problem is triggered. Apparently something in the shutdown code is "poisoning the well" for the next boot. > > The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as of yesterday. > > This (heavy-handed and wrong) patch (to HEAD) lets me avoid the issue: > > --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 13:19:07.400981580 -0600 > +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 -0600 > @@ -593,7 +593,7 @@ > void > cpu_reset() > { > -#ifdef SMP > +#if 0 > cpuset_t map; > u_int cnt; > > I've tried skipping or disabling smaller chunks of code within the #if block but haven't found a consistent winner yet. > > I'm hoping the list will have suggestions on how I can further narrow down the problem, or theories on what might be going on. Can you try forcing the reboot to occur on the BSP (via 'cpuset -l 0 reboot') or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? It might not, but if it does it would help narrow down the code to consider. -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 17:55:57 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58B1461B; Thu, 8 May 2014 17:55:57 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 216DDE61; Thu, 8 May 2014 17:55:56 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s48HtqfL029562 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 8 May 2014 13:55:53 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: consistent VM hang during reboot From: John Nielsen In-Reply-To: <201405081303.17079.jhb@freebsd.org> Date: Thu, 8 May 2014 11:55:53 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201405081303.17079.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1874) X-DCC-sonic.net-Metrics: ns1.jnielsen.net 1156; Body=3 Fuz1=3 Fuz2=3 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org, freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 17:55:57 -0000 On May 8, 2014, at 11:03 AM, John Baldwin wrote: > On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >> I am trying to solve a problem with amd64 FreeBSD virtual machines = running on a Linux+KVM hypervisor. To be honest I'm not sure if the = problem is in FreeBSD or=20 > the hypervisor, but I'm trying to rule out the OS first. >>=20 >> The _second_ time FreeBSD boots in a virtual machine with more than = one core, the boot hangs just before the kernel would normally print = e.g. "SMP: AP CPU #1=20 > Launched!" (The last line on the console is "usbus0: 12Mbps Full Speed = USB v1.0", but the problem persists even without USB). The VM will boot = fine a first time,=20 > but running either "shutdown -r now" OR "reboot" will lead to a hung = second boot. Stopping and starting the host qemu-kvm process is the only = way to continue. >>=20 >> The problem seems to be triggered by something in the SMP portion of = cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual = "reset" button the next=20 > boot is fine. If I have 'kern.smp.disabled=3D"1"' set for the initial = boot then subsequent boots are fine (but I can only use one CPU core, of = course). However, if I=20 > boot normally the first time then set 'kern.smp.disabled=3D"1"' for = the second (re)boot, the problem is triggered. Apparently something in = the shutdown code is=20 > "poisoning the well" for the next boot. >>=20 >> The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as of = yesterday. >>=20 >> This (heavy-handed and wrong) patch (to HEAD) lets me avoid the = issue: >>=20 >> --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 = 13:19:07.400981580 -0600 >> +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 = -0600 >> @@ -593,7 +593,7 @@ >> void >> cpu_reset() >> { >> -#ifdef SMP >> +#if 0 >> cpuset_t map; >> u_int cnt; >>=20 >> I've tried skipping or disabling smaller chunks of code within the = #if block but haven't found a consistent winner yet. >>=20 >> I'm hoping the list will have suggestions on how I can further narrow = down the problem, or theories on what might be going on. >=20 > Can you try forcing the reboot to occur on the BSP (via 'cpuset -l 0 = reboot') > or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? It = might > not, but if it does it would help narrow down the code to consider. Hello jhb, thanks for responding. I tried your suggestion but unfortunately it does not make any = difference. The reboot hangs regardless of which CPU I assign the = command to. Any other suggestions? JN From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 18:42:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E30B2A9; Thu, 8 May 2014 18:42:36 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0139.outbound.protection.outlook.com [207.46.163.139]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16402354; Thu, 8 May 2014 18:42:34 +0000 (UTC) Received: from BY2PR05MB584.namprd05.prod.outlook.com (10.141.219.153) by BY2PR05MB773.namprd05.prod.outlook.com (10.141.224.140) with Microsoft SMTP Server (TLS) id 15.0.929.12; Thu, 8 May 2014 18:42:26 +0000 Received: from BY2PR05MB582.namprd05.prod.outlook.com (10.141.219.146) by BY2PR05MB584.namprd05.prod.outlook.com (10.141.219.153) with Microsoft SMTP Server (TLS) id 15.0.934.12; Thu, 8 May 2014 18:42:25 +0000 Received: from BY2PR05MB582.namprd05.prod.outlook.com ([10.141.219.146]) by BY2PR05MB582.namprd05.prod.outlook.com ([10.141.219.146]) with mapi id 15.00.0934.000; Thu, 8 May 2014 18:42:25 +0000 From: Andrew Duane To: John Nielsen , John Baldwin Subject: RE: consistent VM hang during reboot Thread-Topic: consistent VM hang during reboot Thread-Index: AQHPauL2mJCY45JsDEqXh7d069Bqsps297yAgAAMMTA= Date: Thu, 8 May 2014 18:42:24 +0000 Message-ID: References: <201405081303.17079.jhb@freebsd.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-forefront-prvs: 0205EDCD76 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(189002)(199002)(377454003)(24454002)(51704005)(52604005)(13464003)(81542001)(4396001)(74316001)(92566001)(21056001)(81342001)(80022001)(66066001)(86362001)(20776003)(64706001)(79102001)(31966008)(74502001)(74662001)(46102001)(83072002)(85852003)(99396002)(15975445006)(99286001)(2656002)(33646001)(76576001)(87936001)(15202345003)(50986999)(77096999)(76176999)(101416001)(54356999)(19580405001)(76482001)(83322001)(19580395003)(77982001)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR05MB584; H:BY2PR05MB582.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (: juniper.net does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=aduane@juniper.net; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Cc: "freebsd-hackers@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 18:42:36 -0000 When I was doing some early work on some of the Octeon multi-core chips, I = encountered something similar. If I remember correctly, there was an issue = in the shutdown sequence that did not properly halt the cores and set up th= e "start jump" vector. So the first core would start, and when it tried to = start the next ones it would hang waiting for the ACK that they were runnin= g (since they didn't have a start vector and hence never started). I know M= IPS, not AMD, so I can't say what the equivalent would be, but I'm sure the= re is one. Check that part, setting up the early state. If Juli and/or Adrian are reading this: do you remember anything about that= , something like 2 years ago? .................................... Andrew L. Duane AT&T Technical Lead JNCIA - JUNOS m=A0=A0=A0+1 603.770.7088 o +1 408.933.6944 (2-6944) skype: andrewlduane aduane@juniper.net -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freeb= sd.org] On Behalf Of John Nielsen Sent: Thursday, May 08, 2014 1:56 PM To: John Baldwin Cc: freebsd-hackers@freebsd.org; freebsd-virtualization@freebsd.org Subject: Re: consistent VM hang during reboot On May 8, 2014, at 11:03 AM, John Baldwin wrote: > On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >> I am trying to solve a problem with amd64 FreeBSD virtual machines runni= ng on a Linux+KVM hypervisor. To be honest I'm not sure if the problem is i= n FreeBSD or=20 > the hypervisor, but I'm trying to rule out the OS first. >>=20 >> The _second_ time FreeBSD boots in a virtual machine with more than one = core, the boot hangs just before the kernel would normally print e.g. "SMP:= AP CPU #1=20 > Launched!" (The last line on the console is "usbus0: 12Mbps Full Speed US= B v1.0", but the problem persists even without USB). The VM will boot fine = a first time,=20 > but running either "shutdown -r now" OR "reboot" will lead to a hung seco= nd boot. Stopping and starting the host qemu-kvm process is the only way to= continue. >>=20 >> The problem seems to be triggered by something in the SMP portion of cpu= _reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual "reset" = button the next=20 > boot is fine. If I have 'kern.smp.disabled=3D"1"' set for the initial boo= t then subsequent boots are fine (but I can only use one CPU core, of cours= e). However, if I=20 > boot normally the first time then set 'kern.smp.disabled=3D"1"' for the s= econd (re)boot, the problem is triggered. Apparently something in the shutd= own code is=20 > "poisoning the well" for the next boot. >>=20 >> The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as of ye= sterday. >>=20 >> This (heavy-handed and wrong) patch (to HEAD) lets me avoid the issue: >>=20 >> --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 13:19:07.400981580 -060= 0 >> +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 -0600 >> @@ -593,7 +593,7 @@ >> void >> cpu_reset() >> { >> -#ifdef SMP >> +#if 0 >> cpuset_t map; >> u_int cnt; >>=20 >> I've tried skipping or disabling smaller chunks of code within the #if b= lock but haven't found a consistent winner yet. >>=20 >> I'm hoping the list will have suggestions on how I can further narrow do= wn the problem, or theories on what might be going on. >=20 > Can you try forcing the reboot to occur on the BSP (via 'cpuset -l 0 rebo= ot') > or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? It mi= ght > not, but if it does it would help narrow down the code to consider. Hello jhb, thanks for responding. I tried your suggestion but unfortunately it does not make any difference. = The reboot hangs regardless of which CPU I assign the command to. Any other suggestions? JN _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Thu May 8 20:41:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4650DD04 for ; Thu, 8 May 2014 20:41:04 +0000 (UTC) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03469FE for ; Thu, 8 May 2014 20:41:03 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 1231128428; Thu, 8 May 2014 22:41:01 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id BFF6D28426; Thu, 8 May 2014 22:40:59 +0200 (CEST) Message-ID: <536BEBDB.405@quip.cz> Date: Thu, 08 May 2014 22:40:59 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: Andreas Nilsson Subject: Re: Disk IO throttling for VM guests? References: <536BA96E.5090800@quip.cz> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 20:41:04 -0000 Andreas Nilsson wrote: > On Thu, May 8, 2014 at 5:57 PM, Miroslav Lachman <000.fbsd@quip.cz > > wrote: > > Is there any possibilities to limit disk IO for virtualization guest > on FreeBSD? > I would like to know, if it is possible to limit IOps for jails, or > Bhyve guest, or VirtualBox quests. There are ways to limit CPU or > RAM for them, but CPU and RAM are really huge these days. On the > other hand, HDDs are still very IO limited and if one guest runs > disk IO hungy task, then all other guest are affected / slow. > > I read about plugable GEOM scheduler few years ago (GEOM_SCHED), but > it seems that it is dead project and there is no module for it to > allow some scheduling according to PID, JID or something like this. > > So do we have anything like this for Jails or Bhyve? > http://wiki.qemu.org/Features/ DiskIOLimits > http://wiki.smartos.org/ display/DOC/Tuning+the+IO+ Throttle > > Miroslav Lachman > > > Well, there is rctl. I haven't tried it yet though. > > Best regards > Andreas As far as I know, it is just another way to limit CPU, memory, swap, SysV semaphores, but no way to limit disk iops or bandwidth. Miroslav Lachman From owner-freebsd-virtualization@FreeBSD.ORG Fri May 9 18:42:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED346402; Fri, 9 May 2014 18:42:00 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CAF885EB; Fri, 9 May 2014 18:42:00 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s49Ifl8s046611 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 9 May 2014 14:41:48 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: consistent VM hang during reboot From: John Nielsen In-Reply-To: Date: Fri, 9 May 2014 12:41:50 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> References: <201405081303.17079.jhb@freebsd.org> To: Andrew Duane X-Mailer: Apple Mail (2.1874) X-DCC-x.dcc-servers-Metrics: ns1.jnielsen.net 104; Body=4 Fuz1=4 Fuz2=4 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean Cc: "freebsd-hackers@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 18:42:01 -0000 On May 8, 2014, at 12:42 PM, Andrew Duane wrote: > From: owner-freebsd-hackers@freebsd.org = [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of John Nielsen >=20 >> On May 8, 2014, at 11:03 AM, John Baldwin wrote: >>=20 >>> On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >>>> I am trying to solve a problem with amd64 FreeBSD virtual machines = running on a Linux+KVM hypervisor. To be honest I'm not sure if the = problem is in FreeBSD or=20 >>> the hypervisor, but I'm trying to rule out the OS first. >>>>=20 >>>> The _second_ time FreeBSD boots in a virtual machine with more than = one core, the boot hangs just before the kernel would normally print = e.g. "SMP: AP CPU #1=20 >>> Launched!" (The last line on the console is "usbus0: 12Mbps Full = Speed USB v1.0", but the problem persists even without USB). The VM will = boot fine a first time,=20 >>> but running either "shutdown -r now" OR "reboot" will lead to a hung = second boot. Stopping and starting the host qemu-kvm process is the only = way to continue. >>>>=20 >>>> The problem seems to be triggered by something in the SMP portion = of cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual = "reset" button the next=20 >>> boot is fine. If I have 'kern.smp.disabled=3D"1"' set for the = initial boot then subsequent boots are fine (but I can only use one CPU = core, of course). However, if I=20 >>> boot normally the first time then set 'kern.smp.disabled=3D"1"' for = the second (re)boot, the problem is triggered. Apparently something in = the shutdown code is=20 >>> "poisoning the well" for the next boot. >>>>=20 >>>> The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as = of yesterday. >>>>=20 >>>> This (heavy-handed and wrong) patch (to HEAD) lets me avoid the = issue: >>>>=20 >>>> --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 = 13:19:07.400981580 -0600 >>>> +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 = -0600 >>>> @@ -593,7 +593,7 @@ >>>> void >>>> cpu_reset() >>>> { >>>> -#ifdef SMP >>>> +#if 0 >>>> cpuset_t map; >>>> u_int cnt; >>>>=20 >>>> I've tried skipping or disabling smaller chunks of code within the = #if block but haven't found a consistent winner yet. >>>>=20 >>>> I'm hoping the list will have suggestions on how I can further = narrow down the problem, or theories on what might be going on. >>>=20 >>> Can you try forcing the reboot to occur on the BSP (via 'cpuset -l 0 = reboot') >>> or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? = It might >>> not, but if it does it would help narrow down the code to consider. >>=20 >> Hello jhb, thanks for responding. >>=20 >> I tried your suggestion but unfortunately it does not make any = difference. The reboot hangs regardless of which CPU I assign the = command to. >>=20 >> Any other suggestions? >=20 > When I was doing some early work on some of the Octeon multi-core = chips, I encountered something similar. If I remember correctly, there = was an issue in the shutdown sequence that did not properly halt the = cores and set up the "start jump" vector. So the first core would start, = and when it tried to start the next ones it would hang waiting for the = ACK that they were running (since they didn't have a start vector and = hence never started). I know MIPS, not AMD, so I can't say what the = equivalent would be, but I'm sure there is one. Check that part, setting = up the early state. >=20 > If Juli and/or Adrian are reading this: do you remember anything about = that, something like 2 years ago? That does sound promising, would love more details if anyone can provide = them. Here's another wrinkle: The KVM machine in question is part of a cluster of identical servers = (hardware, OS, software revisions). The problem is present on all = servers in the cluster. I also have access to a second homogenous cluster. The OS and software = revisions on this cluster are identical to the first. The hardware is = _nearly_ identical--slightly different mainboards from the same = manufacturer and slightly older CPUs. The same VMs (identical disk image = and definition, including CPU flags passed to the guest) that have a = problem on the first cluster work flawlessly on this one. Not sure if that means the bad behavior only appears on certain CPUs or = if it's timing-related or something else entirely. I'd welcome = speculation at this point. CPU details below in case it makes a difference. =3D=3D Problem Host =3D=3D model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand = lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi = flexpriority ept vpid fsgsbase smep erms =3D=3D Good Host =3D=3D model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat = epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid Thanks, JN From owner-freebsd-virtualization@FreeBSD.ORG Mon May 12 11:06:53 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E12DDC55 for ; Mon, 12 May 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDB5626E4 for ; Mon, 12 May 2014 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4CB6rM8068017 for ; Mon, 12 May 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4CB6r7b068015 for freebsd-virtualization@FreeBSD.org; Mon, 12 May 2014 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 12 May 2014 11:06:53 GMT Message-Id: <201405121106.s4CB6r7b068015@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon May 12 13:00:44 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EB68C79 for ; Mon, 12 May 2014 13:00:44 +0000 (UTC) Received: from mail-oa0-x22e.google.com (mail-oa0-x22e.google.com [IPv6:2607:f8b0:4003:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 344482328 for ; Mon, 12 May 2014 13:00:44 +0000 (UTC) Received: by mail-oa0-f46.google.com with SMTP id i4so8142121oah.33 for ; Mon, 12 May 2014 06:00:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=FDSqp6ejX+nocLNvjvnZkMGpNP/H4qVTPulOJebAOpU=; b=bPa+DEDPdvs+YkwJ9BCOo5xM7z/ayQdFqIF9XBQo3bBXOwHRCkae3K6fiLiZKuKwdQ VE6kueN8TdQKaXK+GuJKZzihCG5zqNULL89YHCMd9IIFiN9J3pOmm4hrvBRuQEUdcx2g hJa1YRnanYFPdKLNtYtZwlEwbYlkr36rOVjgbh7KuLPnqWMDWbdi9wh7YS9/pYZuLZZk fNXOIrZhL3Py3ntTQHMl+KQnnGlE8vTEeU9DfzO66KaYrnwXxT7hqLeuXh3+30xrubT3 8vivtSqapNHaeUDBfc3BfHXK03uR4FG9e5XMcJnVCZeuVd9bAozya3TegxbNyY9azByE lGRQ== MIME-Version: 1.0 X-Received: by 10.60.62.178 with SMTP id z18mr3587417oer.61.1399899643410; Mon, 12 May 2014 06:00:43 -0700 (PDT) Received: by 10.182.33.136 with HTTP; Mon, 12 May 2014 06:00:43 -0700 (PDT) Date: Mon, 12 May 2014 15:00:43 +0200 Message-ID: Subject: =?UTF-8?Q?=5BVHPC=E2=80=9914=5D_LAST_Call_for_Papers_=2D_Deadline_in_4_wee?= =?UTF-8?Q?ks?= From: VHPC 14 To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 13:00:44 -0000 ================================================================= CALL FOR PAPERS 9th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '14) held in conjunction with Euro-Par 2014, August 25-29, Porto, Portugal (Springer LNCS) ================================================================= Date: August 26, 2014 Workshop URL: http://vhpc.org Paper Submission Deadline: June 9, 2014 (extended) Confirmed Keynote Speakers: Ron Brightwell, Sandia National Laboratory Hobbes: Using Virtualization to Enable Exascale Applications and Helge Meinhard, CERN CALL FOR PAPERS Virtualization technologies constitute a key enabling factor for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to dynamically manage complex infrastructures in a seamless fashion for varying workloads and hosted applications, independently of the customers deploying software or users submitting highly dynamic and heterogeneous workloads. Thanks to virtualization, we have the ability to manage vast computing and networking resources dynamically and close to the marginal cost of providing the services, which is unprecedented in the history of scientific and commercial computing. Various virtualization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple under-utilized servers with heterogeneous software and operating systems (OSes), and its capability to live-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-level virtualization, with its capability to isolate multiple user-space environments and to allow for their co-existence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical NICs/HBAs to take traffic from multiple VMs; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology and IP addressing, provides the fundamental ground on top of which evolved network services can be realized with an unprecedented level of dynamicity and flexibility; the increasingly adopted paradigm of Software-Defined Networking (SDN) promises to extend this flexibility to the control and data planes of network paths. These technologies have to be inter-mixed and integrated in an intelligent way, to support workloads that are increasingly demanding in terms of absolute performance, responsiveness and interactivity, and have to respect well-specified Service- Level Agreements (SLAs), as needed for industrial-grade provided services. Indeed, among emerging and increasingly interesting application domains for virtualization, we can find big-data application workloads in cloud infrastructures, interactive and real-time multimedia services in the cloud, including real-time big-data streaming platforms such as used in real-time analytics supporting nowadays a plethora of application domains. Distributed cloud infrastructures promise to offer unprecedented responsiveness levels for hosted applications, but that is only possible if the underlying virtualization technologies can overcome most of the latency impairments typical of current virtualized infrastructures (e.g., far worse tail-latency). What is more, in data communications Network Function Virtualization (NFV) is becoming a key technology enabling a shift from supplying hardware-based network functions, to providing them in a software-based and elastic way. In conjunction with (public and private) cloud technologies, NFV may be used for constructing the foundation for cost-effective network functions that can easily and seamlessly adapt to demand, still keeping their major carrier-grade characteristics in terms of QoS and reliability. The Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, and lightning talks, limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. TOPICS Topics of interest include, but are not limited to: - Management, deployment and monitoring of virtualized environments - Language-process virtual machines - Performance monitoring for virtualized/cloud workloads - Virtual machine monitor platforms - Topology management and optimization for distributed virtualized applications - Paravirtualized I/O - Improving I/O and network virtualization including use of RDMA, Infiniband, PCIe - Improving performance in VM access to GPUs, GPU clusters, GP-GPUs - HPC storage virtualization - Virtualized systems for big-data and analytics workloads - Optimizations and enhancements to OS virtualization support - Improving OS-level virtualization and its integration within cloud management - Performance modelling for virtualized/cloud applications - Heterogeneous virtualized environments - Parallel virtualized - virtualization aware file systems - Network virtualization - Software defined networking - Network function virtualization - Hypervisor and network virtualization QoS and SLAs - Cloudbursting - Evolved European grid architectures including such based on network virtualization - Workload characterization for VM-based environments - Optimized communication libraries/protocols in the cloud - System and process/bytecode VM convergence - Cloud frameworks and APIs - Checkpointing/migration of VM-based large compute jobs - Job scheduling/control/policy with VMs - Instrumentation interfaces and languages - VMM performance (auto-)tuning on various load types - Cloud reliability, fault-tolerance, and security - Research, industrial and educational use cases - Virtualization in cloud, cluster and grid environments - Cross-layer VM optimizations - Cloud HPC use cases including optimizations - Services in cloud HPC - Hypervisor extensions and tools for cluster and grid computing - Cluster provisioning in the cloud - Performance and cost modelling - Languages for describing highly-distributed compute jobs - VM cloud and cluster distribution algorithms, load balancing - Instrumentation interfaces and languages - Energy-aware virtualization Important Dates Rolling Paper registration June 9, 2014 - Full paper submission (extended) July 4, 2014 - Acceptance notification October 3, 2014 - Camera-ready version due August 26, 2014 - Workshop Date TPC CHAIR Michael Alexander (chair), TU Wien, Austria Anastassios Nanos (co-chair), NTUA, Greece Tommaso Cucinotta (co-chair), Bell Labs, Dublin, Ireland PROGRAM COMMITTEE Costas Bekas, IBM Jakob Blomer, CERN Roberto Canonico, University of Napoli Federico II, Italy Piero Castoldi, Sant'Anna School of Advanced Studies Paolo Costa, MS Research Cambridge, England Jorge Ejarque Artigas, Barcelona Supercomputing Center, Spain William Gardner, University of Guelph, USA Balazs Gerofi, University of Tokyo, Japan Krishna Kant, Temple University, USA Romeo Kinzler, IBM Nectarios Koziris, National Technical University of Athens, Greece Giuseppe Lettieri, University of Pisa, Italy Jean-Marc Menaud, Ecole des Mines de Nantes, France Christine Morin, INRIA, France Dimitrios Nikolopoulos, Queen's University of Belfast, UK Herbert Poetzl, VServer, Austria Luigi Rizzo, University of Pisa, Italy Josh Simons, VMware, USA Borja Sotomayor, University of Chicago, USA Vangelis Tasoulas, Simula Research Lab, Norway Yoshio Turner, HP Labs, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden Chao-Tung Yang, Tunghai University, Taiwan PAPER SUBMISSION-PUBLICATION Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, key words, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in the Springer LNCS series - the format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Format Guidelines: http://www.springer.de/comp/lncs/authors.html EasyChair Abstract Submission Link: https://www.easychair.org/conferences/?conf=europar2014ws GENERAL INFORMATION The workshop is one day in length and will be held in conjunction with Euro-Par 2014, 25-29 August, Porto, Portugal From owner-freebsd-virtualization@FreeBSD.ORG Mon May 12 20:09:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00991753 for ; Mon, 12 May 2014 20:09:24 +0000 (UTC) Received: from mailserver.itaserv.net (nexus.itaserv.net [5.9.118.236]) by mx1.freebsd.org (Postfix) with ESMTP id AB51E29FC for ; Mon, 12 May 2014 20:09:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailserver.itaserv.net (Postfix) with ESMTP id C53E66C00D1 for ; Mon, 12 May 2014 22:01:57 +0200 (CEST) Received: from mailserver.itaserv.net ([127.0.0.1]) by localhost (mailserver.itaserv.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id EQClYP7cylF2 for ; Mon, 12 May 2014 22:01:57 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mailserver.itaserv.net (Postfix) with ESMTP id 76B596C00CF for ; Mon, 12 May 2014 22:01:57 +0200 (CEST) X-Virus-Scanned: amavisd-new at itaserv.net Received: from mailserver.itaserv.net ([127.0.0.1]) by localhost (mailserver.itaserv.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yHPuc1xj1_Rt for ; Mon, 12 May 2014 22:01:57 +0200 (CEST) Received: from mailserver.itaserv.net (mailserver.itaserv.net [10.255.255.253]) by mailserver.itaserv.net (Postfix) with ESMTP id 5CCAC6C009E for ; Mon, 12 May 2014 22:01:57 +0200 (CEST) Date: Mon, 12 May 2014 22:01:57 +0200 (CEST) From: Brando Beaumont To: freebsd-virtualization@freebsd.org Message-ID: <1950869458.737.1399924917328.JavaMail.zimbra@itaserv.net> In-Reply-To: <1183991895.735.1399924759341.JavaMail.zimbra@itaserv.net> Subject: bhyve rocks! MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [78.134.16.125] X-Mailer: Zimbra 8.0.7_GA_6021 (ZimbraWebClient - GC37 (Mac)/8.0.7_GA_6021) Thread-Topic: bhyve rocks! Thread-Index: fvJxRlzabYBzZQP0KYGdhOpLkw8JyQ== X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 20:09:25 -0000 Good evening, I know that this might not be the best place to write this kind of messages but... I must say thanks to the creators of bhyve and all of you, who gave me support some time ago. Thanks to you guys i now have a bhyve host ( running FreeBSD 11-CURRENT ) with 5 linux guests running on it! Keep up the good work guys! :-) From owner-freebsd-virtualization@FreeBSD.ORG Mon May 12 20:45:59 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B05C9DB for ; Mon, 12 May 2014 20:45:59 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id F275D2E51 for ; Mon, 12 May 2014 20:45:58 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 66E4A12529; Tue, 13 May 2014 06:45:58 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-24-9-162-48.hsd1.co.comcast.net [24.9.162.48] (may be forged)) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUD32482 (AUTH peterg@ptree32.com.au); Tue, 13 May 2014 06:45:57 +1000 Message-ID: <53713302.5090907@freebsd.org> Date: Mon, 12 May 2014 14:45:54 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Brando Beaumont Subject: Re: bhyve rocks! References: <1950869458.737.1399924917328.JavaMail.zimbra@itaserv.net> In-Reply-To: <1950869458.737.1399924917328.JavaMail.zimbra@itaserv.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 20:45:59 -0000 > I must say thanks to the creators of bhyve and all of you, who gave > me support some time ago. Thanks to you guys i now have a bhyve host > ( running FreeBSD 11-CURRENT ) with 5 linux guests running on it! Great ! Any particular versions of Linux ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon May 12 20:52:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E236BBCC; Mon, 12 May 2014 20:52:30 +0000 (UTC) Received: from mailserver.itaserv.net (nexus.itaserv.net [5.9.118.236]) by mx1.freebsd.org (Postfix) with ESMTP id 9A0A52F06; Mon, 12 May 2014 20:52:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailserver.itaserv.net (Postfix) with ESMTP id A00ED6C00CF; Mon, 12 May 2014 22:52:45 +0200 (CEST) Received: from mailserver.itaserv.net ([127.0.0.1]) by localhost (mailserver.itaserv.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ynTAEuW0Bkvy; Mon, 12 May 2014 22:52:45 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mailserver.itaserv.net (Postfix) with ESMTP id 5DF4C6C00D0; Mon, 12 May 2014 22:52:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at itaserv.net Received: from mailserver.itaserv.net ([127.0.0.1]) by localhost (mailserver.itaserv.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id sj9a3AHKFomd; Mon, 12 May 2014 22:52:45 +0200 (CEST) Received: from mailserver.itaserv.net (mailserver.itaserv.net [10.255.255.253]) by mailserver.itaserv.net (Postfix) with ESMTP id 4A1736C00CF; Mon, 12 May 2014 22:52:45 +0200 (CEST) Date: Mon, 12 May 2014 22:52:45 +0200 (CEST) From: Brando Beaumont To: Peter Grehan Message-ID: <835809863.756.1399927965241.JavaMail.zimbra@itaserv.net> In-Reply-To: <53713302.5090907@freebsd.org> References: <1950869458.737.1399924917328.JavaMail.zimbra@itaserv.net> <53713302.5090907@freebsd.org> Subject: Re: bhyve rocks! MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [78.134.16.125] X-Mailer: Zimbra 8.0.7_GA_6021 (ZimbraWebClient - GC37 (Mac)/8.0.7_GA_6021) Thread-Topic: bhyve rocks! Thread-Index: sCVWUNIhNruodXC5CxJC8GnH4jsuwA== Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 20:52:31 -0000 Two Guests are running Ubuntu 14.04 while the others are running CentOS 6.5= (this was a bit tricky, but i found in a previous post the method to get i= t to boot correctly! ). I'm planning to test RHEL7 Beta this weekend. I'm also running a guest equipped with Ubuntu 12.04 and i'm running Zimbra = inside of it! ----- Messaggio originale ----- Da: "Peter Grehan" A: "Brando Beaumont" Cc: freebsd-virtualization@freebsd.org Inviato: Luned=C3=AC, 12 maggio 2014 22:45:54 Oggetto: Re: bhyve rocks! > I must say thanks to the creators of bhyve and all of you, who gave > me support some time ago. Thanks to you guys i now have a bhyve host > ( running FreeBSD 11-CURRENT ) with 5 linux guests running on it! Great ! Any particular versions of Linux ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue May 13 15:50:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55FA6A05; Tue, 13 May 2014 15:50:06 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 35228235B; Tue, 13 May 2014 15:50:05 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s4DFnvs1019554 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 13 May 2014 11:49:57 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: consistent VM hang during reboot From: John Nielsen In-Reply-To: <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> Date: Tue, 13 May 2014 09:50:00 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <83DA2398-0004-49EC-8AC1-9AA64F33A194@jnielsen.net> References: <201405081303.17079.jhb@freebsd.org> <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> To: "freebsd-hackers@freebsd.org" , "freebsd-virtualization@freebsd.org" X-Mailer: Apple Mail (2.1874) X-DCC--Metrics: ns1.jnielsen.net 1282; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 15:50:06 -0000 On May 9, 2014, at 12:41 PM, John Nielsen wrote: > On May 8, 2014, at 12:42 PM, Andrew Duane wrote: >=20 >> From: owner-freebsd-hackers@freebsd.org = [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of John Nielsen >>=20 >>> On May 8, 2014, at 11:03 AM, John Baldwin wrote: >>>=20 >>>> On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >>>>> I am trying to solve a problem with amd64 FreeBSD virtual machines = running on a Linux+KVM hypervisor. To be honest I'm not sure if the = problem is in FreeBSD or=20 >>>> the hypervisor, but I'm trying to rule out the OS first. >>>>>=20 >>>>> The _second_ time FreeBSD boots in a virtual machine with more = than one core, the boot hangs just before the kernel would normally = print e.g. "SMP: AP CPU #1=20 >>>> Launched!" (The last line on the console is "usbus0: 12Mbps Full = Speed USB v1.0", but the problem persists even without USB). The VM will = boot fine a first time,=20 >>>> but running either "shutdown -r now" OR "reboot" will lead to a = hung second boot. Stopping and starting the host qemu-kvm process is the = only way to continue. >>>>>=20 >>>>> The problem seems to be triggered by something in the SMP portion = of cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual = "reset" button the next=20 >>>> boot is fine. If I have 'kern.smp.disabled=3D"1"' set for the = initial boot then subsequent boots are fine (but I can only use one CPU = core, of course). However, if I=20 >>>> boot normally the first time then set 'kern.smp.disabled=3D"1"' for = the second (re)boot, the problem is triggered. Apparently something in = the shutdown code is=20 >>>> "poisoning the well" for the next boot. >>>>>=20 >>>>> The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT as = of yesterday. >>>>>=20 >>>>> This (heavy-handed and wrong) patch (to HEAD) lets me avoid the = issue: >>>>>=20 >>>>> --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 = 13:19:07.400981580 -0600 >>>>> +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 = -0600 >>>>> @@ -593,7 +593,7 @@ >>>>> void >>>>> cpu_reset() >>>>> { >>>>> -#ifdef SMP >>>>> +#if 0 >>>>> cpuset_t map; >>>>> u_int cnt; >>>>>=20 >>>>> I've tried skipping or disabling smaller chunks of code within the = #if block but haven't found a consistent winner yet. >>>>>=20 >>>>> I'm hoping the list will have suggestions on how I can further = narrow down the problem, or theories on what might be going on. >>>>=20 >>>> Can you try forcing the reboot to occur on the BSP (via 'cpuset -l = 0 reboot') >>>> or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? = It might >>>> not, but if it does it would help narrow down the code to consider. >>>=20 >>> Hello jhb, thanks for responding. >>>=20 >>> I tried your suggestion but unfortunately it does not make any = difference. The reboot hangs regardless of which CPU I assign the = command to. >>>=20 >>> Any other suggestions? >>=20 >> When I was doing some early work on some of the Octeon multi-core = chips, I encountered something similar. If I remember correctly, there = was an issue in the shutdown sequence that did not properly halt the = cores and set up the "start jump" vector. So the first core would start, = and when it tried to start the next ones it would hang waiting for the = ACK that they were running (since they didn't have a start vector and = hence never started). I know MIPS, not AMD, so I can't say what the = equivalent would be, but I'm sure there is one. Check that part, setting = up the early state. >>=20 >> If Juli and/or Adrian are reading this: do you remember anything = about that, something like 2 years ago? >=20 > That does sound promising, would love more details if anyone can = provide them. >=20 > Here's another wrinkle: >=20 > The KVM machine in question is part of a cluster of identical servers = (hardware, OS, software revisions). The problem is present on all = servers in the cluster. >=20 > I also have access to a second homogenous cluster. The OS and software = revisions on this cluster are identical to the first. The hardware is = _nearly_ identical--slightly different mainboards from the same = manufacturer and slightly older CPUs. The same VMs (identical disk image = and definition, including CPU flags passed to the guest) that have a = problem on the first cluster work flawlessly on this one. >=20 > Not sure if that means the bad behavior only appears on certain CPUs = or if it's timing-related or something else entirely. I'd welcome = speculation at this point. >=20 > CPU details below in case it makes a difference. >=20 > =3D=3D Problem Host =3D=3D > model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand = lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi = flexpriority ept vpid fsgsbase smep erms >=20 > =3D=3D Good Host =3D=3D > model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat = epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid Still haven't found a solution but I did learn something else = interesting: an ACPI reboot allows the system to come back up = successfully. What is different from the system or CPU point of view = about an ACPI reboot versus running "reboot" or "shutdown" from = userland? JN From owner-freebsd-virtualization@FreeBSD.ORG Thu May 15 09:11:11 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA5BBE24 for ; Thu, 15 May 2014 09:11:11 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DF532226C for ; Thu, 15 May 2014 09:11:10 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA07271 for ; Thu, 15 May 2014 12:11:02 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Wkrgs-0003if-1J for freebsd-virtualization@freebsd.org; Thu, 15 May 2014 12:11:02 +0300 Message-ID: <53748481.8010108@FreeBSD.org> Date: Thu, 15 May 2014 12:10:25 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-virtualization@FreeBSD.org Subject: bhyve: svm (amd-v) update X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 09:11:11 -0000 It seems that the bhyve_svm branch is a bit behind the latest interface changes in head. Is anyone working on sync-ing up the branch with the head? Some examples: - change of init method in vmm_ops - addition of resume, vlapic_init and vlapic_cleanup methods to vmm_ops - replacement of lapic_pending_intr and lapic_intr_accepted with vlapic_pending_intr and vlapic_intr_accepted - changes in struct vm_exit - VMEXIT_EPT_FAULT ==> VMEXIT_NESTED_FAULT Thanks! -- Andriy Gapon From owner-freebsd-virtualization@FreeBSD.ORG Thu May 15 15:56:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E300870; Thu, 15 May 2014 15:56:10 +0000 (UTC) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41EA5263A; Thu, 15 May 2014 15:56:09 +0000 (UTC) Received: by mail-we0-f176.google.com with SMTP id q59so1268613wes.21 for ; Thu, 15 May 2014 08:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6XQJDrQwJiodaRswZKFrcEvzOcOmxGczj5O/XgHd2i8=; b=f2bAtr7b6ip+qpaUcIPZfJRRjQaqB7smy8wMjEtuypUuyHQev+Z5USkrR9uy9Ct4mN 1vALIjVHO6CkNetr0SBsmgJqfVpOArDpWsu6PnHfQVNJdU/8EHn+OdudizE460dX9GAc 0wnpW+4T30RfS8FaVv5edmLMO2uUuQzL7zl07SAh6rRZ8SxVcFxHv0VyaCyIcj2f1lVy +F1UrnWe9txGFrqJo7hiDiR4W07leuhiqwYYihyCBHXy3VAiqXqZQRl+Aj4Gwf3TGP6h h1cQESv3rtNehnXvhyg7aJ6fFnhkwrAyhHf/W5kuYSDJ1gRMiaR6JV4E25PF5gzsE4kg rTBQ== MIME-Version: 1.0 X-Received: by 10.194.61.193 with SMTP id s1mr9416631wjr.36.1400169367290; Thu, 15 May 2014 08:56:07 -0700 (PDT) Received: by 10.194.135.49 with HTTP; Thu, 15 May 2014 08:56:07 -0700 (PDT) In-Reply-To: <53748481.8010108@FreeBSD.org> References: <53748481.8010108@FreeBSD.org> Date: Thu, 15 May 2014 08:56:07 -0700 Message-ID: Subject: Re: bhyve: svm (amd-v) update From: Anish To: Andriy Gapon Content-Type: multipart/mixed; boundary=047d7b86cdc2c5d01904f97255e5 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD virtualization , Neel Natu X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 15:56:10 -0000 --047d7b86cdc2c5d01904f97255e5 Content-Type: text/plain; charset=UTF-8 Hi Andriy, Thanks for your interest in SVM port of bhyve. I do have patch to sync it to http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). If patches looks good to you, we can submit it. I have been testing it on Phenom box which lacks some of newer SVM features. Thanks and regards, Anish On Thu, May 15, 2014 at 2:10 AM, Andriy Gapon wrote: > > It seems that the bhyve_svm branch is a bit behind the latest interface > changes > in head. Is anyone working on sync-ing up the branch with the head? > > Some examples: > - change of init method in vmm_ops > - addition of resume, vlapic_init and vlapic_cleanup methods to vmm_ops > - replacement of lapic_pending_intr and lapic_intr_accepted with > vlapic_pending_intr and vlapic_intr_accepted > - changes in struct vm_exit > - VMEXIT_EPT_FAULT ==> VMEXIT_NESTED_FAULT > > Thanks! > -- > Andriy Gapon > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > --047d7b86cdc2c5d01904f97255e5 Content-Type: application/octet-stream; name="bhyve_svm_HEAD_r263780.patch" Content-Disposition: attachment; filename="bhyve_svm_HEAD_r263780.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hv880xux0 ZGlmZiAtYXVyUCAteCAnKi5vcmlnJyAteCAuc3ZuIC9tbnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1f SEVBRF9yMjYzNzgwX2Jhc2UvL3N5cy9hbWQ2NC92bW0vYW1kL25wdC5jIC9tbnQvc3luY19uZXh0 Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX1dJUC9zeXMvYW1kNjQvdm1tL2FtZC9ucHQuYwotLS0g L21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3IyNjM3ODBfYmFzZS8vc3lzL2FtZDY0L3Zt bS9hbWQvbnB0LmMJMjAxNC0wNC0xMSAxMzo1MTo0NS4wMDAwMDAwMDAgKzAwMDAKKysrIC9tbnQv c3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX1dJUC9zeXMvYW1kNjQvdm1tL2FtZC9u cHQuYwkyMDE0LTA0LTA2IDA0OjM0OjE3LjAwMDAwMDAwMCArMDAwMApAQCAtNTIsMTQgKzUyLDE3 IEBACiBzdGF0aWMgaW50IG5wdF9mbGFnczsKIFNZU0NUTF9JTlQoX2h3X3ZtbV9ucHQsIE9JRF9B VVRPLCBwbWFwX2ZsYWdzLCBDVExGTEFHX1JELAogCSZucHRfZmxhZ3MsIDAsIE5VTEwpOworCisj ZGVmaW5lIE5QVF9JUElNQVNLCTB4RkYKIC8qCiAgKiBBTUQgbmVzdGVkIHBhZ2UgdGFibGUgaW5p dC4KICAqLwogaW50Ci1zdm1fbnB0X2luaXQodm9pZCkKK3N2bV9ucHRfaW5pdChpbnQgaXBpbnVt KQogewogCWludCBlbmFibGVfc3VwZXJwYWdlID0gMTsKIAorCW5wdF9mbGFncyA9IGlwaW51bSAm IE5QVF9JUElNQVNLOwogCVRVTkFCTEVfSU5UX0ZFVENIKCJody52bW0ubnB0LmVuYWJsZV9zdXBl cnBhZ2UiLCAmZW5hYmxlX3N1cGVycGFnZSk7CiAJaWYgKGVuYWJsZV9zdXBlcnBhZ2UpCiAJCW5w dF9mbGFncyB8PSBQTUFQX1BERV9TVVBFUlBBR0U7IApkaWZmIC1hdXJQIC14ICcqLm9yaWcnIC14 IC5zdm4gL21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3IyNjM3ODBfYmFzZS8vc3lzL2Ft ZDY0L3ZtbS9hbWQvbnB0LmggL21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3IyNjM3ODBf V0lQL3N5cy9hbWQ2NC92bW0vYW1kL25wdC5oCi0tLSAvbW50L3N5bmNfbmV4dDIvYmh5dmVfc3Zt X0hFQURfcjI2Mzc4MF9iYXNlLy9zeXMvYW1kNjQvdm1tL2FtZC9ucHQuaAkyMDE0LTA0LTExIDEz OjUxOjQ1LjAwMDAwMDAwMCArMDAwMAorKysgL21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFE X3IyNjM3ODBfV0lQL3N5cy9hbWQ2NC92bW0vYW1kL25wdC5oCTIwMTQtMDQtMDYgMDQ6MzQ6MTcu MDAwMDAwMDAwICswMDAwCkBAIC0zMSw3ICszMSw3IEBACiAKIHN0cnVjdCBzdm1fc29mdGM7CiAK LWludCAJc3ZtX25wdF9pbml0KHZvaWQpOworaW50IAlzdm1fbnB0X2luaXQoaW50IGlwaW51bSk7 CiBzdHJ1Y3QJdm1zcGFjZSAqc3ZtX25wdF9hbGxvYyh2bV9vZmZzZXRfdCBtaW4sIHZtX29mZnNl dF90IG1heCk7CiB2b2lkCXN2bV9ucHRfZnJlZShzdHJ1Y3Qgdm1zcGFjZSAqdm1zcGFjZSk7CiAj ZW5kaWYgLyogX1NWTV9OUFRfSF8gKi8KZGlmZiAtYXVyUCAteCAnKi5vcmlnJyAteCAuc3ZuIC9t bnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX2Jhc2UvL3N5cy9hbWQ2NC92bW0v YW1kL3N2bS5jIC9tbnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX1dJUC9zeXMv YW1kNjQvdm1tL2FtZC9zdm0uYwotLS0gL21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3Iy NjM3ODBfYmFzZS8vc3lzL2FtZDY0L3ZtbS9hbWQvc3ZtLmMJMjAxNC0wNC0xMSAxMzo1MTo0NS4w MDAwMDAwMDAgKzAwMDAKKysrIC9tbnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgw X1dJUC9zeXMvYW1kNjQvdm1tL2FtZC9zdm0uYwkyMDE0LTA0LTEyIDExOjUxOjUzLjAwMDAwMDAw MCArMDAwMApAQCAtNDYsNiArNDYsNyBAQAogI2luY2x1ZGUgPG1hY2hpbmUvc3BlY2lhbHJlZy5o PgogI2luY2x1ZGUgPG1hY2hpbmUvc2VnbWVudHMuaD4KICNpbmNsdWRlIDxtYWNoaW5lL3ZtbS5o PgorI2luY2x1ZGUgPG1hY2hpbmUvdm1tX2Rldi5oPgogCiAjaW5jbHVkZSA8eDg2L2FwaWNyZWcu aD4KIApAQCAtNTMsNiArNTQsOCBAQAogI2luY2x1ZGUgInZtbV9tc3IuaCIKICNpbmNsdWRlICJ2 bW1fc3RhdC5oIgogI2luY2x1ZGUgInZtbV9rdHIuaCIKKyNpbmNsdWRlICJ2bGFwaWMuaCIKKyNp bmNsdWRlICJ2bGFwaWNfcHJpdi5oIgogCiAjaW5jbHVkZSAieDg2LmgiCiAjaW5jbHVkZSAidm1j Yi5oIgpAQCAtNzUsNiArNzgsNyBAQAogI2RlZmluZSBBTURfQ1BVSURfU1ZNX1BBVVNFX0ZUSAkJ QklUKDEyKSAvKiBQYXVzZSBmaWx0ZXIgdGhyZXNob2xkICovCiAKIE1BTExPQ19ERUZJTkUoTV9T Vk0sICJzdm0iLCAic3ZtIik7CitNQUxMT0NfREVGSU5FKE1fU1ZNX1ZMQVBJQywgInN2bS12bGFw aWMiLCAic3ZtLXZsYXBpYyIpOwogCiAvKiBQZXItQ1BVIGNvbnRleHQgYXJlYS4gKi8KIGV4dGVy biBzdHJ1Y3QgcGNwdSBfX3BjcHVbXTsKQEAgLTI0OSw3ICsyNTMsNyBAQAogICogRW5hYmxlIFNW TSBvbiBDUFUgYW5kIGluaXRpYWxpemUgbmVzdGVkIHBhZ2UgdGFibGUgaC93LgogICovCiBzdGF0 aWMgaW50Ci1zdm1faW5pdCh2b2lkKQorc3ZtX2luaXQoaW50IGlwaW51bSkKIHsKIAlpbnQgZXJy OwogCkBAIC0yNTgsNyArMjYyLDcgQEAKIAkJcmV0dXJuIChlcnIpOwogCQogCi0Jc3ZtX25wdF9p bml0KCk7CisJc3ZtX25wdF9pbml0KGlwaW51bSk7CiAJCiAJLyogU3RhcnQgU1ZNIG9uIGFsbCBD UFVzICovCiAJc21wX3JlbmRlenZvdXMoTlVMTCwgc3ZtX2VuYWJsZSwgTlVMTCwgTlVMTCk7CkBA IC0yNjYsNiArMjcwLDExIEBACiAJcmV0dXJuICgwKTsKIH0KIAorc3RhdGljIHZvaWQKK3N2bV9y ZXN0b3JlKHZvaWQpCit7CisJc3ZtX2VuYWJsZShOVUxMKTsKK30JCQogLyoKICAqIEdldCBpbmRl eCBhbmQgYml0IHBvc2l0aW9uIGZvciBhIE1TUiBpbiBNU1IgcGVybWlzc2lvbgogICogYml0bWFw LiBUd28gYml0cyBhcmUgdXNlZCBmb3IgZWFjaCBNU1IsIGxvd2VyIGJpdCBpcwpAQCAtMzY1LDcg KzM3NCw2IEBACiAJCiAJcmV0dXJuICgwKTsKIH0KLQkJCiAvKgogICogSW5pdGlhbGlzZSBhIHZp cnR1YWwgbWFjaGluZS4KICAqLwpAQCAtNTE5LDYgKzUyNywyOSBAQAogCX0KIH0KIAorc3RhdGlj IGVudW0gdmllX2NwdV9tb2RlCitzdm1fdmNwdV9tb2RlKHVpbnQ2NF90IGVmZXIpCit7CisKKwlp ZiAoZWZlciAmIEVGRVJfTE1BKQorCQlyZXR1cm4gKENQVV9NT0RFXzY0QklUKTsKKwllbHNlCisJ CXJldHVybiAoQ1BVX01PREVfQ09NUEFUSUJJTElUWSk7Cit9CisKK3N0YXRpYyBlbnVtIHZpZV9w YWdpbmdfbW9kZQorc3ZtX3BhZ2luZ19tb2RlKHVpbnQ2NF90IGNyMCwgdWludDY0X3QgY3I0LCB1 aW50NjRfdCBlZmVyKQoreworCisJaWYgKChjcjAgJiBDUjBfUEcpID09IDApCisJCXJldHVybiAo UEFHSU5HX01PREVfRkxBVCk7CisJaWYgKChjcjQgJiBDUjRfUEFFKSA9PSAwKQorCQlyZXR1cm4g KFBBR0lOR19NT0RFXzMyKTsKKwlpZiAoZWZlciAmIEVGRVJfTE1FKQorCQlyZXR1cm4gKFBBR0lO R19NT0RFXzY0KTsKKwllbHNlCisJCXJldHVybiAoUEFHSU5HX01PREVfUEFFKTsKK30KIC8qCiAg KiBEZXRlcm1pbmUgdGhlIGNhdXNlIG9mIHZpcnR1YWwgY3B1IGV4aXQgYW5kIGhhbmRsZSBWTUVY SVQuCiAgKiBSZXR1cm46IGZhbHNlIC0gQnJlYWsgdmNwdSBleGVjdXRpb24gbG9vcCBhbmQgaGFu ZGxlIHZtZXhpdApAQCAtNTQ3LDcgKzU3OCw3IEBACiAJdXBkYXRlX3JpcCA9IHRydWU7CiAJbG9v cCA9IHRydWU7CiAJdm1leGl0LT5leGl0Y29kZSA9IFZNX0VYSVRDT0RFX1ZNWDsKLQl2bWV4aXQt PnUudm14LmVycm9yID0gMDsKKwl2bWV4aXQtPnUudm14LnN0YXR1cyA9IDA7CiAKIAlzd2l0Y2gg KGNvZGUpIHsKIAkJY2FzZQlWTUNCX0VYSVRfTUM6IC8qIE1hY2hpbmUgQ2hlY2suICovCkBAIC02 NjcsNyArNjk4LDYgQEAKIAkJY2FzZSBWTUNCX0VYSVRfTlBGOgogCQkJbG9vcCA9IGZhbHNlOwog CQkJdXBkYXRlX3JpcCA9IGZhbHNlOwotCQkJdm1tX3N0YXRfaW5jcihzdm1fc2MtPnZtLCB2Y3B1 LCBWTUVYSVRfRVBUX0ZBVUxULCAxKTsKIAogICAgICAgICAJCWlmIChpbmZvMSAmIFZNQ0JfTlBG X0lORk8xX1JTVikgewogIAkJCQlWQ1BVX0NUUjIoc3ZtX3NjLT52bSwgdmNwdSwgIlNWTV9FUlI6 TlBUIgpAQCAtNjg2LDE1ICs3MTYsMjQgQEAKIAkJCQl2bWV4aXQtPnUucGFnaW5nLmdwYSA9IGlu Zm8yOwogCQkJCXZtZXhpdC0+dS5wYWdpbmcuZmF1bHRfdHlwZSA9IAogCQkJCQlzdm1fbnBmX3Bh Z2luZyhpbmZvMSk7CisJCQkJdm1tX3N0YXRfaW5jcihzdm1fc2MtPnZtLCB2Y3B1LCAKKwkJCQkJ Vk1FWElUX05FU1RFRF9GQVVMVCwgMSk7CiAJCQl9IGVsc2UgaWYgKHN2bV9ucGZfZW11bF9mYXVs dChpbmZvMSkpIHsKLSAJCQkJVkNQVV9DVFIzKHN2bV9zYy0+dm0sIHZjcHUsICJTVk06TlBGLWlu c3RfZW11bCwiCisgCQkJCVZDUFVfQ1RSMyhzdm1fc2MtPnZtLCB2Y3B1LCAiU1ZNOk5QRiBpbnN0 X2VtdWwsIgogCQkJCQkiUklQOjB4JWx4IElORk8xOjB4JWx4IElORk8yOjB4JWx4IC5cbiIsCiAJ CQkJCXN0YXRlLT5yaXAsIGluZm8xLCBpbmZvMik7CiAJCQkJdm1leGl0LT5leGl0Y29kZSA9IFZN X0VYSVRDT0RFX0lOU1RfRU1VTDsKIAkJCQl2bWV4aXQtPnUuaW5zdF9lbXVsLmdwYSA9IGluZm8y OwogCQkJCXZtZXhpdC0+dS5pbnN0X2VtdWwuZ2xhID0gVklFX0lOVkFMSURfR0xBOwogCQkJCXZt ZXhpdC0+dS5pbnN0X2VtdWwuY3IzID0gc3RhdGUtPmNyMzsKKwkJCQl2bWV4aXQtPnUuaW5zdF9l bXVsLmNwdV9tb2RlID0gCisJCQkJCXN2bV92Y3B1X21vZGUoc3RhdGUtPmVmZXIpOworCQkJCXZt ZXhpdC0+dS5pbnN0X2VtdWwucGFnaW5nX21vZGUgPQorCQkJCQlzdm1fcGFnaW5nX21vZGUoc3Rh dGUtPmNyMCwgc3RhdGUtPmNyNCwKKwkJCQkJCSBzdGF0ZS0+ZWZlcik7CiAJCQkJdm1leGl0LT5p bnN0X2xlbmd0aCA9IFZJRV9JTlNUX1NJWkU7CisJCQkJdm1tX3N0YXRfaW5jcihzdm1fc2MtPnZt LCB2Y3B1LCAKKwkJCQkJVk1FWElUX0lOU1RfRU1VTCwgMSk7CiAJCQl9CiAJCQkKIAkJCWJyZWFr OwpAQCAtNzYyLDcgKzgwMSw3IEBACiAJCXJldHVybiAoMCk7CiAKIAkgLyogSW5qZWN0IE5NSSwg dmVjdG9yIG51bWJlciBpcyBub3QgdXNlZC4qLwotCWlmICh2bWNiX2V2ZW50aW5qZWN0KGN0cmws IFZNX05NSSwgSURUX05NSSwgMCwgZmFsc2UpKSB7CisJaWYgKHZtY2JfZXZlbnRpbmplY3QoY3Ry bCwgVk1DQl9FVkVOVElOSl9UWVBFX05NSSwgSURUX05NSSwgMCwgZmFsc2UpKSB7CiAJCVZDUFVf Q1RSMChzdm1fc2MtPnZtLCB2Y3B1LCAiU1ZNOk5NSSBpbmplY3Rpb24gZmFpbGVkLlxuIik7CiAJ CXJldHVybiAoRUlPKTsKIAl9CkBAIC03NzksMTAgKzgxOCwxMSBAQAogICogSW5qZWN0IGV2ZW50 IHRvIHZpcnR1YWwgY3B1LgogICovCiBzdGF0aWMgdm9pZAotc3ZtX2lual9pbnRlcnJ1cHRzKHN0 cnVjdCBzdm1fc29mdGMgKnN2bV9zYywgaW50IHZjcHUpCitzdm1faW5qX2ludGVycnVwdHMoc3Ry dWN0IHN2bV9zb2Z0YyAqc3ZtX3NjLCBpbnQgdmNwdSwgc3RydWN0IHZsYXBpYyAqdmxhcGljKQog ewogCXN0cnVjdCB2bWNiX2N0cmwgKmN0cmw7CiAJc3RydWN0IHZtY2Jfc3RhdGUgKnN0YXRlOwor CXN0cnVjdCB2bV9leGNlcHRpb24gZXhjOwogCWludCB2ZWN0b3I7CiAJCiAJS0FTU0VSVCh2Y3B1 IDwgc3ZtX3NjLT52Y3B1X2NudCwgKCJHdWVzdCBkb2Vzbid0IGhhdmUgVkNQVSVkIiwgdmNwdSkp OwpAQCAtODAyLDE3ICs4NDIsMjYgQEAKIAkJVkNQVV9DVFIwKHN2bV9zYy0+dm0sIHZjcHUsICJT Vk06R3Vlc3QgaW4gaW50ZXJydXB0IHNoYWRvdy5cbiIpOwogCQlyZXR1cm47CiAJfQotCisJCisJ aWYgKHZtX2V4Y2VwdGlvbl9wZW5kaW5nKHN2bV9zYy0+dm0sIHZjcHUsICZleGMpKSB7CisJCUtB U1NFUlQoZXhjLnZlY3RvciA+PSAwICYmIGV4Yy52ZWN0b3IgPCAzMiwKKwkJCSgiRXhjZXB0aW9u IHZlY3RvciUgaW52YWxpZCIsIGV4Yy52ZWN0b3IpKTsKKwkJaWYgKHZtY2JfZXZlbnRpbmplY3Qo Y3RybCwgVk1DQl9FVkVOVElOSl9UWVBFX0VYQ0VQVElPTiwgCisJCQlleGMudmVjdG9yLCBleGMu ZXJyb3JfY29kZSwgCisJCQlleGMuZXJyb3JfY29kZV92YWxpZCkpIHsKKwkJCVZDUFVfQ1RSMShz dm1fc2MtPnZtLCB2Y3B1LCAiU1ZNOkV4Y2VwdGlvbiVkIGluamVjdGlvbiIKKwkJCQkiIGZhaWxl ZC5cbiIsIGV4Yy52ZWN0b3IpOworCQkJcmV0dXJuOworCQl9CisJfQogCS8qIE5NSSBldmVudCBo YXMgcHJpb3JpdHkgb3ZlciBpbnRlcnJ1cHRzLiovCiAJaWYgKHN2bV9pbmplY3Rfbm1pKHN2bV9z YywgdmNwdSkpIHsKIAkJcmV0dXJuOwogCX0KIAotCXZlY3RvciA9IGxhcGljX3BlbmRpbmdfaW50 cihzdm1fc2MtPnZtLCB2Y3B1KTsKLQotCS8qIE5vIGludGVycnVwdCBpcyBwZW5kaW5nLiAqLwkK LQlpZiAodmVjdG9yIDwgMCkKLQkJcmV0dXJuOworICAgICAgICAvKiBBc2sgdGhlIGxvY2FsIGFw aWMgZm9yIGEgdmVjdG9yIHRvIGluamVjdCAqLworICAgICAgICBpZiAoIXZsYXBpY19wZW5kaW5n X2ludHIodmxhcGljLCAmdmVjdG9yKSkKKyAgICAgICAgICAgICAgICByZXR1cm47CiAJCiAJaWYg KHZlY3RvciA8IDMyIHx8IHZlY3RvciA+IDI1NSkgewogCQlWQ1BVX0NUUjEoc3ZtX3NjLT52bSwg dmNwdSwgIlNWTV9FUlI6RXZlbnQgaW5qZWN0aW9uIgpAQCAtODI2LDE0ICs4NzUsMTQgQEAKIAkJ cmV0dXJuOwogCX0KIAotCWlmICh2bWNiX2V2ZW50aW5qZWN0KGN0cmwsIFZNX0hXX0lOVFIsIHZl Y3RvciwgMCwgZmFsc2UpKSB7CisJaWYgKHZtY2JfZXZlbnRpbmplY3QoY3RybCwgVk1DQl9FVkVO VElOSl9UWVBFX0lOVFIsIHZlY3RvciwgMCwgZmFsc2UpKSB7CiAJCVZDUFVfQ1RSMShzdm1fc2Mt PnZtLCB2Y3B1LCAiU1ZNOkV2ZW50IGluamVjdGlvbiBmYWlsZWQgdG8iCiAJCQkiIHZlY3Rvcj0l ZC5cbiIsIHZlY3Rvcik7CiAJCXJldHVybjsKIAl9CQogCiAJLyogQWNrbm93bGVkZ2UgdGhhdCBl dmVudCBpcyBhY2NlcHRlZC4qLwotCWxhcGljX2ludHJfYWNjZXB0ZWQoc3ZtX3NjLT52bSwgdmNw dSwgdmVjdG9yKTsKKwl2bGFwaWNfaW50cl9hY2NlcHRlZCh2bGFwaWMsIHZlY3Rvcik7CiAJVkNQ VV9DVFIxKHN2bV9zYy0+dm0sIHZjcHUsICJTVk06ZXZlbnQgaW5qZWN0ZWQsdmVjdG9yPSVkLlxu IiwgdmVjdG9yKTsKIH0KIApAQCAtODg3LDcgKzkzNiw4IEBACiAgKiBTdGFydCB2Y3B1IHdpdGgg c3BlY2lmaWVkIFJJUC4KICAqLwogc3RhdGljIGludAotc3ZtX3ZtcnVuKHZvaWQgKmFyZywgaW50 IHZjcHUsIHJlZ2lzdGVyX3QgcmlwLCBwbWFwX3QgcG1hcCkKK3N2bV92bXJ1bih2b2lkICphcmcs IGludCB2Y3B1LCByZWdpc3Rlcl90IHJpcCwgcG1hcF90IHBtYXAsIAorCXZvaWQgKnJlbmRfY29v a2llLCB2b2lkICpzdXNwZW5kZWRfY29va2llKQogewogCXN0cnVjdCBzdm1fcmVnY3R4ICpoY3R4 LCAqZ2N0eDsKIAlzdHJ1Y3Qgc3ZtX3NvZnRjICpzdm1fc2M7CkBAIC04OTUsMTcgKzk0NSwyMSBA QAogCXN0cnVjdCB2bWNiX3N0YXRlICpzdGF0ZTsKIAlzdHJ1Y3Qgdm1jYl9jdHJsICpjdHJsOwog CXN0cnVjdCB2bV9leGl0ICp2bWV4aXQ7CisJc3RydWN0IHZsYXBpYyAqdmxhcGljOworCXN0cnVj dCB2bSAqdm07CiAJdWludDY0X3Qgdm1jYl9wYTsKIAlzdGF0aWMgdWludDY0X3QgaG9zdF9jcjI7 CiAJYm9vbCBsb29wOwkvKiBDb250aW51ZSB2Y3B1IGV4ZWN1dGlvbiBsb29wLiAqLwogCiAJbG9v cCA9IHRydWU7CiAJc3ZtX3NjID0gYXJnOwotCQorCXZtID0gc3ZtX3NjLT52bTsKKwogCXZjcHVz dGF0ZSA9IHN2bV9nZXRfdmNwdShzdm1fc2MsIHZjcHUpOwogCXN0YXRlID0gc3ZtX2dldF92bWNi X3N0YXRlKHN2bV9zYywgdmNwdSk7CiAJY3RybCA9IHN2bV9nZXRfdm1jYl9jdHJsKHN2bV9zYywg dmNwdSk7Ci0Jdm1leGl0ID0gdm1fZXhpdGluZm8oc3ZtX3NjLT52bSwgdmNwdSk7CisJdm1leGl0 ID0gdm1fZXhpdGluZm8odm0sIHZjcHUpOworCXZsYXBpYyA9IHZtX2xhcGljKHZtLCB2Y3B1KTsK IAogCWdjdHggPSBzdm1fZ2V0X2d1ZXN0X3JlZ2N0eChzdm1fc2MsIHZjcHUpOwogCWhjdHggPSAm aG9zdF9jdHhbY3VyY3B1XTsgCkBAIC05MTMsNyArOTY3LDcgQEAKIAogCWlmICh2Y3B1c3RhdGUt Pmxhc3RjcHUgIT0gY3VyY3B1KSB7CiAJCS8qIFZpcnR1YWwgQ1BVIGlzIHJ1bm5pbmcgb24gYSBk aWlmZXJlbnQgQ1BVIG5vdy4qLwotCQl2bW1fc3RhdF9pbmNyKHN2bV9zYy0+dm0sIHZjcHUsIFZD UFVfTUlHUkFUSU9OUywgMSk7CisJCXZtbV9zdGF0X2luY3Iodm0sIHZjcHUsIFZDUFVfTUlHUkFU SU9OUywgMSk7CiAKIAkJLyoKIAkJICogRmx1c2ggYWxsIFRMQiBtYXBwaW5nIGZvciB0aGlzIGd1 ZXN0IG9uIHRoaXMgQ1BVLApAQCAtOTQ2LDcgKzEwMDAsNyBAQAogCX0KIAogCXZjcHVzdGF0ZS0+ bGFzdGNwdSA9IGN1cmNwdTsKLQlWQ1BVX0NUUjMoc3ZtX3NjLT52bSwgdmNwdSwgIlNWTTpFbnRl ciB2bXJ1biBvbGQgUklQOjB4JWx4IgorCVZDUFVfQ1RSMyh2bSwgdmNwdSwgIlNWTTpFbnRlciB2 bXJ1biBvbGQgUklQOjB4JWx4IgogCQkiIG5ldyBSSVA6MHglbHggaW5zdCBsZW49JWRcbiIsCiAJ CXN0YXRlLT5yaXAsIHJpcCwgdm1leGl0LT5pbnN0X2xlbmd0aCk7CiAJLyogVXBkYXRlIEd1ZXN0 IFJJUCAqLwpAQCAtOTU3LDkgKzEwMTEsMjUgQEAKIAkJIC8qIFdlIGFyZSBhc2tlZCB0byBnaXZl IHRoZSBjcHUgYnkgc2NoZWR1bGVyLiAqLwogCQlpZiAoY3VydGhyZWFkLT50ZF9mbGFncyAmIChU REZfQVNUUEVORElORyB8IFRERl9ORUVEUkVTQ0hFRCkpIHsKIAkJCXZtZXhpdC0+ZXhpdGNvZGUg PSBWTV9FWElUQ09ERV9CT0dVUzsKLQkJCXZtbV9zdGF0X2luY3Ioc3ZtX3NjLT52bSwgdmNwdSwg Vk1FWElUX0FTVFBFTkRJTkcsIDEpOwotCQkJVkNQVV9DVFIxKHN2bV9zYy0+dm0sIHZjcHUsIAot CQkJCSJTVk06IGdhdmUgdXAgQ1BVLCBSSVA6MHglbHhcbiIsIHN0YXRlLT5yaXApOworCQkJdm1t X3N0YXRfaW5jcih2bSwgdmNwdSwgVk1FWElUX0FTVFBFTkRJTkcsIDEpOworCQkJVkNQVV9DVFIx KHZtLCB2Y3B1LCAKKwkJCQkiU1ZNOiBBU1RQRU5ESU5HLCBSSVA6MHglbHhcbiIsIHN0YXRlLT5y aXApOworCQkJdm1leGl0LT5yaXAgPSBzdGF0ZS0+cmlwOworCQkJYnJlYWs7CisJCX0KKworCQlp ZiAodmNwdV9zdXNwZW5kZWQoc3VzcGVuZGVkX2Nvb2tpZSkpIHsKKwkJCXZtZXhpdC0+ZXhpdGNv ZGUgPSBWTV9FWElUQ09ERV9TVVNQRU5ERUQ7CisJCQl2bWV4aXQtPnJpcCA9IHN0YXRlLT5yaXA7 CisJCQlicmVhazsKKwkJfQorCisJCWlmICh2Y3B1X3JlbmRlenZvdXNfcGVuZGluZyhyZW5kX2Nv b2tpZSkpIHsKKwkJCXZtZXhpdC0+ZXhpdGNvZGUgPSBWTV9FWElUQ09ERV9SRU5ERVpWT1VTOwor CQkJdm1tX3N0YXRfaW5jcih2bSwgdmNwdSwgVk1FWElUX1JFTkRFWlZPVVMsIDEpOworCQkJVkNQ VV9DVFIxKHZtLCB2Y3B1LCAKKwkJCQkiU1ZNOiBWQ1BVIHJlbmRlenZvdXMsIFJJUDoweCVseFxu IiwgCisJCQkJc3RhdGUtPnJpcCk7CiAJCQl2bWV4aXQtPnJpcCA9IHN0YXRlLT5yaXA7CiAJCQli cmVhazsKIAkJfQpAQCAtOTY4LDcgKzEwMzgsNyBAQAogCQkKIAkJc3ZtX2hhbmRsZV9leGl0aW50 aW5mbyhzdm1fc2MsIHZjcHUpOwogCQkKLQkJKHZvaWQpc3ZtX2lual9pbnRlcnJ1cHRzKHN2bV9z YywgdmNwdSk7CisJCSh2b2lkKXN2bV9pbmpfaW50ZXJydXB0cyhzdm1fc2MsIHZjcHUsIHZsYXBp Yyk7CiAJCiAJCS8qIENoYW5nZSBUU1MgdHlwZSB0byBhdmFpbGFibGUuKi8KIAkJc2V0dXBfdHNz X3R5cGUoKTsKQEAgLTEwMTcsNyArMTA4Nyw3IEBACiAJCS8qIEhhbmRsZSAjVk1FWElUIGFuZCBp ZiByZXF1aXJlZCByZXR1cm4gdG8gdXNlciBzcGFjZS4gKi8KIAkJbG9vcCA9IHN2bV92bWV4aXQo c3ZtX3NjLCB2Y3B1LCB2bWV4aXQpOwogCQl2Y3B1c3RhdGUtPmxvb3ArKzsKLQkJdm1tX3N0YXRf aW5jcihzdm1fc2MtPnZtLCB2Y3B1LCBWTUVYSVRfQ09VTlQsIDEpOworCQl2bW1fc3RhdF9pbmNy KHZtLCB2Y3B1LCBWTUVYSVRfQ09VTlQsIDEpOwogCiAJfSB3aGlsZSAobG9vcCk7CiAJCQpAQCAt MTIwOSwyNCArMTI3OSw2IEBACiB9CiAKIHN0YXRpYyBpbnQKLXN2bV9pbmplY3RfZXZlbnQodm9p ZCAqYXJnLCBpbnQgdmNwdSwgaW50IHR5cGUsIGludCB2ZWN0b3IsCi0JCSAgdWludDMyX3QgZXJy b3IsIGludCBlY192YWxpZCkKLXsKLQlzdHJ1Y3Qgc3ZtX3NvZnRjICpzdm1fc2M7Ci0Jc3RydWN0 IHZtY2JfY3RybCAqY3RybDsKLQotCXN2bV9zYyA9IGFyZzsKLQlLQVNTRVJUKHZjcHUgPCBzdm1f c2MtPnZjcHVfY250LCAoIkd1ZXN0IGRvZXNuJ3QgaGF2ZSBWQ1BVJWQiLCB2Y3B1KSk7Ci0KLQlj dHJsID0gc3ZtX2dldF92bWNiX2N0cmwoc3ZtX3NjLCB2Y3B1KTsKLQlWQ1BVX0NUUjMoc3ZtX3Nj LT52bSwgdmNwdSwgIkluamVjdGluZyBldmVudCB0eXBlOjB4JXggdmVjdG9yOjB4JXgiCi0JCSJl cnJvcjoweCV4XG4iLCB0eXBlLCB2ZWN0b3IsIGVycm9yKTsKLQotCXJldHVybiAodm1jYl9ldmVu dGluamVjdChjdHJsLCB0eXBlLCB2ZWN0b3IsIGVycm9yLAotCQllY192YWxpZCA/IFRSVUUgOiBG QUxTRSkpOwotfQotCi1zdGF0aWMgaW50CiBzdm1fc2V0Y2FwKHZvaWQgKmFyZywgaW50IHZjcHUs IGludCB0eXBlLCBpbnQgdmFsKQogewogCXN0cnVjdCBzdm1fc29mdGMgKnN2bV9zYzsKQEAgLTEz MjQsOSArMTM3NiwzNSBAQAogCXJldHVybiAoMCk7CiB9CiAKK3N0YXRpYyBzdHJ1Y3QgdmxhcGlj ICoKK3N2bV92bGFwaWNfaW5pdCh2b2lkICphcmcsIGludCB2Y3B1aWQpCit7CisJc3RydWN0IHN2 bV9zb2Z0YyAqc3ZtX3NjOworCXN0cnVjdCB2bGFwaWMgKnZsYXBpYzsKKworCXN2bV9zYyA9IGFy ZzsKKwl2bGFwaWMgPSBtYWxsb2Moc2l6ZW9mKHN0cnVjdCB2bGFwaWMpLCBNX1NWTV9WTEFQSUMs IE1fV0FJVE9LIHwgTV9aRVJPKTsKKwl2bGFwaWMtPnZtID0gc3ZtX3NjLT52bTsKKwl2bGFwaWMt PnZjcHVpZCA9IHZjcHVpZDsKKwl2bGFwaWMtPmFwaWNfcGFnZSA9IChzdHJ1Y3QgTEFQSUMgKikm c3ZtX3NjLT5hcGljX3BhZ2VbdmNwdWlkXTsKKworCXZsYXBpY19pbml0KHZsYXBpYyk7CisJCisJ cmV0dXJuICh2bGFwaWMpOworfQorCitzdGF0aWMgdm9pZAorc3ZtX3ZsYXBpY19jbGVhbnVwKHZv aWQgKmFyZywgc3RydWN0IHZsYXBpYyAqdmxhcGljKQoreworCisgICAgICAgIHZsYXBpY19jbGVh bnVwKHZsYXBpYyk7CisgICAgICAgIGZyZWUodmxhcGljLCBNX1NWTV9WTEFQSUMpOworfQorCiBz dHJ1Y3Qgdm1tX29wcyB2bW1fb3BzX2FtZCA9IHsKIAlzdm1faW5pdCwKIAlzdm1fY2xlYW51cCwK Kwlzdm1fcmVzdG9yZSwKIAlzdm1fdm1pbml0LAogCXN2bV92bXJ1biwKIAlzdm1fdm1jbGVhbnVw LApAQCAtMTMzNCw5ICsxNDEyLDEwIEBACiAJc3ZtX3NldHJlZywKIAlzdm1fZ2V0ZGVzYywKIAlz dm1fc2V0ZGVzYywKLQlzdm1faW5qZWN0X2V2ZW50LAogCXN2bV9nZXRjYXAsCiAJc3ZtX3NldGNh cCwKIAlzdm1fbnB0X2FsbG9jLAotCXN2bV9ucHRfZnJlZQorCXN2bV9ucHRfZnJlZSwKKwlzdm1f dmxhcGljX2luaXQsCisJc3ZtX3ZsYXBpY19jbGVhbnVwCQogfTsKZGlmZiAtYXVyUCAteCAnKi5v cmlnJyAteCAuc3ZuIC9tbnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX2Jhc2Uv L3N5cy9hbWQ2NC92bW0vYW1kL3N2bV9zb2Z0Yy5oIC9tbnQvc3luY19uZXh0Mi9iaHl2ZV9zdm1f SEVBRF9yMjYzNzgwX1dJUC9zeXMvYW1kNjQvdm1tL2FtZC9zdm1fc29mdGMuaAotLS0gL21udC9z eW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3IyNjM3ODBfYmFzZS8vc3lzL2FtZDY0L3ZtbS9hbWQv c3ZtX3NvZnRjLmgJMjAxNC0wNC0xMSAxMzo1MTo0NS4wMDAwMDAwMDAgKzAwMDAKKysrIC9tbnQv c3luY19uZXh0Mi9iaHl2ZV9zdm1fSEVBRF9yMjYzNzgwX1dJUC9zeXMvYW1kNjQvdm1tL2FtZC9z dm1fc29mdGMuaAkyMDE0LTA0LTA2IDA0OjM0OjE3LjAwMDAwMDAwMCArMDAwMApAQCAtNjEsNiAr NjEsNyBAQAogCSAqLwogCXVpbnQ4X3QJbXNyX2JpdG1hcFtTVk1fTVNSX0JJVE1BUF9TSVpFXTsK IAorCXVpbnQ4X3QgYXBpY19wYWdlW1ZNX01BWENQVV1bUEFHRV9TSVpFXTsKIAkvKiBOZXN0ZWQg UGFnaW5nICovCiAJdm1fb2Zmc2V0X3QgCW5wdHA7CQogCmRpZmYgLWF1clAgLXggJyoub3JpZycg LXggLnN2biAvbW50L3N5bmNfbmV4dDIvYmh5dmVfc3ZtX0hFQURfcjI2Mzc4MF9iYXNlLy9zeXMv YW1kNjQvdm1tL2FtZC92bWNiLmMgL21udC9zeW5jX25leHQyL2JoeXZlX3N2bV9IRUFEX3IyNjM3 ODBfV0lQL3N5cy9hbWQ2NC92bW0vYW1kL3ZtY2IuYwotLS0gL21udC9zeW5jX25leHQyL2JoeXZl X3N2bV9IRUFEX3IyNjM3ODBfYmFzZS8vc3lzL2FtZDY0L3ZtbS9hbWQvdm1jYi5jCTIwMTQtMDQt MTEgMTM6NTE6NDUuMDAwMDAwMDAwICswMDAwCisrKyAvbW50L3N5bmNfbmV4dDIvYmh5dmVfc3Zt X0hFQURfcjI2Mzc4MF9XSVAvc3lzL2FtZDY0L3ZtbS9hbWQvdm1jYi5jCTIwMTQtMDQtMDYgMDQ6 MzQ6MTcuMDAwMDAwMDAwICswMDAwCkBAIC0zNTQsMjUgKzM1NCwxMiBAQAogICogSW5qZWN0IGFu IGV2ZW50IHRvIHZjcHUgYXMgZGVzY3JpYmVkIGluIHNlY3Rpb24gMTUuMjAsICJFdmVudCBpbmpl Y3Rpb24iLgogICovCiBpbnQKLXZtY2JfZXZlbnRpbmplY3Qoc3RydWN0IHZtY2JfY3RybCAqY3Ry bCwgaW50IHR5cGUsIGludCB2ZWN0b3IsCit2bWNiX2V2ZW50aW5qZWN0KHN0cnVjdCB2bWNiX2N0 cmwgKmN0cmwsIGludCBpbnRyX3R5cGUsIGludCB2ZWN0b3IsCiAJCSB1aW50MzJfdCBlcnJvciwg Ym9vbCBlY192YWxpZCkKIHsKLQlpbnQgaW50cl90eXBlOwotCi0Jc3RhdGljIHVpbnQ4X3QgIHN2 bV9pbnRyX3R5cGVfbWFwW1ZNX0VWRU5UX01BWF0gPSB7Ci0JCS0xLAkJCQkvKiBWTV9FVkVOVF9O T05FICovCi0JCVZNQ0JfRVZFTlRJTkpfVFlQRV9JTlRSLAkvKiBWTV9IV19JTlRSICovCi0JCVZN Q0JfRVZFTlRJTkpfVFlQRV9OTUksCSAJLyogVk1fTk1JICovCi0JCVZNQ0JfRVZFTlRJTkpfVFlQ RV9FWENFUFRJT04sCS8qIFZNX0hXX0VYQ0VQVElPTiAqLwotCQlWTUNCX0VWRU5USU5KX1RZUEVf SU5UbiwgCS8qIFZNX1NXX0lOVFIsIElOVCAqLwotCQlWTUNCX0VWRU5USU5KX1RZUEVfSU5Ubiwg CS8qIFZNX1BSSVZfU1dfRVhDRVBUSU9OICovCi0JCVZNQ0JfRVZFTlRJTkpfVFlQRV9JTlRuLCAJ LyogVk1fU1dfRVhDRVBUSU9OICovCi0JfTsKLQotCWludHJfdHlwZSA9IHN2bV9pbnRyX3R5cGVf bWFwW3R5cGVdOwogCWlmIChpbnRyX3R5cGUgPCBWTUNCX0VWRU5USU5KX1RZUEVfSU5UUiB8fAog CSAgICBpbnRyX3R5cGUgPiBWTUNCX0VWRU5USU5KX1RZUEVfSU5UbikgewotCQlFUlIoIkV2ZW50 OiVkIGlzIG5vdCBzdXBwb3J0ZWQgYnkgU1ZNLlxuIiwgdHlwZSk7CisJCUVSUigiRXZlbnQ6JWQg aXMgbm90IHN1cHBvcnRlZCBieSBTVk0uXG4iLCBpbnRyX3R5cGUpOwogCQlyZXR1cm4gKEVJTlZB TCk7CiAJfQogCg== --047d7b86cdc2c5d01904f97255e5-- From owner-freebsd-virtualization@FreeBSD.ORG Fri May 16 21:21:56 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D18E34A for ; Fri, 16 May 2014 21:21:56 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id 844F52A79 for ; Fri, 16 May 2014 21:21:54 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id 11B09141483D for ; Fri, 16 May 2014 23:17:45 +0200 (CEST) Received: from navigator.renzel.net (unknown [10.0.0.254]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTPS id 5BF111A067D for ; Fri, 16 May 2014 23:17:43 +0200 (CEST) Received: from navigator.renzel.net (localhost [127.0.0.1]) by navigator.renzel.net (8.14.7/8.14.5) with ESMTP id s4GLHhbv033546 for ; Fri, 16 May 2014 23:17:43 +0200 (CEST) (envelope-from nbe@renzel.net) Received: (from www@localhost) by navigator.renzel.net (8.14.7/8.14.5/Submit) id s4GLHhQC033545; Fri, 16 May 2014 23:17:43 +0200 (CEST) (envelope-from nbe@renzel.net) To: freebsd-virtualization@freebsd.org Subject: Re: bhyve: svm (amd-v) update X-PHP-Originating-Script: 80:rcube.php Received: from so84mBimvz8j4468VZ4F2PmXs7wyCaU38wY3NLK0/3nr/k2T8ZGqww== (ZbnHhM+HjKp9Ri5ZS78wWj0D87ZVdFpY) by webmail.renzel.net with HTTP (HTTP/1.1 POST); Fri, 16 May 2014 23:17:43 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset= Content-Transfer-Encoding: 7bit Date: Fri, 16 May 2014 23:17:43 +0200 From: Nils Beyer Organization: VKF Renzel GmbH Message-ID: <045ce77ed17da4bd515bcc3cafe9c7f8@webmail.renzel.net.local> X-Sender: nbe@renzel.net User-Agent: Roundcube Webmail/0.8.4 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-7.0 required=7.0 tests=BAYES_00,UNPARSEABLE_RELAY autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 21:21:56 -0000 Hi Anish, Anish wrote: > If patches looks good to you, we can submit it. I have been testing it on > Phenom box which lacks some of newer SVM features. Your patch applied cleanly to the working copy of the "bhyve_svm"-project. I was then able to merge with HEAD (using "theirs-full" on one file) and compile the kernel. So, to me it looks OK to commit. Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom 1055T. It produces 200% load on the host CPU, and the emulated machine generates endlessly: ======================================================================================= BUG: soft lockup - CPU#0 stuck for 67s! [swapper:1] Modules linked in: CPU 0 Modules linked in: Pid: 1, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1 BHYVE RIP: 0010:[] [] rc_is_bit_0+0x3a/0x69 RSP: 0018:ffff88013e79dca0 EFLAGS: 00000a96 RAX: 000000000000009c RBX: ffff88013e79dcc0 RCX: ffff880004bdcc7c RDX: 00000000002f9dee RSI: ffffc9000402c538 RDI: ffff88013e79ddb0 RBP: ffffffff8100b9ce R08: ffffc9000402c788 R09: ffffffff81de32b8 R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000003 R13: ffffffff81157602 R14: ffff88013e79dc20 R15: 00000000000000d2 FS: 0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001a85000 CR4: 00000000000007b0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 1, threadinfo ffff88013e79c000, task ffff88013e79b500) Stack: ffffc9000402c644 00000000000000e5 00000000000003b1 ffff88013e79ddb0 ffff88013e79dcf0 ffffffff81c549b7 ffffffffffffffb6 0000000000000000 0000000000000000 ffffc9000402c000 ffff88013e79de30 ffffffff81c554e2 Call Trace: [] ? rc_get_bit+0x1b/0x79 [] ? unlzma+0xa42/0xc67 [] ? flush_buffer+0x0/0xa3 [] ? do_utimes+0xdb/0x170 [] ? nofill+0x0/0x10 [] ? unpack_to_rootfs+0x167/0x27a [] ? error+0x0/0x17 [] ? pci_get_subsys+0x35/0x40 [] ? populate_rootfs+0x0/0xd3 [] ? populate_rootfs+0x5b/0xd3 [] ? do_one_initcall+0x3c/0x1d0 [] ? kernel_init+0x29b/0x2f7 [] ? child_rip+0xa/0x20 [] ? kernel_init+0x0/0x2f7 [] ? child_rip+0x0/0x20 Code: ff ff ff 00 77 35 48 8b 47 18 48 39 47 08 72 0d 48 89 75 e8 e8 95 ff ff ff 48 8b 75 e8 48 8b 4b 08 c1 63 28 08 8b 53 24 0f b6 01 <48> 83 c1 01 c1 e2 08 48 89 4b 08 09 d0 89 43 24 0f b7 06 8b 53 Call Trace: [] ? rc_get_bit+0x1b/0x79 [] ? unlzma+0xa42/0xc67 [] ? flush_buffer+0x0/0xa3 [] ? do_utimes+0xdb/0x170 [] ? nofill+0x0/0x10 [] ? unpack_to_rootfs+0x167/0x27a [] ? error+0x0/0x17 [] ? pci_get_subsys+0x35/0x40 [] ? populate_rootfs+0x0/0xd3 [] ? populate_rootfs+0x5b/0xd3 [] ? do_one_initcall+0x3c/0x1d0 [] ? kernel_init+0x29b/0x2f7 [] ? child_rip+0xa/0x20 [] ? kernel_init+0x0/0x2f7 [] ? child_rip+0x0/0x20 ======================================================================================= Additionally, It produces a lot of MSR requests: ======================================================================================= May 16 09:32:03 10.255.255.96 kernel: emulate_rdmsr 0xc0010015 May 16 09:32:18 10.255.255.96 kernel: emulate_rdmsr 0x1b May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010112 May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 May 16 09:32:23 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0x8b May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010140 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010000 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010001 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010002 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010003 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:25 10.255.255.96 last message repeated 314 times May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:25 10.255.255.96 last message repeated 49 times May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:25 10.255.255.96 last message repeated 885 times May 16 09:32:25 10.255.255.96 kernel: 010055 May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:25 10.255.255.96 last message repeated 4820 times May 16 09:32:26 10.255.255.96 kernel: 010055 May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:26 10.255.255.96 last message repeated 4364 times May 16 09:32:26 10.255.255.96 kernel: 010055 May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:26 10.255.255.96 last message repeated 25 times May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:26 10.255.255.96 last message repeated 391 times May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:34 10.255.255.96 last message repeated 73074 times May 16 09:32:34 10.255.255.96 kernel: 010055 May 16 09:32:34 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:35 10.255.255.96 last message repeated 14648 times May 16 09:32:35 10.255.255.96 kernel: 010055 May 16 09:32:35 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:35 10.255.255.96 last message repeated 8098 times May 16 09:32:36 10.255.255.96 kernel: 010055 May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:36 10.255.255.96 last message repeated 7895 times May 16 09:32:36 10.255.255.96 kernel: 010055 May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:36 10.255.255.96 last message repeated 8272 times May 16 09:32:36 10.255.255.96 kernel: 010055 May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:37 10.255.255.96 last message repeated 8696 times May 16 09:32:37 10.255.255.96 kernel: 010055 May 16 09:32:37 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:37 10.255.255.96 last message repeated 12333 times May 16 09:32:38 10.255.255.96 kernel: 010055 May 16 09:32:38 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:41 10.255.255.96 last message repeated 30370 times May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc001103a May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:49 10.255.255.96 last message repeated 85577 times May 16 09:32:49 10.255.255.96 kernel: 010055 May 16 09:32:49 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:56 10.255.255.96 last message repeated 9534 times May 16 09:32:57 10.255.255.96 kernel: 010055 May 16 09:32:57 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 May 16 09:32:58 10.255.255.96 last message repeated 17524 times [and so on] ======================================================================================= I'd love to see CentOS perfectly running on my Phenom as it runs perfectly on an Intel i3. If you need any further information/debug, please let me know... TIA and regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Sun May 18 14:44:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B2C338F for ; Sun, 18 May 2014 14:44:32 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACBBA2FC7 for ; Sun, 18 May 2014 14:44:31 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id l18so6713423wgh.23 for ; Sun, 18 May 2014 07:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Sm5Ofj8N2Tc0l33Xg+LzjNGy22KAZR04VJk0h8k/n7M=; b=VQ/arsrieqTUUXl8PEVgPTm5W9JMnoFJWUiWc90YKd9GRRDVDRsI0UGv5ezxUbi6J0 C9/Hx7RCLBLAglwhsDmeQSc7yvOFxI8k/cunXfF+6akH/niwThx9Yb4iY8aOVtOOpjTd btDWSgdD0XnizVhmjYX2cvZEQHMe4jQSaF9k/D1Tq2P+GHBYTvCno/nUP6j4ek03BKK5 +8R6DDEQC+Me0B0DXmTcxr7miVI+m8uSentgGOjWiIWsmLPqXPbPwWoi9ra4qVENtXHF dARNk5W4poU/iBHuyC7qrLhYK4SUsz6kuNrKDm64QAVExSuItRd9jT1lXlh1KcqG0LKt IDYg== MIME-Version: 1.0 X-Received: by 10.180.82.7 with SMTP id e7mr8076641wiy.6.1400424269886; Sun, 18 May 2014 07:44:29 -0700 (PDT) Received: by 10.194.135.49 with HTTP; Sun, 18 May 2014 07:44:29 -0700 (PDT) In-Reply-To: <045ce77ed17da4bd515bcc3cafe9c7f8@webmail.renzel.net.local> References: <045ce77ed17da4bd515bcc3cafe9c7f8@webmail.renzel.net.local> Date: Sun, 18 May 2014 07:44:29 -0700 Message-ID: Subject: Re: bhyve: svm (amd-v) update From: Anish To: Nils Beyer Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 14:44:32 -0000 Thanks for testing it. >Your patch applied cleanly to the working copy of the "bhyve_svm"-project. I was then able to merge with HEAD (using "theirs-full" on one file) and compile the kernel. So, to me it looks OK to commit. Yes, that's correct. You have to retain changes in sys/amd64/vmm/amd/amdv.c from bhyve_svm branch. >Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom 1055T. It produces 200% load on the host CPU, and the emulated machine generates endlessly: Its 200% load because of 2 vcpus to guest. It stuck in loop even with single processor(1 vcpu) after PCI probing[debug messages with linux .....earlyprintk=serial debug] [ 3.684243] UDP hash table entries: 1024 (order: 3, 32768 bytes) [ 3.686484] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) [ 3.691987] NET: Registered protocol family 1 [ 3.693382] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 3.695214] PCI: CLS 64 bytes, default 64 [ 3.698176] Trying to unpack rootfs image as initramfs... [ 30.595279] BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:1] [ 3.505631] pnp: PnP ACPI: found 5 devices [ 3.506417] ACPI: bus type PNP unregistered [ 3.635781] pci 0000:00:06.0: no compatible bridge window for [mem 0xfe440000 -0xfe45ffff pref] [ 3.637555] pci 0000:00:06.0: BAR 6: assigned [mem 0x80000000-0x8001ffff pref ] [ 3.638986] pci 0000:00:01.0: BAR 6: assigned [mem 0x80020000-0x800207ff pref ] [ 3.640416] pci 0000:00:04.0: BAR 6: assigned [mem 0x80020800-0x80020fff pref ] [ 3.641864] pci 0000:00:05.0: BAR 6: assigned [mem 0x80021000-0x800217ff pref ] [ 3.643259] pci 0000:00:00.0: not setting up bridge for bus 0000:01 [ 3.644550] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7] [ 3.645670] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff] [ 3.646795] pci_bus 0000:00: resource 6 [mem 0x80000000-0xdfffffff] [ 3.648031] pci_bus 0000:00: resource 7 [mem 0xd000000000-0xfcffffffff] [ 3.650970] NET: Registered protocol family 2 [ 3.661491] TCP established hash table entries: 16384 (order: 6, 262144 bytes ) [ 3.671854] TCP bind hash table entries: 16384 (order: 6, 262144 bytes) [ 3.681116] TCP: Hash tables configured (established 16384 bind 16384) [ 3.683335] TCP: reno registered [ 3.684243] UDP hash table entries: 1024 (order: 3, 32768 bytes) [ 3.686484] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) [ 3.691987] NET: Registered protocol family 1 [ 3.693382] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 3.695214] PCI: CLS 64 bytes, default 64 [ 3.698176] Trying to unpack rootfs image as initramfs... [ 30.595279] BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:1] [ 30.596366] Modules linked in: >Additionally, It produces a lot of MSR requests: Yes, on AMD Linux is touching more MSRs( AMD specific -address 0xC00XXXX) compared to Intel. Thanks and regards, Anish On Fri, May 16, 2014 at 2:17 PM, Nils Beyer wrote: > Hi Anish, > > Anish wrote: > > If patches looks good to you, we can submit it. I have been testing it on > > Phenom box which lacks some of newer SVM features. > > Your patch applied cleanly to the working copy of the "bhyve_svm"-project. > I was then able to merge with HEAD > (using "theirs-full" on one file) and compile the kernel. So, to me it > looks OK to commit. > > Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom > 1055T. It produces 200% load on the > host CPU, and the emulated machine generates endlessly: > > ======================================================================================= > BUG: soft lockup - CPU#0 stuck for 67s! [swapper:1] > Modules linked in: > CPU 0 > Modules linked in: > > Pid: 1, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1 BHYVE > RIP: 0010:[] [] rc_is_bit_0+0x3a/0x69 > RSP: 0018:ffff88013e79dca0 EFLAGS: 00000a96 > RAX: 000000000000009c RBX: ffff88013e79dcc0 RCX: ffff880004bdcc7c > RDX: 00000000002f9dee RSI: ffffc9000402c538 RDI: ffff88013e79ddb0 > RBP: ffffffff8100b9ce R08: ffffc9000402c788 R09: ffffffff81de32b8 > R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000003 > R13: ffffffff81157602 R14: ffff88013e79dc20 R15: 00000000000000d2 > FS: 0000000000000000(0000) GS:ffff880028200000(0000) > knlGS:0000000000000000 > CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b > CR2: 0000000000000000 CR3: 0000000001a85000 CR4: 00000000000007b0 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > Process swapper (pid: 1, threadinfo ffff88013e79c000, task > ffff88013e79b500) > Stack: > ffffc9000402c644 00000000000000e5 00000000000003b1 ffff88013e79ddb0 > ffff88013e79dcf0 ffffffff81c549b7 ffffffffffffffb6 0000000000000000 > 0000000000000000 ffffc9000402c000 ffff88013e79de30 ffffffff81c554e2 > Call Trace: > [] ? rc_get_bit+0x1b/0x79 > [] ? unlzma+0xa42/0xc67 > [] ? flush_buffer+0x0/0xa3 > [] ? do_utimes+0xdb/0x170 > [] ? nofill+0x0/0x10 > [] ? unpack_to_rootfs+0x167/0x27a > [] ? error+0x0/0x17 > [] ? pci_get_subsys+0x35/0x40 > [] ? populate_rootfs+0x0/0xd3 > [] ? populate_rootfs+0x5b/0xd3 > [] ? do_one_initcall+0x3c/0x1d0 > [] ? kernel_init+0x29b/0x2f7 > [] ? child_rip+0xa/0x20 > [] ? kernel_init+0x0/0x2f7 > [] ? child_rip+0x0/0x20 > Code: ff ff ff 00 77 35 48 8b 47 18 48 39 47 08 72 0d 48 89 75 e8 e8 95 ff > ff ff 48 8b 75 e8 48 8b 4b 08 c1 63 28 08 8b 53 24 0f b6 01 <48> 83 c1 01 > c1 e2 08 48 89 4b 08 09 d0 89 43 24 0f b7 06 8b 53 > Call Trace: > [] ? rc_get_bit+0x1b/0x79 > [] ? unlzma+0xa42/0xc67 > [] ? flush_buffer+0x0/0xa3 > [] ? do_utimes+0xdb/0x170 > [] ? nofill+0x0/0x10 > [] ? unpack_to_rootfs+0x167/0x27a > [] ? error+0x0/0x17 > [] ? pci_get_subsys+0x35/0x40 > [] ? populate_rootfs+0x0/0xd3 > [] ? populate_rootfs+0x5b/0xd3 > [] ? do_one_initcall+0x3c/0x1d0 > [] ? kernel_init+0x29b/0x2f7 > [] ? child_rip+0xa/0x20 > [] ? kernel_init+0x0/0x2f7 > [] ? child_rip+0x0/0x20 > > ======================================================================================= > > > Additionally, It produces a lot of MSR requests: > > ======================================================================================= > May 16 09:32:03 10.255.255.96 kernel: emulate_rdmsr 0xc0010015 > May 16 09:32:18 10.255.255.96 kernel: emulate_rdmsr 0x1b > May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010112 > May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 > May 16 09:32:23 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 > May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0x8b > May 16 09:32:23 10.255.255.96 kernel: emulate_rdmsr 0xc0010140 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010000 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010001 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010002 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010003 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:25 10.255.255.96 last message repeated 314 times > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x1b > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010048 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010048 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0x8b > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:25 10.255.255.96 last message repeated 49 times > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010004 > May 16 09:32:25 10.255.255.96 kernel: emulate_wrmsr 0xc0010000 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:25 10.255.255.96 last message repeated 885 times > May 16 09:32:25 10.255.255.96 kernel: 010055 > May 16 09:32:25 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:25 10.255.255.96 last message repeated 4820 times > May 16 09:32:26 10.255.255.96 kernel: 010055 > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:26 10.255.255.96 last message repeated 4364 times > May 16 09:32:26 10.255.255.96 kernel: 010055 > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:26 10.255.255.96 last message repeated 25 times > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:26 10.255.255.96 last message repeated 391 times > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_wrmsr 0xc001001f > May 16 09:32:26 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:34 10.255.255.96 last message repeated 73074 times > May 16 09:32:34 10.255.255.96 kernel: 010055 > May 16 09:32:34 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:35 10.255.255.96 last message repeated 14648 times > May 16 09:32:35 10.255.255.96 kernel: 010055 > May 16 09:32:35 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:35 10.255.255.96 last message repeated 8098 times > May 16 09:32:36 10.255.255.96 kernel: 010055 > May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:36 10.255.255.96 last message repeated 7895 times > May 16 09:32:36 10.255.255.96 kernel: 010055 > May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:36 10.255.255.96 last message repeated 8272 times > May 16 09:32:36 10.255.255.96 kernel: 010055 > May 16 09:32:36 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:37 10.255.255.96 last message repeated 8696 times > May 16 09:32:37 10.255.255.96 kernel: 010055 > May 16 09:32:37 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:37 10.255.255.96 last message repeated 12333 times > May 16 09:32:38 10.255.255.96 kernel: 010055 > May 16 09:32:38 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:41 10.255.255.96 last message repeated 30370 times > May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc001103a > May 16 09:32:41 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:49 10.255.255.96 last message repeated 85577 times > May 16 09:32:49 10.255.255.96 kernel: 010055 > May 16 09:32:49 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:56 10.255.255.96 last message repeated 9534 times > May 16 09:32:57 10.255.255.96 kernel: 010055 > May 16 09:32:57 10.255.255.96 kernel: emulate_rdmsr 0xc0010055 > May 16 09:32:58 10.255.255.96 last message repeated 17524 times > [and so on] > > ======================================================================================= > > I'd love to see CentOS perfectly running on my Phenom as it runs perfectly > on an Intel i3. > > If you need any further information/debug, please let me know... > > > > TIA and regards, > Nils > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Mon May 19 04:15:22 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6F39901 for ; Mon, 19 May 2014 04:15:22 +0000 (UTC) Received: from mail-ve0-x233.google.com (mail-ve0-x233.google.com [IPv6:2607:f8b0:400c:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77C932BCA for ; Mon, 19 May 2014 04:15:22 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id oy12so5741588veb.24 for ; Sun, 18 May 2014 21:15:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=T6NyVnVOtkjKmMffdNndHh/cc3DrGDXOl373DUElEFY=; b=TzCS3sl3lcJu2M0oafDutY6uxzu2kxJbvDZRlilxHMQp16t8UOY9bKjd1RzPlReGvW v1HvIhhYq1qmrgd5FHW0NKcgEcTW0LgTBkAvT2+OqpmdtY+ITQeSD3si1thRge9UNlE3 5eSi/RIBqDWaOkVrvbjgHYfjxNDDFbSFt4D9+wMJPLWGYvZV7gok8H3kF6tuUKyc8zaw 2hW1Vhr+8YL4wSQHyIS52sSL07od/IrLb3VHQlLxzy4yqOIgSu6hJte48NJAzcByqOQv VdeUUIeLJNlQima67DobDKaGikj3sN7AxaGVqHfu1ZS/aXrGk0wL5VUMvtCCTjHiB1ED yvug== MIME-Version: 1.0 X-Received: by 10.58.134.101 with SMTP id pj5mr4271752veb.38.1400472921646; Sun, 18 May 2014 21:15:21 -0700 (PDT) Sender: jpm820@gmail.com Received: by 10.220.104.135 with HTTP; Sun, 18 May 2014 21:15:21 -0700 (PDT) Date: Sun, 18 May 2014 23:15:21 -0500 X-Google-Sender-Auth: bZCwT14bgOkZZCl7dbNqupjQ-YY Message-ID: Subject: clock wrong in bhyve vm's From: Joe Maloney To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 04:15:22 -0000 So far I've created 2 VM's in bhyve with 11 CURRENT updated world as of today. One VM is running FreeBSD 10. The other is running FreeNAS 9.2.1.6 Beta. It seems every VM I create has the time wrong about 5 or 6 hours. Using ntpdate will correct the time until the VM reboots. Has anyone else noticed this or know of a possible fix other than using ntpd? Joe Maloney From owner-freebsd-virtualization@FreeBSD.ORG Mon May 19 04:19:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 020CCA38 for ; Mon, 19 May 2014 04:19:29 +0000 (UTC) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B741B2BE2 for ; Mon, 19 May 2014 04:19:28 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id sa20so5957092veb.9 for ; Sun, 18 May 2014 21:19:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=pd6u/s6eo6C396GhpsHXLrMbsY9P0CKcD9JFnl0trjs=; b=EgVDtb4qkxXnixZDW8d86PE4+cg+m8sdYp5xfHmvYowKSbBd38f4f6bi9PhIOzjtKg WRS3OWXzVffo5SNs2Rm4zqY65INXBKc/R6Rv0YnGgEmXI+pHLPUSE94mks3pSPw2nmnj Vi0Y3uLiwAsClbbxilU+YIuLK3LT1TLMyFooQsbD5iGE1n3IOIgA/fhi73vbaIiTSXRp /JhC+buj6XVf337IIQGMUPiqXcwU9Dg2xT35/kBm4VuRGiAy2qbcM5ZcmIusGyCW89GB niP/oPmBUVMacNprGEE0Kk9aXxxJPhRKHi4fZo47rLpxpsbZLTXVCGukGN6fj7dWQdm4 2wlw== MIME-Version: 1.0 X-Received: by 10.58.134.101 with SMTP id pj5mr4288895veb.38.1400473167855; Sun, 18 May 2014 21:19:27 -0700 (PDT) Sender: jpm820@gmail.com Received: by 10.220.104.135 with HTTP; Sun, 18 May 2014 21:19:27 -0700 (PDT) Date: Sun, 18 May 2014 23:19:27 -0500 X-Google-Sender-Auth: rmZdVdYvslsODThoJ53EME4BbV0 Message-ID: Subject: bhyve + freenas + vimage jails From: Joe Maloney To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 04:19:29 -0000 I've been noticing that when VIMAGE jails are created in FreeNAS within a bhyve VM the system VM kernel panics quite a bit. I'm running 11 Current updated as of today. I've tried with options VIMAGE turned on and with a GENERIC kernel and see the same result either way. Sometimes it works, sometimes it doesn't. Not really a huge deal as I've just been testing for fun but if I will provide logs if anyone would like to see. I would just need to know how to pull them. Joe Maloney From owner-freebsd-virtualization@FreeBSD.ORG Mon May 19 11:06:55 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42CC15F8 for ; Mon, 19 May 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2FBB82DD4 for ; Mon, 19 May 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4JB6t1Q080213 for ; Mon, 19 May 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4JB6sRt080211 for freebsd-virtualization@FreeBSD.org; Mon, 19 May 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 19 May 2014 11:06:54 GMT Message-Id: <201405191106.s4JB6sRt080211@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon May 19 13:56:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1C8FF18 for ; Mon, 19 May 2014 13:56:04 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C71A2EAA for ; Mon, 19 May 2014 13:56:04 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 764C2B94A; Mon, 19 May 2014 09:56:03 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: clock wrong in bhyve vm's Date: Mon, 19 May 2014 09:52:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405190952.16343.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 19 May 2014 09:56:03 -0400 (EDT) Cc: Joe Maloney X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:56:04 -0000 On Monday, May 19, 2014 12:15:21 am Joe Maloney wrote: > So far I've created 2 VM's in bhyve with 11 CURRENT updated world as of > today. One VM is running FreeBSD 10. The other is running FreeNAS 9.2.1.6 > Beta. It seems every VM I create has the time wrong about 5 or 6 hours. > Using ntpdate will correct the time until the VM reboots. Has anyone else > noticed this or know of a possible fix other than using ntpd? Is that your offset from UTC? Do you have /etc/localtime set correctly in your guests? -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Mon May 19 13:56:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7157EF19 for ; Mon, 19 May 2014 13:56:05 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B30D2EAB for ; Mon, 19 May 2014 13:56:05 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 423F1B986; Mon, 19 May 2014 09:56:04 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: bhyve + freenas + vimage jails Date: Mon, 19 May 2014 09:53:02 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405190953.02807.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 19 May 2014 09:56:04 -0400 (EDT) Cc: Joe Maloney X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 13:56:05 -0000 On Monday, May 19, 2014 12:19:27 am Joe Maloney wrote: > I've been noticing that when VIMAGE jails are created in FreeNAS within a > bhyve VM the system VM kernel panics quite a bit. > > I'm running 11 Current updated as of today. I've tried with options VIMAGE > turned on and with a GENERIC kernel and see the same result either way. > Sometimes it works, sometimes it doesn't. Not really a huge deal as I've > just been testing for fun but if I will provide logs if anyone would like > to see. I would just need to know how to pull them. You can start with the /var/crash/core.txt.N file which includes the kernel messages and kgdb stacktrace (assuming you are getting crashdumps) -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Tue May 20 02:46:57 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7CD0050C; Tue, 20 May 2014 02:46:57 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 291522193; Tue, 20 May 2014 02:46:57 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id hy4so10554244vcb.39 for ; Mon, 19 May 2014 19:46:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9OuluYMOYr2Kjz0DqqSbPuN7/OEnB/lFmzfkNOFDceU=; b=F+bFZn4V3kYEjb0FkwjtDD8xj+tIhiUys9n7PJhA0Kt2xoSjEch1Mt1tDM2m/TEC9T zByHnYAnd3Z87Ea+z1Uw/Wp3XOZj+gyRATKyJmMMj72/hIc1eTJ9tCnt4s3IvlYxPlXc DGcd+mBpeekl5TX7CNvxHj4nDoIfJCbf+El4Ably7HT1z61QpT186HHjh3fRrgbt9E93 /a4yYdaY/RNtpobbR9JJrDM65RWW2duK7X9LotOMG4pfWI9f3zV6ajtBNaO2BwNuEy+r hF5YmdGcb4ewdnmjYaZ68t0tHn8P77lyPL6nfKtm7SoSCzd+pN670Cfomfq+6napTdVG bVrQ== MIME-Version: 1.0 X-Received: by 10.52.95.135 with SMTP id dk7mr1143426vdb.38.1400554015910; Mon, 19 May 2014 19:46:55 -0700 (PDT) Sender: jpm820@gmail.com Received: by 10.220.104.135 with HTTP; Mon, 19 May 2014 19:46:55 -0700 (PDT) In-Reply-To: <201405190952.16343.jhb@freebsd.org> References: <201405190952.16343.jhb@freebsd.org> Date: Mon, 19 May 2014 21:46:55 -0500 X-Google-Sender-Auth: bq5zcZCr3gjROoPPB_nPEV-0E1g Message-ID: Subject: Re: clock wrong in bhyve vm's From: Joe Maloney To: John Baldwin Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 02:46:57 -0000 It was original off by my offset. Not sure what happened but now it's working without using ntpd to update time manually. I did confirm that /etc/localtime matched on host and VM. Perhaps the last kernel world rebuild sunday fixed and I just didn't reboot enough times. I'll try to dig up more info if it happens again. Joe Maloney On Mon, May 19, 2014 at 8:52 AM, John Baldwin wrote: > On Monday, May 19, 2014 12:15:21 am Joe Maloney wrote: > > So far I've created 2 VM's in bhyve with 11 CURRENT updated world as of > > today. One VM is running FreeBSD 10. The other is running FreeNAS > 9.2.1.6 > > Beta. It seems every VM I create has the time wrong about 5 or 6 hours. > > Using ntpdate will correct the time until the VM reboots. Has anyone > else > > noticed this or know of a possible fix other than using ntpd? > > Is that your offset from UTC? Do you have /etc/localtime set correctly in > your guests? > > -- > John Baldwin > From owner-freebsd-virtualization@FreeBSD.ORG Tue May 20 03:05:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 233C0D87; Tue, 20 May 2014 03:05:18 +0000 (UTC) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C340A238F; Tue, 20 May 2014 03:05:17 +0000 (UTC) Received: by mail-ve0-f173.google.com with SMTP id pa12so7436890veb.4 for ; Mon, 19 May 2014 20:05:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=q7LDnYLgZ7wHM8OgQOy7shZIvbBFjmAHLEqDFeogGKU=; b=OIgz4fVgiV6GZU5Wa/oXNtLl6pGvdqKwwP/9gF7CD5Nd7+MI/NJPwK2f2Sjy/e8Kug mni2YgRg4RIAKuoDGKbj4YBNNioFzJK3ISqjRA60wRFdOkP/GuiSrm1+ZwrX4JDYWkjv 4bGaO6RHqqzUz4ZQpG5DnXt5S15bYQ8w3UIuv7Z8c7x44JChJ4Ej2Y+/L8eQhc8fR8mn GSoYtfxVganLW8oY06SwLTyg7DFoL8UI2G6z7vbQvh7ZzrRXoa6aONp90Lw5ZJhOoRNn 4n2Sq0aUewb3yNYega1Wx/VQ47uKS+08R6Cm8WutHJyUsvvuQCi9PMbZmph9oNhVCoRP 11Aw== MIME-Version: 1.0 X-Received: by 10.220.95.204 with SMTP id e12mr1352884vcn.37.1400555116886; Mon, 19 May 2014 20:05:16 -0700 (PDT) Sender: jpm820@gmail.com Received: by 10.220.104.135 with HTTP; Mon, 19 May 2014 20:05:16 -0700 (PDT) In-Reply-To: <201405190953.02807.jhb@freebsd.org> References: <201405190953.02807.jhb@freebsd.org> Date: Mon, 19 May 2014 22:05:16 -0500 X-Google-Sender-Auth: 2TI7hnAFZOZbph1jXf0UFNZ6nxE Message-ID: Subject: Re: bhyve + freenas + vimage jails From: Joe Maloney To: John Baldwin Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 03:05:18 -0000 Nothing for me in /var/crash. I was able to grab this by running vm attach before it crashed through ssh. I'm going to link to pastebin for this one because it is horribly long. This crash was created as a result of stopping a vimage jail in FreeNAS running in a bhyve vm in order to attempt to delete it. http://pastebin.com/haiMrQ9M Joe Maloney On Mon, May 19, 2014 at 8:53 AM, John Baldwin wrote: > On Monday, May 19, 2014 12:19:27 am Joe Maloney wrote: > > I've been noticing that when VIMAGE jails are created in FreeNAS within a > > bhyve VM the system VM kernel panics quite a bit. > > > > I'm running 11 Current updated as of today. I've tried with options > VIMAGE > > turned on and with a GENERIC kernel and see the same result either way. > > Sometimes it works, sometimes it doesn't. Not really a huge deal as > I've > > just been testing for fun but if I will provide logs if anyone would like > > to see. I would just need to know how to pull them. > > You can start with the /var/crash/core.txt.N file which includes the kernel > messages and kgdb stacktrace (assuming you are getting crashdumps) > > -- > John Baldwin > From owner-freebsd-virtualization@FreeBSD.ORG Tue May 20 20:11:53 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCD9ED74 for ; Tue, 20 May 2014 20:11:53 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 675042D20 for ; Tue, 20 May 2014 20:11:53 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 83BFE153434; Tue, 20 May 2014 22:11:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vxbnra8F09ga; Tue, 20 May 2014 22:11:43 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:65d0:df9:94a1:3ef5] (unknown [IPv6:2001:4cb8:3:1:65d0:df9:94a1:3ef5]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 71E88153AC2; Tue, 20 May 2014 22:11:43 +0200 (CEST) Message-ID: <537BB6FF.5080909@digiware.nl> Date: Tue, 20 May 2014 22:11:43 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish , Nils Beyer Subject: Re: bhyve: svm (amd-v) update References: <045ce77ed17da4bd515bcc3cafe9c7f8@webmail.renzel.net.local> In-Reply-To: Content-Type: multipart/mixed; boundary="------------030600060806040702030601" Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 20:11:53 -0000 This is a multi-part message in MIME format. --------------030600060806040702030601 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 18-5-2014 16:44, Anish wrote: > Thanks for testing it. >> Your patch applied cleanly to the working copy of the "bhyve_svm"-project. > I was then able to merge with HEAD > (using "theirs-full" on one file) and compile the kernel. So, to me it > looks OK to commit. > Yes, that's correct. You have to retain changes in sys/amd64/vmm/amd/amdv.c > from bhyve_svm branch. > >> Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom > 1055T. It produces 200% load on the > host CPU, and the emulated machine generates endlessly: > Its 200% load because of 2 vcpus to guest. It stuck in loop even with > single processor(1 vcpu) after PCI probing[debug messages with linux > .....earlyprintk=serial debug] > > [ 3.684243] UDP hash table entries: 1024 (order: 3, 32768 bytes) > > [ 3.686484] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) > > [ 3.691987] NET: Registered protocol family 1 > > [ 3.693382] pci 0000:00:01.0: Activating ISA DMA hang workarounds > > [ 3.695214] PCI: CLS 64 bytes, default 64 > > [ 3.698176] Trying to unpack rootfs image as initramfs... > > [ 30.595279] BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:1] > > [ 3.505631] pnp: PnP ACPI: found 5 devices > > [ 3.506417] ACPI: bus type PNP unregistered > > [ 3.635781] pci 0000:00:06.0: no compatible bridge window for [mem > 0xfe440000 > > -0xfe45ffff pref] > > [ 3.637555] pci 0000:00:06.0: BAR 6: assigned [mem 0x80000000-0x8001ffff > pref > > ] > > [ 3.638986] pci 0000:00:01.0: BAR 6: assigned [mem 0x80020000-0x800207ff > pref > > ] > > [ 3.640416] pci 0000:00:04.0: BAR 6: assigned [mem 0x80020800-0x80020fff > pref > > ] > > [ 3.641864] pci 0000:00:05.0: BAR 6: assigned [mem 0x80021000-0x800217ff > pref > > ] > > [ 3.643259] pci 0000:00:00.0: not setting up bridge for bus 0000:01 > > [ 3.644550] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7] > > [ 3.645670] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff] > > [ 3.646795] pci_bus 0000:00: resource 6 [mem 0x80000000-0xdfffffff] > > [ 3.648031] pci_bus 0000:00: resource 7 [mem 0xd000000000-0xfcffffffff] > > [ 3.650970] NET: Registered protocol family 2 > > [ 3.661491] TCP established hash table entries: 16384 (order: 6, 262144 > bytes > > ) > > [ 3.671854] TCP bind hash table entries: 16384 (order: 6, 262144 bytes) > > [ 3.681116] TCP: Hash tables configured (established 16384 bind 16384) > > [ 3.683335] TCP: reno registered > > [ 3.684243] UDP hash table entries: 1024 (order: 3, 32768 bytes) > > [ 3.686484] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) > > [ 3.691987] NET: Registered protocol family 1 > > [ 3.693382] pci 0000:00:01.0: Activating ISA DMA hang workarounds > > [ 3.695214] PCI: CLS 64 bytes, default 64 > > [ 3.698176] Trying to unpack rootfs image as initramfs... > > [ 30.595279] BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:1] > > [ 30.596366] Modules linked in: >> Additionally, It produces a lot of MSR requests: > Yes, on AMD Linux is touching more MSRs( AMD specific -address 0xC00XXXX) > compared to Intel. > > Thanks and regards, > Anish > > > On Fri, May 16, 2014 at 2:17 PM, Nils Beyer wrote: > >> Hi Anish, >> >> Anish wrote: >>> If patches looks good to you, we can submit it. I have been testing it on >>> Phenom box which lacks some of newer SVM features. >> >> Your patch applied cleanly to the working copy of the "bhyve_svm"-project. >> I was then able to merge with HEAD >> (using "theirs-full" on one file) and compile the kernel. So, to me it >> looks OK to commit. >> >> Unfortunately, I am still not able to boot CentOS 6.5 using my Phenom >> 1055T. It produces 200% load on the >> host CPU, and the emulated machine generates endlessly: >> >> ======================================================================================= >> BUG: soft lockup - CPU#0 stuck for 67s! [swapper:1] >> Modules linked in: >> CPU 0 >> Modules linked in: >> >> Pid: 1, comm: swapper Not tainted 2.6.32-431.el6.x86_64 #1 BHYVE And more... >> I'd love to see CentOS perfectly running on my Phenom as it runs perfectly >> on an Intel i3. >> >> If you need any further information/debug, please let me know... I've been trying to get Ubuntu, CentOS and like to run on AMDs, and currently I'm compiling a kernel, but it goes dirt slow. Attached a patch I have to debug more of the MSRs and it does what I do to get the TSC running.... It helps, but things are still like molases. For Ubuntu I also needed to fix part of the AHCI code since it bails out on ATA FLUSH. I'm going to take a look at the recently posted diff which should get bhyve_svm in line with head. And see if that speeds up my Ubuntu kernels. --WjW --------------030600060806040702030601 Content-Type: text/plain; charset=windows-1252; name="msr-tsc.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="msr-tsc.patch" Index: sys/amd64/vmm/amd/svm.c =================================================================== --- sys/amd64/vmm/amd/svm.c (revision 264582) +++ sys/amd64/vmm/amd/svm.c (working copy) @@ -82,6 +82,8 @@ static bool svm_vmexit(struct svm_softc *svm_sc, int vcpu, struct vm_exit *vmexit); static int svm_msr_rw_ok(uint8_t *btmap, uint64_t msr); +static int svm_msr_ro_ok(uint8_t *btmap, uint64_t msr); +static int svm_msr_rw_ro_ok(uint8_t *btmap, uint64_t msr, int mask); static int svm_msr_index(uint64_t msr, int *index, int *bit); static uint32_t svm_feature; /* AMD SVM features. */ @@ -315,9 +317,24 @@ /* * Give virtual cpu the complete access to MSR(read & write). */ +#define MSR_RO 1 +#define MSR_RW 3 + static int svm_msr_rw_ok(uint8_t *perm_bitmap, uint64_t msr) { + return svm_msr_rw_ro_ok(perm_bitmap, msr, MSR_RW); +} + +static int +svm_msr_ro_ok(uint8_t *perm_bitmap, uint64_t msr) +{ + return svm_msr_rw_ro_ok(perm_bitmap, msr, MSR_RO); +} + +static int +svm_msr_rw_ro_ok(uint8_t *perm_bitmap, uint64_t msr, int mask) +{ int index, bit, err; err = svm_msr_index(msr, &index, &bit); @@ -336,8 +353,12 @@ } /* Disable intercept for read and write. */ - perm_bitmap[index] &= ~(3 << bit); - CTR1(KTR_VMM, "Guest has full control on SVM:MSR(0x%lx).\n", msr); + perm_bitmap[index] &= ~(mask << bit); + if (mask==MSR_RW) { + CTR1(KTR_VMM, "Guest has Read/Write control on SVM:MSR(0x%lx).\n", msr ); + } else { + CTR1(KTR_VMM, "Guest has Read/Write control on SVM:MSR(0x%lx).\n", msr ); + } return (0); } @@ -415,10 +436,26 @@ svm_msr_rw_ok(svm_sc->msr_bitmap, MSR_SYSENTER_CS_MSR); svm_msr_rw_ok(svm_sc->msr_bitmap, MSR_SYSENTER_ESP_MSR); svm_msr_rw_ok(svm_sc->msr_bitmap, MSR_SYSENTER_EIP_MSR); - + +#define AMD_MSR_TSEG_BASE 0xc0010112 +#define AMD_MSR_OSVW_ID_LENGTH 0xc0010140 /* read */ +#define AMD_MSR_OSVW_STATUS 0xc0010141 /* read */ +#define AMD_MSR_MC4_CTL_MASK 0xc0010048 + /* For Nested Paging/RVI only. */ svm_msr_rw_ok(svm_sc->msr_bitmap, MSR_PAT); + svm_msr_rw_ok(svm_sc->msr_bitmap, AMD_MSR_OSVW_ID_LENGTH); + svm_msr_rw_ok(svm_sc->msr_bitmap, AMD_MSR_OSVW_STATUS); + /* + * MSRs that are allowed to be read. + * most obvious one is the TSC read which could be time critical + */ + svm_msr_ro_ok(svm_sc->msr_bitmap, MSR_TSC); + svm_msr_ro_ok(svm_sc->msr_bitmap, MSR_HWCR); + svm_msr_ro_ok(svm_sc->msr_bitmap, AMD_MSR_TSEG_BASE); + svm_msr_ro_ok(svm_sc->msr_bitmap, AMD_MSR_MC4_CTL_MASK); + /* Intercept access to all I/O ports. */ memset(svm_sc->iopm_bitmap, 0xFF, sizeof(svm_sc->iopm_bitmap)); @@ -566,6 +603,13 @@ svm_efer(svm_sc, vcpu, info1); break; } + if (ecx == MSR_TSC) { + uint64_t tscval = rdtsc(); + VCPU_CTR0(svm_sc->vm, vcpu,"VMEXIT TSC MSR\n"); + state->rax = tscval & 0xffffffff; + ctx->e.g.sctx_rdx = tscval >> 32; + break; + } retu = false; if (info1) { Index: sys/amd64/vmm/intel/vmx.c =================================================================== --- sys/amd64/vmm/intel/vmx.c (revision 264582) +++ sys/amd64/vmm/intel/vmx.c (working copy) @@ -109,6 +109,9 @@ #define guest_msr_rw(vmx, msr) \ msr_bitmap_change_access((vmx)->msr_bitmap, (msr), MSR_BITMAP_ACCESS_RW) +#define guest_msr_ro(vmx, msr) \ + msr_bitmap_change_access((vmx)->msr_bitmap, (msr), MSR_BITMAP_ACCESS_READ) + #define HANDLED 1 #define UNHANDLED 0 @@ -786,6 +789,11 @@ * MSR_EFER is saved and restored in the guest VMCS area on a * VM exit and entry respectively. It is also restored from the * host VMCS area on a VM exit. + * + * The TSC MSR is exposed read-only. Writes are disallowed as that + * will impact the host TSC. + * XXX Writes would be implemented with a wrmsr trap, and + * then modifying the TSC offset in the VMCS. */ if (guest_msr_rw(vmx, MSR_GSBASE) || guest_msr_rw(vmx, MSR_FSBASE) || @@ -793,7 +801,8 @@ guest_msr_rw(vmx, MSR_SYSENTER_ESP_MSR) || guest_msr_rw(vmx, MSR_SYSENTER_EIP_MSR) || guest_msr_rw(vmx, MSR_KGSBASE) || - guest_msr_rw(vmx, MSR_EFER)) + guest_msr_rw(vmx, MSR_EFER) || + guest_msr_ro(vmx, MSR_TSC)) panic("vmx_vminit: error setting guest msr access"); /* Index: sys/amd64/vmm/io/vlapic.c =================================================================== --- sys/amd64/vmm/io/vlapic.c (revision 264582) +++ sys/amd64/vmm/io/vlapic.c (working copy) @@ -143,7 +143,7 @@ #define VLAPIC_TIMER_UNLOCK(vlapic) mtx_unlock_spin(&((vlapic)->timer_mtx)) #define VLAPIC_TIMER_LOCKED(vlapic) mtx_owned(&((vlapic)->timer_mtx)) -#define VLAPIC_BUS_FREQ tsc_freq +#define VLAPIC_BUS_FREQ (128*1024*1024) static __inline uint32_t vlapic_get_id(struct vlapic *vlapic) Index: sys/amd64/vmm/vmm_msr.c =================================================================== --- sys/amd64/vmm/vmm_msr.c (revision 264582) +++ sys/amd64/vmm/vmm_msr.c (working copy) @@ -113,6 +113,9 @@ case MSR_MCG_CAP: guest_msrs[i] = 0; break; + case MSR_TSC: + guest_msrs[i] = rdtsc(); + break; case MSR_PAT: guest_msrs[i] = PAT_VALUE(0, PAT_WRITE_BACK) | PAT_VALUE(1, PAT_WRITE_THROUGH) | Index: sys/amd64/vmm/vmm_msr.h =================================================================== --- sys/amd64/vmm/vmm_msr.h (revision 264582) +++ sys/amd64/vmm/vmm_msr.h (working copy) @@ -29,7 +29,7 @@ #ifndef _VMM_MSR_H_ #define _VMM_MSR_H_ -#define VMM_MSR_NUM 16 +#define VMM_MSR_NUM 17 struct vm; void vmm_msr_init(void); Index: usr.sbin/bhyve/bhyverun.c =================================================================== --- usr.sbin/bhyve/bhyverun.c (revision 264582) +++ usr.sbin/bhyve/bhyverun.c (working copy) @@ -52,6 +52,7 @@ #include #include "bhyverun.h" +#include "compiledate.h" #include "acpi.h" #include "inout.h" #include "dbgport.h" @@ -75,6 +76,8 @@ #define MB (1024UL * 1024) #define GB (1024UL * MB) +#define FALSE 0 +#define TRUE (!FALSE) typedef int (*vmexit_handler_t)(struct vmctx *, struct vm_exit *, int *vcpu); @@ -139,8 +142,8 @@ " -S: legacy PCI slot config\n" " -l: LPC device configuration\n" " -m: memory size in MB\n" - " -w: ignore unimplemented MSRs\n", - progname, (int)strlen(progname), ""); + " -w: ignore unimplemented MSRs\n" + ,progname, (int)strlen(progname), ""); exit(code); } @@ -287,10 +290,6 @@ if (vme->u.inout.string || vme->u.inout.rep) return (VMEXIT_ABORT); - /* Special case of guest reset */ - if (out && port == 0x64 && (uint8_t)eax == 0xFE) - return (vmexit_catch_reset()); - /* Extra-special case of host notifications */ if (out && port == GUEST_NIO_PORT) return (vmexit_handle_notify(ctx, vme, pvcpu, eax)); @@ -315,16 +314,16 @@ uint64_t val; uint32_t eax, edx; int error; + val = 0; - val = 0; error = emulate_rdmsr(ctx, *pvcpu, vme->u.msr.code, &val); + if (error != 0) { - fprintf(stderr, "rdmsr to register %#x on vcpu %d\n", + fprintf(stderr, "rdmsr to register %#x ignored on vcpu %d\n\r", vme->u.msr.code, *pvcpu); if (strictmsr) return (VMEXIT_ABORT); } - eax = val; error = vm_set_register(ctx, *pvcpu, VM_REG_GUEST_RAX, eax); assert(error == 0); @@ -332,7 +331,6 @@ edx = val >> 32; error = vm_set_register(ctx, *pvcpu, VM_REG_GUEST_RDX, edx); assert(error == 0); - return (VMEXIT_CONTINUE); } @@ -343,7 +341,7 @@ error = emulate_wrmsr(ctx, *pvcpu, vme->u.msr.code, vme->u.msr.wval); if (error != 0) { - fprintf(stderr, "wrmsr to register %#x(%#lx) on vcpu %d\n", + fprintf(stderr, "wrmsr to register %#x(%#lx) ignored on vcpu %d\n\r", vme->u.msr.code, vme->u.msr.wval, *pvcpu); if (strictmsr) return (VMEXIT_ABORT); @@ -676,6 +674,7 @@ argc -= optind; argv += optind; + printf("BHyve compiled: %s \n\r\n\r", compiledate ); if (argc != 1) usage(1); Index: usr.sbin/bhyve/xmsr.c =================================================================== --- usr.sbin/bhyve/xmsr.c (revision 264582) +++ usr.sbin/bhyve/xmsr.c (working copy) @@ -38,24 +38,72 @@ #include #include "xmsr.h" +#include "xmsr-info.h" +#define BIT(b) (1< Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C206470; Tue, 20 May 2014 21:03:21 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1948D22E2; Tue, 20 May 2014 21:03:20 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 584A6153ABF; Tue, 20 May 2014 23:03:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x_08s-bK0EcB; Tue, 20 May 2014 23:03:08 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:65d0:df9:94a1:3ef5] (unknown [IPv6:2001:4cb8:3:1:65d0:df9:94a1:3ef5]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 8392F153434; Tue, 20 May 2014 23:03:08 +0200 (CEST) Message-ID: <537BC30C.4070208@digiware.nl> Date: Tue, 20 May 2014 23:03:08 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish , Andriy Gapon Subject: Re: bhyve: svm (amd-v) update References: <53748481.8010108@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Neel Natu , FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 21:03:21 -0000 On 15-5-2014 17:56, Anish wrote: > Hi Andriy, > Thanks for your interest in SVM port of bhyve. I do have patch to sync it > to http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). If > patches looks good to you, we can submit it. I have been testing it on > Phenom box which lacks some of newer SVM features. I don't quite understand against what this patch is? Do I run it over head, to get SVM code into head? Or do I patch against bhyve_SVM, because in the later case I get complaints that fatal error: 'vlapic_priv.h' file not found # locate vlapic_priv.h /usr/srcs/head/sys/amd64/vmm/io/vlapic_priv.h So I'm guessing that is against head. But last time I looked at head, more than just the interrupt stuff was missing.... --WjW From owner-freebsd-virtualization@FreeBSD.ORG Wed May 21 04:55:05 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04A83E7A; Wed, 21 May 2014 04:55:05 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ABFD282D; Wed, 21 May 2014 04:55:04 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hi2so7855067wib.0 for ; Tue, 20 May 2014 21:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FyxmnFOWlAlL1txsd2k1rTnnhKxuGbmmRQPLmZSP65o=; b=GoqP1Av4j+GNggnDFyHUmspUMRrHpUt2Y+lgJeAes/tomJ7xRYTSry69YXoyosjRbD ggWl0dHuMg3NplIfhB4WWCn137KIcPYPmN1z8ViMJwkreyk6LaVCJyo0uZbNy4BPe/S2 q3143IW11aKomcuSmq4F2JPlSsMvXrSPVJmswP2ipdNK6WEEraUlWoENfo0TsTAB1zYw ACHvyloFQdXtzsznEwZ7gE85FhY/E+b0xM2n56EvPFDi2csjxyBnR68rbK3EsWyDBri+ hXwgYju7quPFXHbt5VGNMfxHKiGUsr4dBpV7/T8+dxowQbnshu/m3JjYmMCcyNlnW/cz FMLQ== MIME-Version: 1.0 X-Received: by 10.180.93.226 with SMTP id cx2mr8188420wib.16.1400648102133; Tue, 20 May 2014 21:55:02 -0700 (PDT) Received: by 10.194.135.49 with HTTP; Tue, 20 May 2014 21:55:02 -0700 (PDT) In-Reply-To: <537BC30C.4070208@digiware.nl> References: <53748481.8010108@FreeBSD.org> <537BC30C.4070208@digiware.nl> Date: Tue, 20 May 2014 21:55:02 -0700 Message-ID: Subject: Re: bhyve: svm (amd-v) update From: Anish To: Willem Jan Withagen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD virtualization , Andriy Gapon , Neel Natu X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 04:55:05 -0000 Hi Willem, > I patch against bhyve_SVM, because in the later case I get complaints that This patch is to sync bhyve_svm project branch with HEAD @263780, so you have to first merge HEAD to bhyve_svm. It will prompt you to resolve conflict in amdv.c, you should accept the changes that are in bhyve_svm and then apply the patch. bhyve HEAD exposed vlapic related interfaces along with some other changes, this patch will enable vlapic interfaces for SVM. Thanks and regards, Anish On Tue, May 20, 2014 at 2:03 PM, Willem Jan Withagen wrote: > On 15-5-2014 17:56, Anish wrote: > > Hi Andriy, > > Thanks for your interest in SVM port of bhyve. I do have patch to sync > it > > to http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). > If > > patches looks good to you, we can submit it. I have been testing it on > > Phenom box which lacks some of newer SVM features. > > I don't quite understand against what this patch is? > > Do I run it over head, to get SVM code into head? > Or do I patch against bhyve_SVM, because in the later case I get > complaints that > fatal error: 'vlapic_priv.h' file not found > > # locate vlapic_priv.h > /usr/srcs/head/sys/amd64/vmm/io/vlapic_priv.h > > So I'm guessing that is against head. > But last time I looked at head, more than just the interrupt stuff was > missing.... > > --WjW > > > From owner-freebsd-virtualization@FreeBSD.ORG Wed May 21 07:11:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4BABE59A for ; Wed, 21 May 2014 07:11:27 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B9BBC2413 for ; Wed, 21 May 2014 07:11:26 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id E81F8153ABA; Wed, 21 May 2014 09:11:20 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9Te0j003yANE; Wed, 21 May 2014 09:11:13 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8] (unknown [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8]) by smtp.digiware.nl (Postfix) with ESMTP id 2C87C153434; Wed, 21 May 2014 09:11:13 +0200 (CEST) Message-ID: <537C518C.7030505@digiware.nl> Date: Wed, 21 May 2014 09:11:08 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish , freebsd-virtualization@freebsd.org Subject: Re: bhyve: svm (amd-v) update References: <53748481.8010108@FreeBSD.org> <537BC30C.4070208@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 07:11:27 -0000 On 2014-05-21 6:59, Anish wrote: > Hi Willem, > Thanks for sharing your patch. I see that you have gone further than > what I have. Are you able to boot Linux with these changes? I'm running ubuntu-{12.x,14.04} with this and I'm able to compile the linux kernel. Peter helped a lot in getting it going. Although that is already takeing something like a week... So it is slow. You might also need my ahci hack/patch, since linux uses a lot of flushes and there is a 'bug' in the FreeBSD end of ahci that does not really cooperate with the way ubuntu-14.04 calls it. So I've emulated a flush to a NO-OP. which then results in a panic because the ahci command-list is empty. So I 'fixed' that too, and then things run, albeit slow. CentOS it tried at the begining, but I'm just not a RH fan, so when that was too slow, I gave up. Give some time and I'll try and extract my ahci-hack-patch, and that is the last part that'll let you get on your way, booting linux. I do not have a lot of time to look at this, so I'm already a few weeks stuck on speeding up the linux-kernels. Also because I do not know enough of the inards of Linux.. --WjW > Thanks and regards, > Anish > > > > On Tue, May 20, 2014 at 2:03 PM, Willem Jan Withagen > wrote: > > On 15-5-2014 17:56, Anish wrote: > > Hi Andriy, > > Thanks for your interest in SVM port of bhyve. I do have patch > to sync it > > to > http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). If > > patches looks good to you, we can submit it. I have been testing > it on > > Phenom box which lacks some of newer SVM features. > > I don't quite understand against what this patch is? > > Do I run it over head, to get SVM code into head? > Or do I patch against bhyve_SVM, because in the later case I get > complaints that > fatal error: 'vlapic_priv.h' file not found > > # locate vlapic_priv.h > /usr/srcs/head/sys/amd64/vmm/io/vlapic_priv.h > > So I'm guessing that is against head. > But last time I looked at head, more than just the interrupt stuff was > missing.... > > --WjW > > > From owner-freebsd-virtualization@FreeBSD.ORG Wed May 21 07:51:20 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA65BCD6; Wed, 21 May 2014 07:51:20 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 762492889; Wed, 21 May 2014 07:51:20 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 6E8E3153ABA; Wed, 21 May 2014 09:51:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x0rdqrjwoEGv; Wed, 21 May 2014 09:51:16 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8] (unknown [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8]) by smtp.digiware.nl (Postfix) with ESMTP id 8B968153434; Wed, 21 May 2014 09:51:16 +0200 (CEST) Message-ID: <537C5AF0.7070403@digiware.nl> Date: Wed, 21 May 2014 09:51:12 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish Subject: Re: bhyve: svm (amd-v) update References: <53748481.8010108@FreeBSD.org> <537BC30C.4070208@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization , Andriy Gapon , Neel Natu X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 07:51:20 -0000 On 2014-05-21 6:55, Anish wrote: > Hi Willem, > > > I patch against bhyve_SVM, because in the later case I get > complaints that > This patch is to sync bhyve_svm project branch with HEAD @263780, so you > have to first merge HEAD to bhyve_svm. It will prompt you to resolve > conflict in amdv.c, you should accept the changes that are in bhyve_svm > and then apply the patch. bhyve HEAD exposed vlapic > related interfaces along with some other changes, this patch will enable > vlapic interfaces for SVM. I'd be interested in the vlapic to if that helps the speed. But you can help me a lot if you give me the SVN commands to do what you described above. I can fetch a clean bhyve_svm brach, but that is as far as my svn goes. I'll see if I can get my patches in as well. Thanx, --WjW > > Thanks and regards, > Anish > > > On Tue, May 20, 2014 at 2:03 PM, Willem Jan Withagen > wrote: > > On 15-5-2014 17:56, Anish wrote: > > Hi Andriy, > > Thanks for your interest in SVM port of bhyve. I do have patch > to sync it > > to > http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). If > > patches looks good to you, we can submit it. I have been testing > it on > > Phenom box which lacks some of newer SVM features. > > I don't quite understand against what this patch is? > > Do I run it over head, to get SVM code into head? > Or do I patch against bhyve_SVM, because in the later case I get > complaints that > fatal error: 'vlapic_priv.h' file not found > > # locate vlapic_priv.h > /usr/srcs/head/sys/amd64/vmm/io/vlapic_priv.h > > So I'm guessing that is against head. > But last time I looked at head, more than just the interrupt stuff was > missing.... > > --WjW > > > From owner-freebsd-virtualization@FreeBSD.ORG Wed May 21 09:31:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EA45990 for ; Wed, 21 May 2014 09:31:48 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id 2DD812431 for ; Wed, 21 May 2014 09:31:47 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id D5D8F14148DF for ; Wed, 21 May 2014 11:31:32 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 8E5B41A056C for ; Wed, 21 May 2014 11:31:31 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Wed, 21 May 2014 11:31:31 +0200 User-Agent: KNode/4.12.4 Content-Transfer-Encoding: 7Bit Subject: Re: bhyve: svm (amd-v) update To: freebsd-virtualization@freebsd.org References: <53748481.8010108@FreeBSD.org> <537BC30C.4070208@digiware.nl> <537C5AF0.7070403@digiware.nl> Lines: 55 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 09:31:48 -0000 Hi Willem, Willem Jan Withagen wrote: > I'd be interested in the vlapic to if that helps the speed. > But you can help me a lot if you give me the SVN commands to do what you > described above. These were my steps: 0) mv /usr/src /usr/src.bak 1) svnlite co svn://svn.freebsd.org/base/projects/bhyve_svm /usr/src 2) cd /usr/src 3) patch -p4 < /tmp/bhyve_svm_HEAD_r263780.patch 4) svnlite merge svn://svn.freebsd.org/base/head one conflict in file "amdv.c" - enter "mf" (mine-full); in my previous post, I mistakenly said "theirs-full"; what is, of course, wrong. 5) manually patch "amdv.c" with: <----------------------- SNIP -----------------------------> Index: sys/amd64/vmm/amd/amdv.c =================================================================== --- sys/amd64/vmm/amd/amdv.c (revision 266491) +++ sys/amd64/vmm/amd/amdv.c (working copy) @@ -99,7 +99,7 @@ } static void -amd_iommu_add_device(void *domain, int bus, int slot, int func) +amd_iommu_add_device(void *domain, uint16_t rid) { printf("amd_iommu_add_device: not implemented\n"); @@ -106,7 +106,7 @@ } static void -amd_iommu_remove_device(void *domain, int bus, int slot, int func) +amd_iommu_remove_device(void *domain, uint16_t rid) { printf("amd_iommu_remove_device: not implemented\n"); <----------------------- SNIP -----------------------------> 6) should be fine now to compile and to integrate your patches Thanks a lot for your work and regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Wed May 21 09:57:11 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1AE2106 for ; Wed, 21 May 2014 09:57:11 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 986A22665 for ; Wed, 21 May 2014 09:57:11 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 31A07153AC5; Wed, 21 May 2014 11:57:08 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pCx-0ArmnwOd; Wed, 21 May 2014 11:57:06 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8] (unknown [IPv6:2001:4cb8:3:1:5099:2185:4a0e:9ad8]) by smtp.digiware.nl (Postfix) with ESMTP id 92073153AC2; Wed, 21 May 2014 11:57:06 +0200 (CEST) Message-ID: <537C786E.3010106@digiware.nl> Date: Wed, 21 May 2014 11:57:02 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-virtualization@freebsd.org Subject: Re: bhyve: svm (amd-v) update References: <53748481.8010108@FreeBSD.org> <537BC30C.4070208@digiware.nl> <537C5AF0.7070403@digiware.nl> <20140521093154.835069CC@hub.freebsd.org> In-Reply-To: <20140521093154.835069CC@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 09:57:12 -0000 On 2014-05-21 11:31, Nils Beyer wrote: > Hi Willem, > > Willem Jan Withagen wrote: >> I'd be interested in the vlapic to if that helps the speed. >> But you can help me a lot if you give me the SVN commands to do what you >> described above. > > These were my steps: > > 0) mv /usr/src /usr/src.bak > > 1) svnlite co svn://svn.freebsd.org/base/projects/bhyve_svm /usr/src > > 2) cd /usr/src > > 3) patch -p4 < /tmp/bhyve_svm_HEAD_r263780.patch > > 4) svnlite merge svn://svn.freebsd.org/base/head > > one conflict in file "amdv.c" - enter "mf" (mine-full); in my previous > post, I mistakenly said "theirs-full"; what is, of course, wrong. > > 5) manually patch "amdv.c" with: > > <----------------------- SNIP -----------------------------> > Index: sys/amd64/vmm/amd/amdv.c > =================================================================== > --- sys/amd64/vmm/amd/amdv.c (revision 266491) > +++ sys/amd64/vmm/amd/amdv.c (working copy) > @@ -99,7 +99,7 @@ > } > > static void > -amd_iommu_add_device(void *domain, int bus, int slot, int func) > +amd_iommu_add_device(void *domain, uint16_t rid) > { > > printf("amd_iommu_add_device: not implemented\n"); > @@ -106,7 +106,7 @@ > } > > static void > -amd_iommu_remove_device(void *domain, int bus, int slot, int func) > +amd_iommu_remove_device(void *domain, uint16_t rid) > { > > printf("amd_iommu_remove_device: not implemented\n"); > <----------------------- SNIP -----------------------------> > > > 6) should be fine now to compile and to integrate your patches Thanx, Very helpfull... And lets see if we can get amd-v (back) up to speed as well. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 15:26:12 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39F06931 for ; Thu, 22 May 2014 15:26:12 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDDDB2D12 for ; Thu, 22 May 2014 15:26:08 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id ED734153A9E for ; Thu, 22 May 2014 17:25:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nU00K9TiVL75; Thu, 22 May 2014 17:25:57 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:8c97:b457:abcf:a887] (unknown [IPv6:2001:4cb8:3:1:8c97:b457:abcf:a887]) by smtp.digiware.nl (Postfix) with ESMTP id 3DBD0153A9F for ; Thu, 22 May 2014 17:25:57 +0200 (CEST) Message-ID: <537E16FE.4070309@digiware.nl> Date: Thu, 22 May 2014 17:25:50 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: virtualization@freebsd.org Subject: bhyve on AMD, linux and high load Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 15:26:12 -0000 Hoi, Just a point on the timeline.... I think somebody asked why his CPU load was so high on AMD running linux. I've completed compiling a "fresh" linux-kernel on my Ubuntu 14.04 system, installed and rebooted it. And where previously a Linux-kernel would drain the CPUs it got assigned to the max... Aka 2 CPU would drive the load +2 and top would diskplay a 200% cpu. With the new kernel, that is no longer the case. On a idle vm the CPU load is like 6-7%.... At the moment I'm running: linux-image-3.13.0-24-generic_3.13.0-24.46_amd64 So somewhere in the interaction between bhyve idle detection and the older linux kernel things do not match... --WjW From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 16:24:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09284876 for ; Thu, 22 May 2014 16:24:41 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id BE83D22FC for ; Thu, 22 May 2014 16:24:39 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id 7667A14148EA for ; Thu, 22 May 2014 18:24:24 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 4F97C1A0763 for ; Thu, 22 May 2014 18:24:23 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Thu, 22 May 2014 18:24:23 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: bhyve on AMD, linux and high load To: freebsd-virtualization@freebsd.org References: <537E16FE.4070309@digiware.nl> Lines: 18 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 16:24:41 -0000 Hi Willem, Willem Jan Withagen wrote: > [...] > With the new kernel, that is no longer the case. On a idle vm the CPU > load is like 6-7%.... > > At the moment I'm running: > linux-image-3.13.0-24-generic_3.13.0-24.46_amd64 Now, that looks promising. Is that with or without your own bhyve-/SVM-patches? If that's with your own patches, have you already integrated Anish's patches in them? Regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 18:38:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C925CA96 for ; Thu, 22 May 2014 18:38:42 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 876B92E94 for ; Thu, 22 May 2014 18:38:42 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 2B2391534D1; Thu, 22 May 2014 20:38:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kKKgU8fi_Y2p; Thu, 22 May 2014 20:38:36 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:8c97:b457:abcf:a887] (unknown [IPv6:2001:4cb8:3:1:8c97:b457:abcf:a887]) by smtp.digiware.nl (Postfix) with ESMTP id 98BBE153434; Thu, 22 May 2014 20:38:36 +0200 (CEST) Message-ID: <537E4425.2030804@digiware.nl> Date: Thu, 22 May 2014 20:38:29 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-virtualization@freebsd.org Subject: Re: bhyve on AMD, linux and high load References: <537E16FE.4070309@digiware.nl> <20140522162447.3A5F08AE@hub.freebsd.org> In-Reply-To: <20140522162447.3A5F08AE@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 18:38:42 -0000 On 2014-05-22 18:24, Nils Beyer wrote: > Hi Willem, > > Willem Jan Withagen wrote: >> [...] >> With the new kernel, that is no longer the case. On a idle vm the CPU >> load is like 6-7%.... >> >> At the moment I'm running: >> linux-image-3.13.0-24-generic_3.13.0-24.46_amd64 > > Now, that looks promising. > > Is that with or without your own bhyve-/SVM-patches? If that's with your own > patches, have you already integrated Anish's patches in them? I've started building a linux kernel like over a week ago.... So that is with my patches on the "basic" bhyve-svm. I've just completed merging and patching with Anish patches... ONce that seems to run for my freebsd vm's, I'll start merging my patches. And hope that it still works. And best would be that the vlapic stuff gives a extra bit of speed, because we're not quite there yet. I'll let you know how I fare --WjW From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 19:29:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C7399BD for ; Thu, 22 May 2014 19:29:25 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id 5CCED22D7 for ; Thu, 22 May 2014 19:29:24 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id 5BC351414809 for ; Thu, 22 May 2014 21:29:16 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 37CD61A04FB for ; Thu, 22 May 2014 21:29:15 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Thu, 22 May 2014 21:29:14 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: bhyve on AMD, linux and high load To: freebsd-virtualization@freebsd.org References: <537E16FE.4070309@digiware.nl> <20140522162447.3A5F08AE@hub.freebsd.org> <537E4425.2030804@digiware.nl> Lines: 40 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 19:29:25 -0000 Hi, Willem Jan Withagen wrote: > I've just completed merging and patching with Anish patches... > ONce that seems to run for my freebsd vm's, I'll start merging my > patches. And hope that it still works. Sounds good. I've tried some Linux versions ranging from 2.6.32, 3.10.x, 3.11.10 to 3.12.8 using an Anish's patched SVM+HEAD variant. They all show the same behaviour being stucked during boot. I haven't tried your patches yet, though. > And best would be that the vlapic stuff gives a extra bit of speed, > because we're not quite there yet. I suppose you start your bhyved Linux instances using the "-A" switch (ACPI tables), right? If you feel interested, you can try to start them without the "-A" switch. For me, the boot process is now stucked at "Calibrating delay loop". As far as I understand, it probably has something to do with a borked "jiffies" value and that has probably something to do with a borked "IRQ 0" that modifes that "jiffies" value at every tick. But these are just wild guesses as I understand much too little about kernels, virtualizing and stuff. For what it's worth, using an Intel i3 and without the "-A" switch, the bhyved Linux instance just boots fine as it does with the "-A" switch. BTW: you haven't the oppurtunity to try bhyve SVN on a Barcelona- or later class Opteron, have you? Thanks again for all your work there. Looking forward to hearing your results... Regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 23:03:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B442A45E for ; Thu, 22 May 2014 23:03:09 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70BF824DC for ; Thu, 22 May 2014 23:03:08 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 10B74153AAE; Fri, 23 May 2014 01:03:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SLaKRTvhML9v; Fri, 23 May 2014 01:03:04 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a00d:a276:299d:79c6] (unknown [IPv6:2001:4cb8:3:1:a00d:a276:299d:79c6]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 03A46153434; Fri, 23 May 2014 01:03:04 +0200 (CEST) Message-ID: <537E8228.3070405@digiware.nl> Date: Fri, 23 May 2014 01:03:04 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-virtualization@freebsd.org Subject: Re: bhyve on AMD, linux and high load References: <537E16FE.4070309@digiware.nl> <20140522162447.3A5F08AE@hub.freebsd.org> <537E4425.2030804@digiware.nl> <20140522192928.2BB2E9F5@hub.freebsd.org> In-Reply-To: <20140522192928.2BB2E9F5@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 23:03:09 -0000 On 22-5-2014 21:29, Nils Beyer wrote: > Hi, > > Willem Jan Withagen wrote: >> I've just completed merging and patching with Anish patches... >> ONce that seems to run for my freebsd vm's, I'll start merging my >> patches. And hope that it still works. > > Sounds good. I've tried some Linux versions ranging from 2.6.32, 3.10.x, > 3.11.10 to 3.12.8 using an Anish's patched SVM+HEAD variant. They all > show the same behaviour being stucked during boot. > > I haven't tried your patches yet, though. Well I'm running Anish SVM+HEAD and I'm no longer able to my old FreeBSD vm's. And I can't even get vmrun.sh to build new ones.... It just doesn't show anything on the commandline output, even if I go: -l com1,stdio Strange part is that the Linux 14.04 VM just starts fine.... So I have a complete reverse situation. Linux runs, but I have a hard time getting FreeBSD to run. >> And best would be that the vlapic stuff gives a extra bit of speed, >> because we're not quite there yet. > > I suppose you start your bhyved Linux instances using the "-A" switch (ACPI > tables), right? If you feel interested, you can try to start them without the > "-A" switch. For me, the boot process is now stucked at "Calibrating delay > loop". As far as I understand, it probably has something to do with a borked > "jiffies" value and that has probably something to do with a borked "IRQ 0" > that modifes that "jiffies" value at every tick. Yes I use -A on all my boots. > But these are just wild guesses as I understand much too little about > kernels, virtualizing and stuff. I'm the last one suggesting that I know more that just a little bit, but I get by.. > For what it's worth, using an Intel i3 and without the "-A" switch, the > bhyved Linux instance just boots fine as it does with the "-A" switch. One of the reasons tinkering with AMD is that I don't have an i{3,5,7} cpus available. > BTW: you haven't the oppurtunity to try bhyve SVN on a Barcelona- or later > class Opteron, have you? Nope, just got a bunch of boards leftover from a previous project. And they are all: CPU: AMD Phenom(tm) II X6 1075T Processor (3013.83-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x100fa0 Family=0x10 Model=0xa Stepping=0 From owner-freebsd-virtualization@FreeBSD.ORG Thu May 22 23:10:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37C1F6F4 for ; Thu, 22 May 2014 23:10:40 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8FE3258C for ; Thu, 22 May 2014 23:10:39 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id A880F153AAF; Fri, 23 May 2014 01:10:37 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xx6fgIpCY3Bi; Fri, 23 May 2014 01:10:35 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a00d:a276:299d:79c6] (unknown [IPv6:2001:4cb8:3:1:a00d:a276:299d:79c6]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id CD026153434; Fri, 23 May 2014 01:10:35 +0200 (CEST) Message-ID: <537E83EC.5090304@digiware.nl> Date: Fri, 23 May 2014 01:10:36 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-virtualization@freebsd.org Subject: Re: bhyve on AMD, linux and high load References: <537E16FE.4070309@digiware.nl> <20140522162447.3A5F08AE@hub.freebsd.org> <537E4425.2030804@digiware.nl> <20140522192928.2BB2E9F5@hub.freebsd.org> In-Reply-To: <20140522192928.2BB2E9F5@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 23:10:40 -0000 On 22-5-2014 21:29, Nils Beyer wrote: > I suppose you start your bhyved Linux instances using the "-A" switch (ACPI > tables), right? If you feel interested, you can try to start them without the > "-A" switch. For me, the boot process is now stucked at "Calibrating delay > loop". As far as I understand, it probably has something to do with a borked > "jiffies" value and that has probably something to do with a borked "IRQ 0" > that modifes that "jiffies" value at every tick. Same here.... --WjW From owner-freebsd-virtualization@FreeBSD.ORG Fri May 23 21:22:23 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9DC373A; Fri, 23 May 2014 21:22:23 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A3E1280D; Fri, 23 May 2014 21:22:23 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id DC873153AB0; Fri, 23 May 2014 23:22:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FskH30yNpdKn; Fri, 23 May 2014 23:22:16 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:cdfd:64b5:d02:67e9] (unknown [IPv6:2001:4cb8:3:1:cdfd:64b5:d02:67e9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id E2113153434; Fri, 23 May 2014 23:22:16 +0200 (CEST) Message-ID: <537FBC09.9020403@digiware.nl> Date: Fri, 23 May 2014 23:22:17 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish , Andriy Gapon Subject: Re: bhyve: svm (amd-v) update References: <53748481.8010108@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Neel Natu , FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2014 21:22:23 -0000 On 15-5-2014 17:56, Anish wrote: > Hi Andriy, > Thanks for your interest in SVM port of bhyve. I do have patch to sync it > to http://svnweb.freebsd.org/base?view=revision&revision=263780(3/26). If > patches looks good to you, we can submit it. I have been testing it on > Phenom box which lacks some of newer SVM features. Hi, With this patchset I see inb(0x40) calls coming thru to the bhyve VM, where I would expect them to stay down in the VMM dev-driver and emulate the vatpit for that VM. default_inout: port 64, in, bytes 1 default_inout: port 64, in, bytes 1 default_inout: port 67, out, bytes 1 Any particular reason that the IO for that register is not intercepted and ends up in userspace? Any suggestions what to watch for? Not really strong in debuging kernel modules. Thanx, --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun May 25 01:50:33 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47441CFC for ; Sun, 25 May 2014 01:50:33 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AEC7202D for ; Sun, 25 May 2014 01:50:29 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s4P1oJcK059690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 25 May 2014 01:50:25 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s4P1oJDp059689 for virtualization@FreeBSD.org; Sun, 25 May 2014 01:50:19 GMT (envelope-from swills) Date: Sun, 25 May 2014 01:50:19 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: bhyve ahci issue Message-ID: <20140525015017.GB50490@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sun, 25 May 2014 01:50:25 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 01:50:33 -0000 Hi, A bhyve VM I run had this: pid 79784 (bhyve), uid 0: exited on signal 6 (core dumped) and the only thing on the console was this: ahcich0: Timeout on slot 27 port 0 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 50 serr 00000000 cmd 1000c017 (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 48 ca a6 6a 40 0b 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command Assertion failed: (aior != NULL), function ahci_handle_dma, file /usr/src/usr.sbin/bhyve/pci_ahci.c, line 493. I guess the disks were just too busy? Steve From owner-freebsd-virtualization@FreeBSD.ORG Sun May 25 19:00:52 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07D08B23; Sun, 25 May 2014 19:00:52 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE5872880; Sun, 25 May 2014 19:00:51 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id AAD06153A9F; Sun, 25 May 2014 21:00:47 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lNihFpMbwNfX; Sun, 25 May 2014 21:00:45 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:9c9a:f8a3:b550:e3bd] (unknown [IPv6:2001:4cb8:3:1:9c9a:f8a3:b550:e3bd]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 6CC4A1534D1; Sun, 25 May 2014 21:00:45 +0200 (CEST) Message-ID: <53823DDE.2040805@digiware.nl> Date: Sun, 25 May 2014 21:00:46 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Steve Wills , virtualization@FreeBSD.org Subject: Re: bhyve ahci issue References: <20140525015017.GB50490@mouf.net> In-Reply-To: <20140525015017.GB50490@mouf.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 19:00:52 -0000 On 25-5-2014 3:50, Steve Wills wrote: > Hi, > > A bhyve VM I run had this: > > pid 79784 (bhyve), uid 0: exited on signal 6 (core dumped) > > and the only thing on the console was this: > > ahcich0: Timeout on slot 27 port 0 > ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 50 serr 00000000 cmd 1000c017 > (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 48 ca a6 6a 40 0b 00 00 00 00 00 > (ada0:ahcich0:0:0:0): CAM status: Command timeout > (ada0:ahcich0:0:0:0): Retrying command > Assertion failed: (aior != NULL), function ahci_handle_dma, file /usr/src/usr.sbin/bhyve/pci_ahci.c, line 493. I've converted this Assert into a regular test of an empty list, and up'till I have not run into trouble.... I needed to remove/convert it, because running Linux with ahci on a AMD cpu ran into ATA FLUSH problems.. So I'm currently ignoring Linux ATA FLUSH. But perhaps others can share some light on why this as to be an Assert in this corner of the code instead of just a regular test. I also do get FPDMA timeouts under Linux with AHCI, but they do not panic, again because of the "fix". --WjW From owner-freebsd-virtualization@FreeBSD.ORG Mon May 26 00:26:10 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B22CB909; Mon, 26 May 2014 00:26:10 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5506721AB; Mon, 26 May 2014 00:26:10 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so11015223qge.22 for ; Sun, 25 May 2014 17:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ous1KBjAa7rZWKa3jRV1LGFdoki9zNcRqJlyJOXiyko=; b=Xcly2lTJwa3AN6zSvk/PHxh0+aJkcJrayznBSqsFeK6TD09hInXRvG4KcIZBHpoHz4 te/nQiIpbgDuClFYOoSdxufEc8RMhJj8NzHmXNqjQ3qt6SJulE5z4bnkvOq/TB0ZVdgt E0/eUB3G2R+9X3EfNjZQTkf1ckrpq4tACm356AueakN47JKTPbkKz8AtoGkCl8Jtwm6k NVjecB0AxDGrDMfDhz5ynL4QVLXzvBtXxOY7Ti0+Wjdl9kps0KboLDHfFSPrZWKjnQgM ILDA7pnwA4K+H8+5l/78Qi2k5TJy9nrcNRqn5zvqVFtnqoEOe12EOmbBf/yGU0512szv P5kg== MIME-Version: 1.0 X-Received: by 10.140.32.97 with SMTP id g88mr25591051qgg.17.1401063969244; Sun, 25 May 2014 17:26:09 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Sun, 25 May 2014 17:26:09 -0700 (PDT) In-Reply-To: <20140525015017.GB50490@mouf.net> References: <20140525015017.GB50490@mouf.net> Date: Sun, 25 May 2014 17:26:09 -0700 Message-ID: Subject: Re: bhyve ahci issue From: Neel Natu To: Steve Wills Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" , Tycho Nightingale X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 00:26:10 -0000 Hi Steve, On Sat, May 24, 2014 at 6:50 PM, Steve Wills wrote: > Hi, > > A bhyve VM I run had this: > > pid 79784 (bhyve), uid 0: exited on signal 6 (core dumped) > > and the only thing on the console was this: > > ahcich0: Timeout on slot 27 port 0 > ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 50 serr 00000000 cmd 1000c017 > (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 48 ca a6 6a 40 0b 00 00 00 00 00 > (ada0:ahcich0:0:0:0): CAM status: Command timeout > (ada0:ahcich0:0:0:0): Retrying command > Assertion failed: (aior != NULL), function ahci_handle_dma, file /usr/src/usr.sbin/bhyve/pci_ahci.c, line 493. > > I guess the disks were just too busy? > Are you running 10.0 or HEAD? I *think* that this was fixed in r263322 but Tycho (cc-ed) can answer this definitively. http://svnweb.freebsd.org/base?view=revision&revision=263322 best Neel > Steve > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Mon May 26 11:06:55 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4205BF9A for ; Mon, 26 May 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E0EE24F9 for ; Mon, 26 May 2014 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4QB6ts0032209 for ; Mon, 26 May 2014 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4QB6sKv032207 for freebsd-virtualization@FreeBSD.org; Mon, 26 May 2014 11:06:54 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 26 May 2014 11:06:54 GMT Message-Id: <201405261106.s4QB6sKv032207@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 11:06:55 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Tue May 27 01:51:23 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EC6D811 for ; Tue, 27 May 2014 01:51:23 +0000 (UTC) Received: from mail-oa0-x234.google.com (mail-oa0-x234.google.com [IPv6:2607:f8b0:4003:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D1BD2FCC for ; Tue, 27 May 2014 01:51:23 +0000 (UTC) Received: by mail-oa0-f52.google.com with SMTP id eb12so8754746oac.25 for ; Mon, 26 May 2014 18:51:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=I/uHRj0oK7oJUTf3/aYWJaRIDQnjkc2fogexU2MINqc=; b=mI2Vlw+6yrAHKSZd6TIod/4r3/NIayEB77bbFBmw3F3JvIWLsqk6cQS7E6ZhhE5pP3 0FSoRVh+9U0M0FRXzj3Iu7wltrMU1fYEFKcqdWN/F1w0MoHcdEj/Pl5y4qT/q7dItafB hJaahUVVPUz3fZdYa4VIuY7B4U1wf7J3MHMp5vtrduj4K7Oo8g5JWOkfw++eYkdEEtP3 JS/9vkOovx1WWx8/Bw1KtmoO8BJRxHk9xDwdlSQAs9OQL6siec9gEQ5drOyEF9QgEhua wt7zw0Qn7vr9GwJlZfGemZqmy4eYIhtZmqz1p8/pxd2WlGVUkTDjUcmSooZ0k7qfv+/4 a4/A== MIME-Version: 1.0 X-Received: by 10.182.126.137 with SMTP id my9mr29178314obb.61.1401155482423; Mon, 26 May 2014 18:51:22 -0700 (PDT) Received: by 10.76.23.130 with HTTP; Mon, 26 May 2014 18:51:22 -0700 (PDT) Date: Mon, 26 May 2014 21:51:22 -0400 Message-ID: Subject: SR-IOV Patch Series 2/7: bhyve integration From: Ryan Stone To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 01:51:23 -0000 The bhyve work to interoperate is quite simple. PCI Passthrough through the ppt driver works just fine with SR-IOV VFs. The three changes are: - allow ppt devices to detach. This would happen if the administrator destroyed VFs with iovctl -D - allow the SR-IOV infrastructure to force the ppt driver to attach to certain devices by using the special BUS_PROBE_NOWILDCARD return value from device_probe(). This will have no effect on regular devices, but the SR-IOV infrastructure can create VF devices in a way where BUS_PROBE_NOWILDCARD will match the device. This allows passthrough of a given VF to be configured through iovctl (see ioctl.conf(5) for an example of how to do this). - Currently if vmm.ko is loaded and no ppt devices are present, the IOMMU is not enabled. If VFs are subsequently created as passthrough devices it will not be possible to pass them through to VMs. This patch adds a new tunable that will force the IOMMU to be enabled. Note: Currently ppt suffers from a lack of locking. This is now a problem because ppt devices can be destroyed, so there would be races here. I tried to address this by adding a mutex but that failed in ways that I could not figure out how to resolve (basically, some parts of ppt are called into from within a critical section, so a spinlock is needed, but other parts of ppt call into the VM subsystem to do M_WAITOK allocations, so an sx lock is needed. It wasn't at all obvious to me how to resolve that conflict, so I'm hoping that a more experienced bhyve person has an idea). http://people.freebsd.org/~rstone/patches/iov/0004-Allow-passthrough-devices-to-be-hinted.patch [PATCH 04/21] Allow passthrough devices to be hinted. Allow the ppt driver to attach to devices that were hinted to be passthrough devices by the PCI code creating them with a driver name of "ppt". Add a tunable that allows the IOMMU to be forced to be used. With SR-IOV passthrough devices the VFs may be created after vmm.ko is loaded. The current code will not initialize the IOMMU in that case, meaning that the passthrough devices can't actually be used. --- sys/amd64/vmm/io/ppt.c | 72 +++++++++++++++++++++++++++----------------------- sys/amd64/vmm/vmm.c | 7 ++++- From owner-freebsd-virtualization@FreeBSD.ORG Tue May 27 14:42:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E3254E7 for ; Tue, 27 May 2014 14:42:27 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEEEF214E for ; Tue, 27 May 2014 14:42:26 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id uy5so9443749obc.1 for ; Tue, 27 May 2014 07:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=V7sK6hyCwLXGIqHT3OGBDbiBa+NX+8WfzD/JPMjF/Mk=; b=pLA+JGiBlYghZXvilqlMMMJyLBqxsg1z1lWzNeBKPPKx/aFm2olTrX4jtZSKe/o5eB dnJ/xyPaJ3DpSCEBBYy4tIR12jV2XiZ7yle3I+D926mpZolktIJwYHOQYBI/YBD7eXPp PwUVzEbafRhU0kbV1U8bhnh7L09YGCxEHYar2LZ9potlKKdlaGn63l8Qc5cSBj3fQT4p 6ruJ34KviP9FoGTVCYWFkA1fZ+UQWzzyCTicavNc3MshFHl9pct5hxT+jjZQ+UJqosu3 hsxn3gKzBP/K5JHBzs5mIliaa5uJ/4FQqcPFsfiaHmYvdErYmAAzqdRKmiYdZkMVYJDw hccw== MIME-Version: 1.0 X-Received: by 10.60.134.109 with SMTP id pj13mr33233278oeb.63.1401201746325; Tue, 27 May 2014 07:42:26 -0700 (PDT) Received: by 10.76.23.130 with HTTP; Tue, 27 May 2014 07:42:26 -0700 (PDT) In-Reply-To: References: Date: Tue, 27 May 2014 10:42:26 -0400 Message-ID: Subject: Re: SR-IOV Patch Series 2/7: bhyve integration From: Ryan Stone To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2014 14:42:27 -0000 On Mon, May 26, 2014 at 9:51 PM, Ryan Stone wrote: > http://people.freebsd.org/~rstone/patches/iov/0004-Allow-passthrough-devices-to-be-hinted.patch https://phabric.freebsd.org/D73 From owner-freebsd-virtualization@FreeBSD.ORG Wed May 28 00:14:05 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEFB1165 for ; Wed, 28 May 2014 00:14:05 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F7DC2566 for ; Wed, 28 May 2014 00:14:05 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 7B1571534D1 for ; Wed, 28 May 2014 02:14:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EYleC2OLhEFg; Wed, 28 May 2014 02:13:58 +0200 (CEST) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id CAB61153434 for ; Wed, 28 May 2014 02:13:58 +0200 (CEST) Message-ID: <53852A48.9080703@digiware.nl> Date: Wed, 28 May 2014 02:14:00 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: "freebsd-virtualization@freebsd.org" Subject: Trying to run DragonFly under bhyve Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 00:14:05 -0000 When I do this under AMD I get: Copyright (c) 2003-2013 The DragonFly Project. Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. Failed to emulate instruction at 0xffffffff8096052c Abort trap (core dumped) To conclude which instruction this is, I need to get at the bytes of that instruction... but that stays hidden in the vmm-driver. Any easy way to get this back into userspace? --WjW From owner-freebsd-virtualization@FreeBSD.ORG Wed May 28 00:22:11 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3F1627A for ; Wed, 28 May 2014 00:22:11 +0000 (UTC) Received: from mail-pb0-f48.google.com (mail-pb0-f48.google.com [209.85.160.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A711A2610 for ; Wed, 28 May 2014 00:22:11 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id rr13so10120529pbb.21 for ; Tue, 27 May 2014 17:22:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=BCLyxb07kriNYd3SeZvbd3vtx/EHARPmypSiG//bkV4=; b=K6/IKhgKr0JyzO3c1bZxuffKSD++JMua02qXxLB55Gtqq1bIuBGWoSGkv+eaJHBrMn +tLSMqywyM8YnUbwTEbf4dYm/D8BG8KurjkDX50aWaEGXE6Pgw4pTWwHzQEwKIrj74Q1 UGFd/6OwJ3gpBXzIl3E1KMUJk+KlSaJcVaDnzFCDJiXXoXpmGK2KewU0AAEi1ph+yNPy aPJbhcRsd/fgNn9+s4VlyzdI7hdPJPPmgmn2OwOOwSL6OZKI2fAv2x8eAxogfdB8TaaO N1MH++eSI2qQQXFbgcUgfH45GC0dj8CufHp2ysDCW2V4W8rWV8S+IKKbeV0UduUoEjxs Pysw== X-Gm-Message-State: ALoCoQnbrvmm4v2a3QcYWSmpnjUkEWrj4tzWpiOqdAlvEFydbRubRWn875zTLP8I6k8wNmQA87vy X-Received: by 10.68.202.194 with SMTP id kk2mr40437835pbc.156.1401236525499; Tue, 27 May 2014 17:22:05 -0700 (PDT) Received: from [10.0.1.4] (209-6-121-211.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com. [209.6.121.211]) by mx.google.com with ESMTPSA id pr4sm25248367pbb.53.2014.05.27.17.22.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 27 May 2014 17:22:04 -0700 (PDT) Subject: Re: Trying to run DragonFly under bhyve Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Tycho Nightingale In-Reply-To: <53852A48.9080703@digiware.nl> Date: Tue, 27 May 2014 20:22:01 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <53852A48.9080703@digiware.nl> To: Willem Jan Withagen X-Mailer: Apple Mail (2.1283) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 00:22:11 -0000 On May 27, 2014, at 8:14 PM, Willem Jan Withagen wrote: > When I do this under AMD I get: >=20 > Copyright (c) 2003-2013 The > DragonFly Project. > Copyright (c) 1992-2003 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, = 1994 > The Regents of the University of California. All rights = reserved. > Failed to emulate instruction at 0xffffffff8096052c > Abort trap (core dumped) >=20 > To conclude which instruction this is, I need to get at the bytes of > that instruction... but that stays hidden in the vmm-driver. >=20 > Any easy way to get this back into userspace? You could try 'objdump -d' on a copy of the guest's kernel to find the = relevant instruction. Tycho= From owner-freebsd-virtualization@FreeBSD.ORG Wed May 28 04:34:21 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76D6B9E2 for ; Wed, 28 May 2014 04:34:21 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1181E28B7 for ; Wed, 28 May 2014 04:34:20 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id l18so10392412wgh.11 for ; Tue, 27 May 2014 21:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=I6pckorh8hZ+OzKVgKBOrqr3ovARE1sqqK1hGESCaLU=; b=hm/sIy7vvoVcZBTYQOU3/mIS4D9aSmxF9I4VNvFAfc4JW2CHCU77q5ZZh5o2spfm4Z a5wTVoLkZsTHH98XvuRktwvy49T7nfSijJ4srquHxg3jIx6BrMI7wE4RL0MyPjgrM6LH THo+q9+DKOYZxzwUqa0O1W8+Y72sifhrLQuL1kAztjJN6QJkGnUGIvfk0dE0l/ojQ489 AwDzO+ncTvwiSsFZTV3WxJf7z040kixfKhW0Xo44lBhr7TM38AirNpBdFbfAKFIt0MSV UR8Y2+5QzzLxMSHFDEALmpLyt7xbfE88jRWbtgqXWPSfgYWiXim2FR7eNDppWf4UpPsj fzSQ== MIME-Version: 1.0 X-Received: by 10.180.126.33 with SMTP id mv1mr44246707wib.6.1401251658134; Tue, 27 May 2014 21:34:18 -0700 (PDT) Received: by 10.194.135.49 with HTTP; Tue, 27 May 2014 21:34:18 -0700 (PDT) In-Reply-To: <53852A48.9080703@digiware.nl> References: <53852A48.9080703@digiware.nl> Date: Tue, 27 May 2014 21:34:18 -0700 Message-ID: Subject: Re: Trying to run DragonFly under bhyve From: Anish To: Willem Jan Withagen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 04:34:21 -0000 >Failed to emulate instruction at 0xffffffff8096052c Abort trap (core dumped) You can also analyze the coredump of bhyve $gdb /usr/sbin/bhyve bhyve.core Look at vie->inst[] from one of stack frame. -Anish On Tue, May 27, 2014 at 5:14 PM, Willem Jan Withagen wrote: > When I do this under AMD I get: > > Copyright (c) 2003-2013 The > DragonFly Project. > Copyright (c) 1992-2003 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > Failed to emulate instruction at 0xffffffff8096052c > Abort trap (core dumped) > > To conclude which instruction this is, I need to get at the bytes of > that instruction... but that stays hidden in the vmm-driver. > > Any easy way to get this back into userspace? > > --WjW > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Wed May 28 10:03:35 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66054ABA for ; Wed, 28 May 2014 10:03:35 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2447F2403 for ; Wed, 28 May 2014 10:03:34 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 1EE791534D1; Wed, 28 May 2014 12:03:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hnFquyWB_73W; Wed, 28 May 2014 12:03:26 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:5cbb:e5e6:6f86:6c35] (unknown [IPv6:2001:4cb8:3:1:5cbb:e5e6:6f86:6c35]) by smtp.digiware.nl (Postfix) with ESMTP id 76050153434; Wed, 28 May 2014 12:03:26 +0200 (CEST) Message-ID: <5385B471.9020703@digiware.nl> Date: Wed, 28 May 2014 12:03:29 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Tycho Nightingale Subject: Re: Trying to run DragonFly under bhyve References: <53852A48.9080703@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 10:03:35 -0000 On 2014-05-28 2:22, Tycho Nightingale wrote: > > On May 27, 2014, at 8:14 PM, Willem Jan Withagen wrote: > >> When I do this under AMD I get: >> >> Copyright (c) 2003-2013 The >> DragonFly Project. >> Copyright (c) 1992-2003 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> Failed to emulate instruction at 0xffffffff8096052c >> Abort trap (core dumped) >> >> To conclude which instruction this is, I need to get at the bytes of >> that instruction... but that stays hidden in the vmm-driver. >> >> Any easy way to get this back into userspace? > > You could try 'objdump -d' on a copy of the guest's kernel to find the relevant instruction. Would that work? I'd expect things to be reloaded and shuffled around... But I'm going to take a peek at the loader code... Since Peter suggests that DFLY has a loader that is not compatible with what the bhyve loader does at the moment. And further: On 2014-05-28 6:34, Anish wrote:> >Failed to emulate instruction at 0xffffffff8096052c > Abort trap (core dumped) > > You can also analyze the coredump of bhyve > $gdb /usr/sbin/bhyve bhyve.core > > Look at vie->inst[] from one of stack frame. > > -Anish I'll give it a spin. Thanx, --WjW From owner-freebsd-virtualization@FreeBSD.ORG Fri May 30 16:44:41 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6DD7B8B for ; Fri, 30 May 2014 16:44:41 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90101290B for ; Fri, 30 May 2014 16:44:40 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 76289153A9E for ; Fri, 30 May 2014 18:44:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HoOIckkV_mq2; Fri, 30 May 2014 18:44:30 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:5cbb:e5e6:6f86:6c35] (unknown [IPv6:2001:4cb8:3:1:5cbb:e5e6:6f86:6c35]) by smtp.digiware.nl (Postfix) with ESMTP id 8168F153AA1 for ; Fri, 30 May 2014 18:44:30 +0200 (CEST) Message-ID: <5388B56D.1000501@digiware.nl> Date: Fri, 30 May 2014 18:44:29 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: virtualization@freebsd.org Subject: Bheve: Slow linux syscalls on AMD Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 16:44:42 -0000 Hi, In my quest to find why Linux is so slow on AMD I executed strace -T -c ls -aslR / > /dev/null On both a Linux running on a real CPU (just a mere Celeron): % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 25.26 0.235827 1 180706 lstat64 23.06 0.215235 1 255949 255942 getxattr 17.65 0.164726 6 29720 getdents64 13.41 0.125168 1 180698 stat64 12.75 0.119004 1 130339 130339 lgetxattr 4.28 0.039929 2 22656 86 readlink 2.08 0.019399 1 14859 openat 0.70 0.006569 0 14910 close 0.54 0.005085 0 14909 fstat64 0.15 0.001417 0 3135 write 0.10 0.000896 0 2831 clock_gettime 0.03 0.000263 3 83 35 open 0.00 0.000000 0 18 read 0.00 0.000000 0 1 execve 0.00 0.000000 0 12 12 access 0.00 0.000000 0 8 brk 0.00 0.000000 0 3 3 ioctl 0.00 0.000000 0 35 munmap 0.00 0.000000 0 12 mprotect 0.00 0.000000 0 57 _llseek 0.00 0.000000 0 67 mmap2 0.00 0.000000 0 1 set_thread_area 0.00 0.000000 0 2 2 statfs64 0.00 0.000000 0 4 socket 0.00 0.000000 0 4 4 connect ------ ----------- ----------- --------- --------- ---------------- 100.00 0.933518 851019 386423 total And on a virtualized Linux: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 55.83 30.198355 666 45331 getdents 17.99 9.733976 429 22665 5 openat 16.62 8.988507 396 22674 close 9.10 4.920301 217 22673 fstat 0.37 0.201331 223 901 write 0.03 0.015656 447 35 23 open 0.02 0.008853 328 27 mmap 0.02 0.008239 358 23 brk 0.01 0.007008 501 14 mprotect 0.01 0.003540 354 10 read 0.01 0.003146 393 8 8 access 0.00 0.001808 452 4 munmap 0.00 0.000660 660 1 mremap 0.00 0.000615 308 2 2 statfs 0.00 0.000582 194 3 3 ioctl 0.00 0.000254 254 1 execve 0.00 0.000236 236 1 stat 0.00 0.000193 193 1 arch_prctl ------ ----------- ----------- --------- --------- ---------------- 100.00 54.093260 114374 41 total One cannot really compare the factual results, but what is significat is the orders of magnitude difference in syscall time. So I ripped this from the net: Which turns out to be a BAD test, since linux caches the getpid value. ==== #include #include #include #include int foo(){ return(10); } long nanosec(struct timeval t){ /* Calculate nanoseconds in a timeval structure */ return((t.tv_sec*1000000+t.tv_usec)*1000); } main(){ int i,j,res; long N_iterations=1000000; /* A million iterations */ float avgTimeSysCall, avgTimeFuncCall; struct timeval t1, t2; /* Find average time for System call */ res=gettimeofday(&t1,NULL); assert(res==0); for (i=0;i Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57BEF687 for ; Fri, 30 May 2014 21:28:57 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id D89B72224 for ; Fri, 30 May 2014 21:28:55 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id D0E451414809 for ; Fri, 30 May 2014 23:27:01 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 926191A2E9A for ; Fri, 30 May 2014 23:27:01 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Fri, 30 May 2014 23:27:01 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Bheve: Slow linux syscalls on AMD To: freebsd-virtualization@freebsd.org References: <5388B56D.1000501@digiware.nl> Lines: 83 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 21:28:57 -0000 Hi Willem, Willem Jan Withagen wrote: > 1) I'm looking for a better basic syscall in Linux that is not cache, > faked or otherwise tweaked to nog give what I want. > Would really be nice if there was a NOP_syscall, just go in and out of > kernel space..... Hmm, I've tried your test with "getuid". Seems not to be cached. Here's the diff: =============================================================================== # diff 0.orig.c 0.c 24c24 < j=getpid(); --- > (void)getuid(); 38c38 < printf("Average time for System call getpid : %f\n",avgTimeSysCall); --- > printf("Average time for System call getuid : %f\n",avgTimeSysCall); =============================================================================== And here is the result: =============================================================================== # strace -c ./0 Average time for System call getuid : 10564.581055 Average time for Function call : 2.285000 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 100.00 0.004590 0 1000000 getuid 0.00 0.000000 0 1 read 0.00 0.000000 0 2 write 0.00 0.000000 0 2 open 0.00 0.000000 0 2 close 0.00 0.000000 0 3 fstat 0.00 0.000000 0 9 mmap 0.00 0.000000 0 3 mprotect 0.00 0.000000 0 1 munmap 0.00 0.000000 0 1 brk 0.00 0.000000 0 1 1 access 0.00 0.000000 0 1 execve 0.00 0.000000 0 1 arch_prctl ------ ----------- ----------- --------- --------- ---------------- 100.00 0.004590 1000027 1 total =============================================================================== > 3) Can somebody do the same test on an intel plaform and see what the > results are. Here is the result from a bhyved CentOS on an Intel i3: =============================================================================== # strace -c ./0.orig Average time for System call getpid : 3.776000 Average time for Function call : 2.326000 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- -nan 0.000000 0 1 read -nan 0.000000 0 2 write -nan 0.000000 0 2 open -nan 0.000000 0 2 close -nan 0.000000 0 3 fstat -nan 0.000000 0 9 mmap -nan 0.000000 0 3 mprotect -nan 0.000000 0 1 munmap -nan 0.000000 0 1 brk -nan 0.000000 0 1 1 access -nan 0.000000 0 1 getpid -nan 0.000000 0 1 execve -nan 0.000000 0 1 arch_prctl ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000000 28 1 total =============================================================================== Regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Sat May 31 00:01:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA15F8E3 for ; Sat, 31 May 2014 00:01:09 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 517592D5E for ; Sat, 31 May 2014 00:01:08 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 14B6E1534D1; Sat, 31 May 2014 02:01:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gnIMp2pK6-L5; Sat, 31 May 2014 02:01:01 +0200 (CEST) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 9E96A153434; Sat, 31 May 2014 02:01:01 +0200 (CEST) Message-ID: <53891BC1.9050701@digiware.nl> Date: Sat, 31 May 2014 02:01:05 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer , freebsd-virtualization@freebsd.org Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> In-Reply-To: <20140530212900.6B74B6BF@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 00:01:09 -0000 On 30-5-2014 23:27, Nils Beyer wrote: > Hi Willem, > > Willem Jan Withagen wrote: >> 1) I'm looking for a better basic syscall in Linux that is not cache, >> faked or otherwise tweaked to nog give what I want. >> Would really be nice if there was a NOP_syscall, just go in and out of >> kernel space..... > > Hmm, I've tried your test with "getuid". Seems not to be cached. Here's the > diff: > > =============================================================================== > # diff 0.orig.c 0.c > 24c24 > < j=getpid(); > --- >> (void)getuid(); > 38c38 > < printf("Average time for System call getpid : %f\n",avgTimeSysCall); > --- >> printf("Average time for System call getuid : %f\n",avgTimeSysCall); > =============================================================================== > > > And here is the result: > > =============================================================================== > # strace -c ./0 > Average time for System call getuid : 10564.581055 > Average time for Function call : 2.285000 > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.004590 0 1000000 getuid > 0.00 0.000000 0 1 read > 0.00 0.000000 0 2 write > 0.00 0.000000 0 2 open > 0.00 0.000000 0 2 close > 0.00 0.000000 0 3 fstat > 0.00 0.000000 0 9 mmap > 0.00 0.000000 0 3 mprotect > 0.00 0.000000 0 1 munmap > 0.00 0.000000 0 1 brk > 0.00 0.000000 0 1 1 access > 0.00 0.000000 0 1 execve > 0.00 0.000000 0 1 arch_prctl > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.004590 1000027 1 total > =============================================================================== Looks a bit like mine on i386 real hardware: root@ubuntu-i386-14:~/src/tests# strace -c ./syscall-getuid Average time for System call getuid : 272.023010 Average time for Function call : 7.169000 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 100.00 0.018939 0 1000000 getuid32 0.00 0.000000 0 1 read 0.00 0.000000 0 2 write 0.00 0.000000 0 2 open 0.00 0.000000 0 2 close 0.00 0.000000 0 1 execve 0.00 0.000000 0 3 3 access 0.00 0.000000 0 1 brk 0.00 0.000000 0 4 gettimeofday 0.00 0.000000 0 1 munmap 0.00 0.000000 0 3 mprotect 0.00 0.000000 0 7 mmap2 0.00 0.000000 0 3 fstat64 0.00 0.000000 0 1 set_thread_area ------ ----------- ----------- --------- --------- ---------------- 100.00 0.018939 1000031 3 total >> 3) Can somebody do the same test on an intel plaform and see what the >> results are. > > Here is the result from a bhyved CentOS on an Intel i3: > > =============================================================================== > # strace -c ./0.orig > Average time for System call getpid : 3.776000 > Average time for Function call : 2.326000 > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > -nan 0.000000 0 1 read > -nan 0.000000 0 2 write > -nan 0.000000 0 2 open > -nan 0.000000 0 2 close > -nan 0.000000 0 3 fstat > -nan 0.000000 0 9 mmap > -nan 0.000000 0 3 mprotect > -nan 0.000000 0 1 munmap > -nan 0.000000 0 1 brk > -nan 0.000000 0 1 1 access > -nan 0.000000 0 1 getpid > -nan 0.000000 0 1 execve > -nan 0.000000 0 1 arch_prctl > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.000000 28 1 total > =============================================================================== Right no long calls at all.... So the question remains: Why is it taking so long on the AMD platform. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sat May 31 00:22:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C86DFBA3 for ; Sat, 31 May 2014 00:22:06 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8B0FE2F35 for ; Sat, 31 May 2014 00:22:06 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 8E3B2125F7; Sat, 31 May 2014 10:13:32 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUQ40034 (AUTH peterg@ptree32.com.au); Sat, 31 May 2014 10:13:31 +1000 Message-ID: <53891EAA.8040105@freebsd.org> Date: Fri, 30 May 2014 17:13:30 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> In-Reply-To: <53891BC1.9050701@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 00:22:06 -0000 Hi Willem, > So the question remains: > Why is it taking so long on the AMD platform. The time difference looks a lot like a VM-exit roundtrip. My new AMD box is arriving shortly so I'll have a look into it. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat May 31 00:37:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55DAFF98; Sat, 31 May 2014 00:37:58 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 146B82007; Sat, 31 May 2014 00:37:58 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0BEFF1534D1; Sat, 31 May 2014 02:37:54 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hWMn0IvRHC6S; Sat, 31 May 2014 02:37:52 +0200 (CEST) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 590E6153434; Sat, 31 May 2014 02:37:52 +0200 (CEST) Message-ID: <53892464.50005@digiware.nl> Date: Sat, 31 May 2014 02:37:56 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> In-Reply-To: <53891EAA.8040105@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 00:37:58 -0000 On 31-5-2014 2:13, Peter Grehan wrote: > Hi Willem, > >> So the question remains: >> Why is it taking so long on the AMD platform. > > The time difference looks a lot like a VM-exit roundtrip. My new AMD > box is arriving shortly so I'll have a look into it. Hi Peter, I would expect something like this. The question however what kind of business is done during the roundtrip. Could it be something like a soft-interrupt 0x80 to get from userspace to the kernel? Supossedly this interface is depreciated around 2.4 I'll see if I can augment the exit-code in bhyve. Catching it with bhyvectl --get-all will probably have too much noise from running the rest of the system. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sat May 31 00:53:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4FA52887; Sat, 31 May 2014 00:53:15 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DB292194; Sat, 31 May 2014 00:53:15 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 566A81534D1; Sat, 31 May 2014 02:53:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KnxUu0waSNip; Sat, 31 May 2014 02:53:11 +0200 (CEST) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 8F104153434; Sat, 31 May 2014 02:53:11 +0200 (CEST) Message-ID: <538927FC.3000903@digiware.nl> Date: Sat, 31 May 2014 02:53:16 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> In-Reply-To: <53891EAA.8040105@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 00:53:15 -0000 On 31-5-2014 2:13, Peter Grehan wrote: > Hi Willem, > >> So the question remains: >> Why is it taking so long on the AMD platform. > > The time difference looks a lot like a VM-exit roundtrip. My new AMD > box is arriving shortly so I'll have a look into it. Before running the getuid test: vm exits due to interrupt window opening 0 vm exits due to nmi window opening 0 vm exits due to nested page fault 311094 vm exits for instruction emulation 159415683 number of vm exits for unknown reason 0 number of vm exits handled in userspace 0 number of vm exits due to exceptions 0 total number of vm exits 353767137 vm exits due to external interrupt 177525597 Then calling getuid() 10^6 times vm exits due to interrupt window opening 0 vm exits due to nmi window opening 0 vm exits due to nested page fault 311094 vm exits for instruction emulation 159589889 number of vm exits for unknown reason 0 number of vm exits handled in userspace 0 number of vm exits due to exceptions 0 total number of vm exits 354157461 vm exits due to external interrupt 177730908 And that gives a difference of: 390324 Which is not the regular amount of vm exits counted in the equal idle time. So there seems to be some relation, but it is not 1:1... --WjW From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 3 01:55:54 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A9AFD9E for ; Tue, 3 Jun 2014 01:55:54 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1AD82C26 for ; Tue, 3 Jun 2014 01:55:53 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id m5so3997096qaj.2 for ; Mon, 02 Jun 2014 18:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=61Cel0scjIfD/QVE1v2n7EBIJx6MuO9jaWzjzxvBONA=; b=C4gDbu9Ds1FCXIsIOwMdMNc+aNs6xzB+4YDfte3vdIFfuVyKhUI+ealkS6VprdY7j+ 4AK2h362YEn8qn4wblKBCZtGObglLH/5d5uTvbVMklDyO8ZfmqxNj0+cLUDag+73CfD0 r0HZlQ20+k7AcrPRRA6B0mbjjZuY1nAZ7oxp4HVaa1TiJ2o8kt2G8yDomFIOYq0wdKr6 Ue0Pvwndr8mo8TuetImeX1JCmSkQalSAqEgKP7+WYqu8bpC8Ur7I8g6sza1lxKnB/Jsa p4DplszOJfzqBbZRtATxjhuWEfPQ0PITNepwSGOeRWwOwgbLylTCPK26uPZfpAsB4oPv b5iw== MIME-Version: 1.0 X-Received: by 10.140.96.162 with SMTP id k31mr50654803qge.38.1401760553141; Mon, 02 Jun 2014 18:55:53 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Mon, 2 Jun 2014 18:55:53 -0700 (PDT) In-Reply-To: References: Date: Mon, 2 Jun 2014 18:55:53 -0700 Message-ID: Subject: Re: SR-IOV Patch Series 2/7: bhyve integration From: Neel Natu To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2014 01:55:54 -0000 Hi Ryan, On Mon, May 26, 2014 at 6:51 PM, Ryan Stone wrote: > The bhyve work to interoperate is quite simple. PCI Passthrough > through the ppt driver works just fine with SR-IOV VFs. The three > changes are: > > - allow ppt devices to detach. This would happen if the administrator > destroyed VFs with iovctl -D > - allow the SR-IOV infrastructure to force the ppt driver to attach to > certain devices by using the special BUS_PROBE_NOWILDCARD return value > from device_probe(). This will have no effect on regular devices, but > the SR-IOV infrastructure can create VF devices in a way where > BUS_PROBE_NOWILDCARD will match the device. This allows passthrough > of a given VF to be configured through iovctl (see ioctl.conf(5) for > an example of how to do this). > - Currently if vmm.ko is loaded and no ppt devices are present, the > IOMMU is not enabled. If VFs are subsequently created as passthrough > devices it will not be possible to pass them through to VMs. This > patch adds a new tunable that will force the IOMMU to be enabled. > > Note: Currently ppt suffers from a lack of locking. This is now a > problem because ppt devices can be destroyed, so there would be races > here. I tried to address this by adding a mutex but that failed in > ways that I could not figure out how to resolve (basically, some parts > of ppt are called into from within a critical section, so a spinlock > is needed, but other parts of ppt call into the VM subsystem to do > M_WAITOK allocations, so an sx lock is needed. It wasn't at all > obvious to me how to resolve that conflict, so I'm hoping that a more > experienced bhyve person has an idea). > Your changes look good. Also, yes, locking is now an issue. Let me try a couple of things and see how to fix this properly. best Neel > > http://people.freebsd.org/~rstone/patches/iov/0004-Allow-passthrough-devices-to-be-hinted.patch > [PATCH 04/21] Allow passthrough devices to be hinted. > > Allow the ppt driver to attach to devices that were hinted to be > passthrough devices by the PCI code creating them with a driver > name of "ppt". > > Add a tunable that allows the IOMMU to be forced to be used. With > SR-IOV passthrough devices the VFs may be created after vmm.ko is > loaded. The current code will not initialize the IOMMU in that > case, meaning that the passthrough devices can't actually be used. > --- > sys/amd64/vmm/io/ppt.c | 72 +++++++++++++++++++++++++++----------------------- > sys/amd64/vmm/vmm.c | 7 ++++- > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 4 00:16:59 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 733B575A for ; Wed, 4 Jun 2014 00:16:59 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C7682A89 for ; Wed, 4 Jun 2014 00:16:58 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s540GmvU095876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 4 Jun 2014 00:16:53 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s540GmZ8095875 for virtualization@FreeBSD.org; Wed, 4 Jun 2014 00:16:48 GMT (envelope-from swills) Date: Wed, 4 Jun 2014 00:16:48 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: bhyve max virtual CPUs Message-ID: <20140604001645.GA89475@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Wed, 04 Jun 2014 00:16:53 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 00:16:59 -0000 Hi, Currently it seems bhyve has a max of 16 virtual CPUs, is that right? Or am I misunderstanding something? And if so, can we get that increased to 32 at least? Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 4 00:22:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9D4E86C for ; Wed, 4 Jun 2014 00:22:33 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id A3B4C2B20 for ; Wed, 4 Jun 2014 00:22:32 +0000 (UTC) Received: from [10.1.1.1] (S01060001abad1dea.hm.shawcable.net [50.70.146.73]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id C8F087D39D for ; Wed, 4 Jun 2014 00:22:25 +0000 (UTC) Message-ID: <538E66C1.8040804@freebsd.org> Date: Tue, 03 Jun 2014 20:22:25 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: bhyve max virtual CPUs References: <20140604001645.GA89475@mouf.net> In-Reply-To: <20140604001645.GA89475@mouf.net> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KVtpGp9rSRNLrAfr8WMU00m3bSRPs7BIt" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 00:22:33 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --KVtpGp9rSRNLrAfr8WMU00m3bSRPs7BIt Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-06-03 20:16, Steve Wills wrote: > Hi, >=20 > Currently it seems bhyve has a max of 16 virtual CPUs, is that right? O= r am I > misunderstanding something? And if so, can we get that increased to 32 = at > least? >=20 > Thanks, > Steve > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 It does have that limit. I asked about increasing it and was told that it can be done by editing a value in one of the .h files or something, but the reason it wasn't higher was that it didn't scale nicely, and they wanted to reword how it works. I would definitely be interested in this, as we run video transcoding (uses a LOT of cpu) on 24 or 32 core machines, and we'd like to pass more of the cores into the VM. --=20 Allan Jude --KVtpGp9rSRNLrAfr8WMU00m3bSRPs7BIt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTjmbFAAoJEJrBFpNRJZKfQGAQAKW+q1+G5sWNJVE6L0ovEa60 /aLPSp6feZ2lgzlEsFLNprkhkKuJfABjSDzuvJkcRjlusuwH2vtetjYBpqxYCFWU SIGvidMArylLLlcdecRHj8VPAVkOcE9Oam9EpOCx9glkgwwyKxKUTEW+sp5/i2LS ZA8zNVlZkPVBZuJpJykSW0A5lgeeEMk0tgN2Z+SiQaH7UcH5i2BPUXxHIV7QfX3d leTqtWHAgUxBwLmCebEiNbMjjF/Pwn3nbR/7eQt8+rPfLVR4kJZU5tUz2pILexrK P/FlgSY3uIysDLRe1ng73DbgOmmdVF4rRn6MQr8fMqyZOXz6AYXbLCQU2yxznbmW w+MYpP0o3UJ3tA57g1LU0c/YnoCjtN/5KqeH/jK37GY/PPiv/u8xpKpXaLKRfgGI X2J+3RvY1d4JloQYdNplLMiON0FpHrPkyJ1c7MtOgXkh0ZBGlQFBgLthDG9WZgbw x/HXWCvNqssYEyTK0uEs2YmSCM33Qvef5pP7RbFCQ13kiHBp84TcLgYt99//TZ1z vn+NSaqSGRI9STvcZviLzS2MRak/wY407XAZgPvtgpqAnjWyEVTiL9xbTz/IrWOi 7cZOedFhb1qGNC2zSVcw5cHPST7V5+JJYkCj2KyUTB2NCD4SYbbl6MpvM6uw1tm1 DZlZM+r6fIoUaodjO7TP =vAZ1 -----END PGP SIGNATURE----- --KVtpGp9rSRNLrAfr8WMU00m3bSRPs7BIt-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 4 04:35:47 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD0A9D38; Wed, 4 Jun 2014 04:35:47 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7070A2074; Wed, 4 Jun 2014 04:35:46 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 0FB64124D0; Wed, 4 Jun 2014 14:35:44 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUS41352 (AUTH peterg@ptree32.com.au); Wed, 4 Jun 2014 14:35:43 +1000 Message-ID: <538EA21D.9060004@freebsd.org> Date: Tue, 03 Jun 2014 21:35:41 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Allan Jude Subject: Re: bhyve max virtual CPUs References: <20140604001645.GA89475@mouf.net> <538E66C1.8040804@freebsd.org> In-Reply-To: <538E66C1.8040804@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 04:35:47 -0000 Hi Allan, > I asked about increasing it and was told that it can be done by editing > a value in one of the .h files or something, but the reason it wasn't > higher was that it didn't scale nicely, and they wanted to reword how it > works. > > I would definitely be interested in this, as we run video transcoding > (uses a LOT of cpu) on 24 or 32 core machines, and we'd like to pass > more of the cores into the VM. You can bump it by modifying: sys/amd64/include/vmm.h:#define VM_MAXCPU 16 /* maximum virtual cpus */ ... and rebuilding world+kernel. Let us know how performance goes with larger values. It will probably be a while before this value can be dynamic, but that's the goal. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 21:24:54 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11E199CF for ; Sat, 7 Jun 2014 21:24:54 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C83712360 for ; Sat, 7 Jun 2014 21:24:53 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s57LOhMj008888 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sat, 7 Jun 2014 21:24:48 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s57LOhLh008887 for virtualization@FreeBSD.org; Sat, 7 Jun 2014 21:24:43 GMT (envelope-from swills) Date: Sat, 7 Jun 2014 21:24:43 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: interrupt storm on ahci Message-ID: <20140607212440.GB3163@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sat, 07 Jun 2014 21:24:48 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 21:24:54 -0000 Hi, I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running FreeBSD CURRENT, r266947. I've gotten this message about 12 times since boot: interrupt storm detected on "irq268:"; throttling interrupt source vmstat -i shows: irq268: ahci1 236514222 839 ahci1 is the second disk connected to the system: ahci1: mem 0xc0002400-0xc00027ff irq 18 at device 4.0 on pci0 The VM itself runs poudriere and was building a bunch of packages. At the moment, the VM seems to be in a rather odd state. The poudriere jails are running, but not doing anything. Ideas? Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 21:28:24 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD301C2A; Sat, 7 Jun 2014 21:28:24 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id A15DB238A; Sat, 7 Jun 2014 21:28:24 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 72A0012307; Sun, 8 Jun 2014 07:28:22 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUV17215 (AUTH peterg@ptree32.com.au); Sun, 8 Jun 2014 07:28:21 +1000 Message-ID: <539383F3.2060307@freebsd.org> Date: Sat, 07 Jun 2014 14:28:19 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Steve Wills Subject: Re: interrupt storm on ahci References: <20140607212440.GB3163@mouf.net> In-Reply-To: <20140607212440.GB3163@mouf.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 21:28:24 -0000 Hi Steve, > I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running > FreeBSD CURRENT, r266947. I've gotten this message about 12 times since boot: > > interrupt storm detected on "irq268:"; throttling interrupt source > > vmstat -i shows: > > irq268: ahci1 236514222 839 > > ahci1 is the second disk connected to the system: > > ahci1: mem 0xc0002400-0xc00027ff irq 18 at device 4.0 on pci0 > > The VM itself runs poudriere and was building a bunch of packages. At the > moment, the VM seems to be in a rather odd state. The poudriere jails are > running, but not doing anything. Ideas? Is this an 8.* host ? I don't believe AHCI has MSI support on that version, and AHCI legacy interrupts haven't had a huge amount of testing under load. If it is 8.*, I'd recommend using virtio-blk for the block device until we can work out what's going wrong. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 21:53:06 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B50ED2ED; Sat, 7 Jun 2014 21:53:06 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 574A62611; Sat, 7 Jun 2014 21:53:06 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id z60so7176368qgd.32 for ; Sat, 07 Jun 2014 14:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9+X3snbGpsACLP9EHOBzAelD3HaaujyOgc4gFT/MJlk=; b=HWGR4YCj8MxWQ+z8zqQ9wff8kt0sMwFMHA0L5BdmAkE/VX6L+dOUiZEFBxUuKEd6qj KIHMFSHVrgypg3l4oD0+ZE+RlUwmxGMDiVRje/pYDDL/ywnAMfH4Bx+HaAf7p7trqHG9 9Q9kR74Cfh+P6Taso+hEoKQCmMgCwVD/NDaArJW0mQ8gGFWWq+nA3tQ73MDBVTFTRvTb HQ8ZAH0BurMBQiSBJzABlJMe7Aatyr4Pbh3lqbJ4AqxC0miqCohS49NgAg4umS5nWVwO yUi8c0teFuURc9H9GyEXP4w1YnjD11LMstY439bFkrnkZUWbKqfOYAVH+2HTsxlKhi7p OgpA== MIME-Version: 1.0 X-Received: by 10.224.28.65 with SMTP id l1mr20894605qac.81.1402177985397; Sat, 07 Jun 2014 14:53:05 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Sat, 7 Jun 2014 14:53:05 -0700 (PDT) In-Reply-To: <539383F3.2060307@freebsd.org> References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> Date: Sat, 7 Jun 2014 14:53:05 -0700 Message-ID: Subject: Re: interrupt storm on ahci From: Neel Natu To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 21:53:06 -0000 Hi Steve, On Sat, Jun 7, 2014 at 2:28 PM, Peter Grehan wrote: > Hi Steve, > > >> I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running >> FreeBSD CURRENT, r266947. I've gotten this message about 12 times since >> boot: >> >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> vmstat -i shows: >> >> irq268: ahci1 236514222 839 >> >> ahci1 is the second disk connected to the system: >> >> ahci1: mem 0xc0002400-0xc00027ff irq 18 >> at device 4.0 on pci0 >> >> The VM itself runs poudriere and was building a bunch of packages. At the >> moment, the VM seems to be in a rather odd state. The poudriere jails are >> running, but not doing anything. Ideas? > > > Is this an 8.* host ? I don't believe AHCI has MSI support on that version, > and AHCI legacy interrupts haven't had a huge amount of testing under load. > > If it is 8.*, I'd recommend using virtio-blk for the block device until we > can work out what's going wrong. > The KTR trace would be useful to figure out what's happening. To do that you can compile the host kernel and vmm.ko with the following options: options KTR options KTR_MASK=(KTR_GEN) options KTR_ENTRIES=(4*1024*1024) And when you see the interrupt storm message in the guest you can execute: sudo ktrdump -cto /tmp/ktrdump.out best Neel > later, > > Peter. > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 21:54:13 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33867357 for ; Sat, 7 Jun 2014 21:54:13 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 10DF12629 for ; Sat, 7 Jun 2014 21:54:12 +0000 (UTC) Received: from [10.1.1.1] (S01060001abad1dea.hm.shawcable.net [50.70.146.73]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 686757EB73 for ; Sat, 7 Jun 2014 21:54:11 +0000 (UTC) Message-ID: <53938A0C.5000603@freebsd.org> Date: Sat, 07 Jun 2014 17:54:20 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: interrupt storm on ahci References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> In-Reply-To: <539383F3.2060307@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 21:54:13 -0000 On 2014-06-07 17:28, Peter Grehan wrote: > Hi Steve, > >> I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running >> FreeBSD CURRENT, r266947. I've gotten this message about 12 times >> since boot: >> >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> vmstat -i shows: >> >> irq268: ahci1 236514222 839 >> >> ahci1 is the second disk connected to the system: >> >> ahci1: mem 0xc0002400-0xc00027ff irq >> 18 at device 4.0 on pci0 >> >> The VM itself runs poudriere and was building a bunch of packages. At the >> moment, the VM seems to be in a rather odd state. The poudriere jails are >> running, but not doing anything. Ideas? > > Is this an 8.* host ? I don't believe AHCI has MSI support on that > version, and AHCI legacy interrupts haven't had a huge amount of testing > under load. > > If it is 8.*, I'd recommend using virtio-blk for the block device until > we can work out what's going wrong. > > later, > > Peter. > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" He mentioned that both the host and guest are recent -CURRENT -- Allan Jude From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 22:01:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31C8642C; Sat, 7 Jun 2014 22:01:26 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id E85A726E8; Sat, 7 Jun 2014 22:01:25 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id DDA4C12307; Sun, 8 Jun 2014 08:01:23 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUV17554 (AUTH peterg@ptree32.com.au); Sun, 8 Jun 2014 08:01:23 +1000 Message-ID: <53938BB1.6060905@freebsd.org> Date: Sat, 07 Jun 2014 15:01:21 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Allan Jude Subject: Re: interrupt storm on ahci References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> <53938A0C.5000603@freebsd.org> In-Reply-To: <53938A0C.5000603@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 22:01:26 -0000 > He mentioned that both the host and guest are recent -CURRENT Doh, was fixated on seeing "irq" in the dmesg :) later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 22:09:29 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03CC9805; Sat, 7 Jun 2014 22:09:29 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D53F273A; Sat, 7 Jun 2014 22:09:28 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s57M9JrQ009439 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 7 Jun 2014 22:09:24 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s57M9Js1009438; Sat, 7 Jun 2014 22:09:19 GMT (envelope-from swills) Date: Sat, 7 Jun 2014 22:09:19 +0000 From: Steve Wills To: Neel Natu Subject: Re: interrupt storm on ahci Message-ID: <20140607220916.GC3163@mouf.net> References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sat, 07 Jun 2014 22:09:24 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 22:09:29 -0000 On Sat, Jun 07, 2014 at 02:53:05PM -0700, Neel Natu wrote: > Hi Steve, > > On Sat, Jun 7, 2014 at 2:28 PM, Peter Grehan wrote: > > Hi Steve, > > > > > >> I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running > >> FreeBSD CURRENT, r266947. I've gotten this message about 12 times since > >> boot: > >> > >> interrupt storm detected on "irq268:"; throttling interrupt source > >> > >> vmstat -i shows: > >> > >> irq268: ahci1 236514222 839 > >> > >> ahci1 is the second disk connected to the system: > >> > >> ahci1: mem 0xc0002400-0xc00027ff irq 18 > >> at device 4.0 on pci0 > >> > >> The VM itself runs poudriere and was building a bunch of packages. At the > >> moment, the VM seems to be in a rather odd state. The poudriere jails are > >> running, but not doing anything. Ideas? > > > > > > Is this an 8.* host ? I don't believe AHCI has MSI support on that version, > > and AHCI legacy interrupts haven't had a huge amount of testing under load. > > > > If it is 8.*, I'd recommend using virtio-blk for the block device until we > > can work out what's going wrong. > > Both the host and guest are running FreeBSD CURRENT r266947. > The KTR trace would be useful to figure out what's happening. > > To do that you can compile the host kernel and vmm.ko with the > following options: > options KTR > options KTR_MASK=(KTR_GEN) > options KTR_ENTRIES=(4*1024*1024) > > And when you see the interrupt storm message in the guest you can execute: > sudo ktrdump -cto /tmp/ktrdump.out Ok, I'll work on getting that added to the host. Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 7 23:06:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D1381C5 for ; Sat, 7 Jun 2014 23:06:16 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id F32D72B50 for ; Sat, 7 Jun 2014 23:06:15 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id A0E87124BD; Sun, 8 Jun 2014 09:06:13 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUV18468 (AUTH peterg@ptree32.com.au); Sun, 8 Jun 2014 09:06:12 +1000 Message-ID: <53939AE2.80804@freebsd.org> Date: Sat, 07 Jun 2014 16:06:10 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> In-Reply-To: <538927FC.3000903@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 23:06:16 -0000 >>> So the question remains: >>> Why is it taking so long on the AMD platform. I believe this is now fixed with r267217 - please test out and let us know how it goes. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 02:45:44 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AA4D2A3 for ; Sun, 8 Jun 2014 02:45:44 +0000 (UTC) Received: from mail-qa0-x233.google.com (mail-qa0-x233.google.com [IPv6:2607:f8b0:400d:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE05D29C6 for ; Sun, 8 Jun 2014 02:45:43 +0000 (UTC) Received: by mail-qa0-f51.google.com with SMTP id w8so6347682qac.24 for ; Sat, 07 Jun 2014 19:45:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=clintarmstrong.net; s=google; h=mime-version:date:message-id:subject:from:to:content-type; bh=CINXjsvSMAN9VOPbliQDt+2zCxhOMGOeVFNc3oKpFfw=; b=HJzAzmgFMqHoew5D77m6iPpp3r6kTPHh6Km6u0VIc2e9G+nNPcNu9ddfiW/ak4Q/lW rv6w2w8vBoa8fVXdM6SwoK85UQ+BKe08RqSiNPcix3F2BkoI4UXunYsKV4vCmMTMKZes gMK2Pq72WtcT0nMJan1gd4nQsyqj19FyRali0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=CINXjsvSMAN9VOPbliQDt+2zCxhOMGOeVFNc3oKpFfw=; b=Bbwb9IUDqAKYfdk28BwW9D4hXfg69t6dDgLjt+599z7E0beLUMqfzrbU66SkUYzgOo yUX5h+10ZBvWlEiUplvp0jDDSn/y7Fd1cRrKgu8AaxGlDV0df6EQXyHZOhs8oXLvDNK2 vPQMyBx9H43nUiAL/jUcDVO3CU0UbK/WJeO95YJLdbFIscxo2GpEMirJOx9pMvIsUDvh EtuXM7aejcIe52CQ8fj505L9U8/4Klz0BZqa28iWJSHg4ttduQ0O7gN9QXBGpCajmgWB h8e3eguTXRbe6hn56MitWq9fXCl+lbGDrJpn9u8hsbtXZkUgNqrV1ZLGWZkzFLma0vrf cRJQ== X-Gm-Message-State: ALoCoQk3/AjduNSotVTEBEpGru3z9LYL4d+6a199YWCGGQOk8mxDBbt3LRejw1OboQ3kLMPuGskG MIME-Version: 1.0 X-Received: by 10.224.47.77 with SMTP id m13mr22261075qaf.69.1402195542429; Sat, 07 Jun 2014 19:45:42 -0700 (PDT) Received: by 10.140.96.107 with HTTP; Sat, 7 Jun 2014 19:45:42 -0700 (PDT) X-Originating-IP: [2601:4:2a80:2d:75ec:7257:1e6a:70f5] Date: Sat, 7 Jun 2014 22:45:42 -0400 Message-ID: Subject: AMD passthru on bhyve_svm branch From: Clint Armstrong To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 02:45:44 -0000 I'm not sure if this is supposed to be fully functional yet, but it appears to be partially working so there may be a fix for my issue. I'm running bhyve_svm r267218. One reason to think it's not supposed to be working yet is I still get this logged during boot: # dmesg | grep iommu amd_iommu_init: not implemented But trying it anyway it appears to almost work. The device shows up during the boot process, but it hangs 20-30 seconds before giving "polling failed". I'm having trouble finding out if this is supposed to work yet or if AMD passthru is not yet fully implemented. Thanks. # pciconf -vl | grep -A 4 pci0:1:0:0 ppt0@pci0:1:0:0: class=0x010700 card=0x30201000 chip=0x00721000 rev=0x03 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]' class = mass storage subclass = SAS # grep pptdevs /boot/loader.conf pptdevs="1/0/0" # bhyveload -m 4G -d FreeBSD-10.0-RELEASE-amd64-bootonly.iso -c stdio vm [...] # bhyve -c 2 -m 4G -A -H -P -s 0:0,hostbridge -s 1:0,passthru,1/0/0 -s 2:0,lpc -s 3:0,virtio-blk,FreeBSD-10.0-RELEASE-amd64-bootonly.iso -l com1,stdio vm [...] mps0: port 0x2000-0x20ff mem 0xc0000000-0xc0003fff,0xc0040000-0xc007ffff irq 18 at device 1.0 on pci0 mps0: Firmware: 15.00.00.00, Driver: 16.00.00.00-fbsd mps0: IOCCapabilities: 1285c mps0: polling failed mps0: Calling Reinit from mps_request_polled mps0: Reinitializing controller, Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x8 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff805d7ed7 stack pointer = 0x28:0xffffffff817f6260 frame pointer = 0x28:0xffffffff817f62a0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (swapper) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff808e7dd0 at kdb_backtrace+0x60 #1 0xffffffff808af8b5 at panic+0x155 #2 0xffffffff80c8e692 at trap_fatal+0x3a2 #3 0xffffffff80c8e969 at trap_pfault+0x2c9 #4 0xffffffff80c8e0f6 at trap+0x5e6 #5 0xffffffff80c75392 at calltrap+0x8 #6 0xffffffff805cb9a9 at mps_reinit+0xf9 #7 0xffffffff805cefbb at mps_request_polled+0xab #8 0xffffffff805ced43 at mps_update_events+0x133 #9 0xffffffff805cd3a3 at mps_iocfacts_allocate+0x1383 #10 0xffffffff805cd76c at mps_attach+0x28c #11 0xffffffff805d5d36 at mps_pci_attach+0x196 #12 0xffffffff808df242 at device_attach+0x3a2 #13 0xffffffff808e031d at bus_generic_attach+0x2d #14 0xffffffff8035e1ff at acpi_pci_attach+0x15f #15 0xffffffff808df242 at device_attach+0x3a2 #16 0xffffffff808e031d at bus_generic_attach+0x2d #17 0xffffffff8036024d at acpi_pcib_attach+0x23d Uptime: 1s From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 02:51:11 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65827481 for ; Sun, 8 Jun 2014 02:51:11 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 288952A52 for ; Sun, 8 Jun 2014 02:51:10 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 75BE31247B; Sun, 8 Jun 2014 12:51:02 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUV21720 (AUTH peterg@ptree32.com.au); Sun, 8 Jun 2014 12:51:01 +1000 Message-ID: <5393CF93.2050807@freebsd.org> Date: Sat, 07 Jun 2014 19:50:59 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Clint Armstrong Subject: Re: AMD passthru on bhyve_svm branch References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 02:51:11 -0000 Hi Clint, > I'm not sure if this is supposed to be fully functional yet ... > # dmesg | grep iommu > amd_iommu_init: not implemented The IOMMU isn't yet functional on AMD. Looks like a bug in bhyve that it doesn't error out if the IOMMU isn't located and ppt devices have been configured. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 10:02:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C315C6F0; Sun, 8 Jun 2014 10:02:42 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8033425EE; Sun, 8 Jun 2014 10:02:42 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 2E464153448; Sun, 8 Jun 2014 12:02:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h--U8a3VuQyf; Sun, 8 Jun 2014 12:02:31 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:1e7:1905:8430:9984] (unknown [IPv6:2001:4cb8:3:1:1e7:1905:8430:9984]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 937C7153434; Sun, 8 Jun 2014 12:02:31 +0200 (CEST) Message-ID: <539434B6.6020804@digiware.nl> Date: Sun, 08 Jun 2014 12:02:30 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> In-Reply-To: <53939AE2.80804@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 10:02:42 -0000 On 8-6-2014 1:06, Peter Grehan wrote: >>>> So the question remains: >>>> Why is it taking so long on the AMD platform. > > I believe this is now fixed with r267217 - please test out and let us > know how it goes. On Head or on bhyve_svm At the moment I do not have a working bhyve setup, tried to merge too much from head in my own branch... So I'm still figuring out which one is the culprit I'll try and look at the diff. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 15:28:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33682FE8; Sun, 8 Jun 2014 15:28:00 +0000 (UTC) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 875BC2F18; Sun, 8 Jun 2014 15:27:59 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id r20so3109821wiv.7 for ; Sun, 08 Jun 2014 08:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8uy+s6WU5luXMc6zqjUgLpYfkCVT3bMWPNWAoYGvj+E=; b=0jWsl0r1hQrclNiMHlXEE0z3hTVAf7yNtvWNGia1cBgWsBukxrm95lm+IDedf3GqVa x6OIVvp2Btzc/TTu/fF9Q7aivnhm07XfF/0GjNYlHYVRNbIq4nGs9ojtv21DH1XfZIIw 5Cvqd3QQFZbiZK09ef5O0YIzomYDX/9amYobnFFH0LHmDYwyZw6CuHZNukO5yDRH3fvW n0fHI3jqJHxmB60c0+H8sC9M7YR9MrhVimv4HjekMvLVcN8/rT11/XTR+KmIPezfRHC0 1QBymH7aCRolljjJAZFb4QynULpZvJQ6++aThvRqocI/r3GtHLKFpJoBDD6nmLmPsJ1z WOMw== MIME-Version: 1.0 X-Received: by 10.194.84.208 with SMTP id b16mr23788660wjz.55.1402241277639; Sun, 08 Jun 2014 08:27:57 -0700 (PDT) Received: by 10.194.71.113 with HTTP; Sun, 8 Jun 2014 08:27:57 -0700 (PDT) In-Reply-To: <539434B6.6020804@digiware.nl> References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> Date: Sun, 8 Jun 2014 08:27:57 -0700 Message-ID: Subject: Re: Bheve: Slow linux syscalls on AMD From: Anish To: Willem Jan Withagen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 15:28:00 -0000 Peter submitted this in bhyve_svm for SVM, Intel VT-x already has this fix, see vmcs.c file in any branch. http://svnweb.freebsd.org/base/projects/bhyve_svm/sys/amd64/vmm/amd/vmcb.c?r1=267217&r2=267216&pathrev=267217 Thanks and regards, Anish On Sun, Jun 8, 2014 at 3:02 AM, Willem Jan Withagen wrote: > On 8-6-2014 1:06, Peter Grehan wrote: > >>>> So the question remains: > >>>> Why is it taking so long on the AMD platform. > > > > I believe this is now fixed with r267217 - please test out and let us > > know how it goes. > > On Head > or on bhyve_svm > > At the moment I do not have a working bhyve setup, tried to merge too > much from head in my own branch... So I'm still figuring out which one > is the culprit > > I'll try and look at the diff. > > --WjW > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 15:30:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FC1EF8; Sun, 8 Jun 2014 15:30:15 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE2612F2B; Sun, 8 Jun 2014 15:30:14 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0DA28153448; Sun, 8 Jun 2014 17:30:11 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ll5CNlrZVS97; Sun, 8 Jun 2014 17:30:08 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:1e7:1905:8430:9984] (unknown [IPv6:2001:4cb8:3:1:1e7:1905:8430:9984]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 970F7153434; Sun, 8 Jun 2014 17:30:08 +0200 (CEST) Message-ID: <5394817F.4030203@digiware.nl> Date: Sun, 08 Jun 2014 17:30:07 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 15:30:15 -0000 On 8-6-2014 17:27, Anish wrote: > Peter submitted this in bhyve_svm for SVM, Intel VT-x already has this > fix, see vmcs.c file in any branch. > > http://svnweb.freebsd.org/base/projects/bhyve_svm/sys/amd64/vmm/amd/vmcb.c?r1=267217&r2=267216&pathrev=267217 Downloaded bhyve_svm again and figured out as much as this. Working to recompile everything but the kitchen sink --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 16:22:11 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6430FA5 for ; Sun, 8 Jun 2014 16:22:11 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 60E9F238A for ; Sun, 8 Jun 2014 16:22:10 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id E4EA3153448; Sun, 8 Jun 2014 18:22:08 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3iQGRFjCATNr; Sun, 8 Jun 2014 18:22:07 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a5f2:f79a:1de:fdad] (unknown [IPv6:2001:4cb8:3:1:a5f2:f79a:1de:fdad]) by smtp.digiware.nl (Postfix) with ESMTP id 0F47A153434; Sun, 8 Jun 2014 18:22:07 +0200 (CEST) Message-ID: <53948DA5.4040000@digiware.nl> Date: Sun, 08 Jun 2014 18:21:57 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Anish Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> In-Reply-To: <5394817F.4030203@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 16:22:11 -0000 On 2014-06-08 17:30, Willem Jan Withagen wrote: > On 8-6-2014 17:27, Anish wrote: >> Peter submitted this in bhyve_svm for SVM, Intel VT-x already has this >> fix, see vmcs.c file in any branch. >> >> http://svnweb.freebsd.org/base/projects/bhyve_svm/sys/amd64/vmm/amd/vmcb.c?r1=267217&r2=267216&pathrev=267217 > > Downloaded bhyve_svm again and figured out as much as this. > Working to recompile everything but the kitchen sink Right this seems to have fixed the performance problem. At least, compiling kernel file is actually making nice progres. Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, but that is another minor challenge. Thanx Peter --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 16:23:46 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8121FA7 for ; Sun, 8 Jun 2014 16:23:46 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3E0C12394 for ; Sun, 8 Jun 2014 16:23:45 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 4988C1235C; Mon, 9 Jun 2014 02:23:43 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUV30354 (AUTH peterg@ptree32.com.au); Mon, 9 Jun 2014 02:23:42 +1000 Message-ID: <53948E1C.2020408@freebsd.org> Date: Sun, 08 Jun 2014 09:23:56 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> In-Reply-To: <53948DA5.4040000@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 16:23:46 -0000 > Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, > but that is another minor challenge. I know what that one is: should have a fix shortly. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 8 20:15:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96C577D0; Sun, 8 Jun 2014 20:15:29 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50E6E251E; Sun, 8 Jun 2014 20:15:29 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 61D9E153448; Sun, 8 Jun 2014 22:15:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qhYtoMbxYbtC; Sun, 8 Jun 2014 22:15:23 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:7193:98fc:d209:62d3] (unknown [IPv6:2001:4cb8:3:1:7193:98fc:d209:62d3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 8D8AB153434; Sun, 8 Jun 2014 22:15:23 +0200 (CEST) Message-ID: <5394C45B.2030608@digiware.nl> Date: Sun, 08 Jun 2014 22:15:23 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53948E1C.2020408@freebsd.org> In-Reply-To: <53948E1C.2020408@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 20:15:29 -0000 On 8-6-2014 18:23, Peter Grehan wrote: >> Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, >> but that is another minor challenge. > > I know what that one is: should have a fix shortly. I've done some of the syscall tests, like before. getuid(2) is now down to 350 nsec... Running the same on the bare metal with FreeBSD gives 140 nsec.... So I can't be complaining.... Comparing umask(2) is more the same: 151 nsec versus VM linux: 364 nsec --WjW From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 19:46:17 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6622B292; Mon, 9 Jun 2014 19:46:17 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 894952B21; Mon, 9 Jun 2014 19:46:16 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id pn19so3343901lab.20 for ; Mon, 09 Jun 2014 12:46:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=94LvD3hcrk5cJg9dZRflx7B7TBxes8td1yvCgpxfg64=; b=haGtG7nkVWhvkB2Zetsummg+w4IZ7lgmJrTEP3WxkHMkBcFOn2lWq9RS3nMWjIk0rf fRbGhq+K06SxDZ04O3KXe0zQiE0RQJy+FpznapXgOUJ5Fjfg7+tw6DmeC64M+M9fw9kd chyzi89J67BeHIqVmFlHJopyTwasmuY9U4CjcU1znuhX9BawY7gNEdRqDZeOSJowT6EE RsVQdQFcPUBCyZ++LmZGBzJ65pf1POBa6R3+FomPGt7roxzIU/i3QJG2rqK5j/1tpeD+ Yty9PSPUiVAFLUjlsR766tj/NM/SSEHoa+qhGvAXv0KU4se6DCYKjGZmOqzJ98kCjx40 LUxg== MIME-Version: 1.0 X-Received: by 10.112.218.74 with SMTP id pe10mr18283328lbc.3.1402343174393; Mon, 09 Jun 2014 12:46:14 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Mon, 9 Jun 2014 12:46:14 -0700 (PDT) Date: Mon, 9 Jun 2014 12:46:14 -0700 X-Google-Sender-Auth: pFoCcppc8T-MHIhv2oKtrUlqw-M Message-ID: Subject: Enabling bhyve support in devel/libvirt port From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: Jason Helfman X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 19:46:17 -0000 Hi, For anyone interested in playing with libvirt and bhyve, I submitted these two patches to the devel/libvirt port: -> "devel/libvirt: enable BHYVE by default on FreeBSD 10" https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190767 -> "devel/libvirt: configure checks for bhyve are broken " https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190783 You need to manually rebuild the libvirt port. The prebuilt libvirt port available via "pkg install libvirt" has bhyve support disabled by default. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 21:04:12 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD0B7BAE for ; Mon, 9 Jun 2014 21:04:12 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6C2712279 for ; Mon, 9 Jun 2014 21:04:12 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 3DC18125E1; Tue, 10 Jun 2014 07:04:05 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUW22823 (AUTH peterg@ptree32.com.au); Tue, 10 Jun 2014 07:04:03 +1000 Message-ID: <53962141.8070702@freebsd.org> Date: Mon, 09 Jun 2014 14:04:01 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> In-Reply-To: <53948DA5.4040000@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 21:04:12 -0000 > Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, > but that is another minor challenge. Fixed in r267305 later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 05:40:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70D55E20; Tue, 10 Jun 2014 05:40:02 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 964602A3B; Tue, 10 Jun 2014 05:40:01 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so3660945lan.9 for ; Mon, 09 Jun 2014 22:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=eKFRhzc9a7+eZZfxvSNsujNiT1HZvo1EXq+YZi6NblQ=; b=wDJOWcZHhYyxXNMMwqkwi+2Ffyzv2DT9KZh48Y8W/EqNhYNXfwzX7rH9560Qjx1rOq jekwijF9JFfHtQBIKTUP26CqwPPU47ShEAOKgpxoVXfKl1gtjLqLbpiqWs8KknLDAnOv 93/7dasIn/cIhm81/ptUjxJnUbSViKTHrg3i+A9BCuuhw2rmWl68GmNWUBzEdm2wU71N k19cg7qZH+dolOKEP28DdlFZF4KhfWJ6ddpZ8q7DVRhFwJng/DgzCoeJ5QD9Sqsd5t// ePfb8RO7T/nm1ceabb87qDjUKtpvigyzVqskfzEdp1NyDWqpJiSks6epZ7ezb7dsjhsw TaqA== MIME-Version: 1.0 X-Received: by 10.153.5.37 with SMTP id cj5mr542032lad.48.1402378799580; Mon, 09 Jun 2014 22:39:59 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Mon, 9 Jun 2014 22:39:59 -0700 (PDT) Date: Mon, 9 Jun 2014 22:39:59 -0700 X-Google-Sender-Auth: Z42-S537V34VLd4BOGoGcInKyn0 Message-ID: Subject: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy , Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 05:40:02 -0000 Roman, I have been reading your blog posts on using libvirt + bhyve here: http://empt1e.blogspot.com/ and the libvirt bhyve documentation here: http://libvirt.org/drvbhyve.html but am having problems reproducing the steps. I have a machine running: FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #6 r266934M: Sat May 31 20:24:37 PDT 2014 root@crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 I applied the following patches to the devel/libvirt port to fix compilation problems for bhyve support: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002586.html I then did the following: BUILD AND CONFIGURE =================== cd /usr/ports/devel/libport make config (I made sure bhyve and QEMU support were enabled make make install LIBVIRT CONFIG ============== I created the following /tmp/bhyve.xml file: bhyve df3be7e7-a104-11e3-aeb0-50e5492bd3dc 219136 219136 1 hvm RUNNING ======== I did the following as root: service libvirtd restart virsh -c 'bhyve:///system' Then I ran these commands from the virsh prompt: virsh # define /tmp/bhyve.xml Domain bhyve defined from /tmp/bhyve.xml virsh # start bhyve error: Failed to start domain bhyve error: End of file while reading data: Input/output error error: Failed to reconnect to the hypervisor At this point libvirtd process crashed with a /libvirtd.core process , but for some reason I could not get a valid stacktrace. Can you provide assistance to get this working? Ultimately, I want to get the Jenkins libvirt plugin ( https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin ) to work. Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 09:36:21 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F36EB4C; Tue, 10 Jun 2014 09:36:21 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3AD2C2E87; Tue, 10 Jun 2014 09:36:21 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id E984A153448; Tue, 10 Jun 2014 11:36:16 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mQ6_jiQtzkbH; Tue, 10 Jun 2014 11:36:15 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a5f2:f79a:1de:fdad] (unknown [IPv6:2001:4cb8:3:1:a5f2:f79a:1de:fdad]) by smtp.digiware.nl (Postfix) with ESMTP id 38F89153434; Tue, 10 Jun 2014 11:36:15 +0200 (CEST) Message-ID: <5396D18C.8000401@digiware.nl> Date: Tue, 10 Jun 2014 11:36:12 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> In-Reply-To: <53962141.8070702@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 09:36:21 -0000 On 2014-06-09 23:04, Peter Grehan wrote: >> Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, >> but that is another minor challenge. > > Fixed in r267305 Ack. There is still a small difference in behaviour between Linux and FreeBSD. But I'd call that negectable... PID SIZE STATE C TIME WCPU COMMAND 70250 2075M vmidle 5 0:17 2.69% bhyve: ubuntu-14.04 (bhyve){vcpu 1} 70250 2075M vmidle 2 0:22 1.37% bhyve: ubuntu-14.04 (bhyve){vcpu 0} 70296 2075M vmidle 1 0:04 0.00% bhyve: freebsd-10 (bhyve){vcpu 0} 70296 2075M vmidle 4 0:02 0.00% bhyve: freebsd-10 (bhyve){vcpu 1} Inside the VMs the perception is totally the other way around when just running top and system processes: ubuntu14: load average: 0.01, 0.02, 0.02 FreeBSD: load averages: 0.11, 0.18, 0.09 So IMHO things get lost tn the margins of (incorrect) measurement. Next thing to check is how older ubuntu's work. Espacially those with TLS and a 2.6 kernel. ( I think there is still one release not EOL) And then start tests for CentOS. Thanx, --WjW From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 10:32:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4F966AC for ; Tue, 10 Jun 2014 10:32:29 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id A15F723D6 for ; Tue, 10 Jun 2014 10:32:28 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id 50BB71414945 for ; Tue, 10 Jun 2014 12:26:58 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 2E4FF1A68E4 for ; Tue, 10 Jun 2014 12:26:59 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Tue, 10 Jun 2014 12:26:58 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Bheve: Slow linux syscalls on AMD To: freebsd-virtualization@freebsd.org References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> Lines: 19 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 10:32:30 -0000 Hi Peter, Peter Grehan wrote: >> Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, >> but that is another minor challenge. > > Fixed in r267305 Confirmed. Running a bhyved 3-vCPU-"CentOS 6.5", the host CPU load for "vcpu 0" is around 12% now. The remaining vcpus are all near or at zero load. Ping times to the VM are fluctuating - ranging from 0.185ms to 35ms. "iperf" throughput tests results around 700Mbit/s though. Now it's time to scrap all ESXi hosts here... ;-) Thanks a lot to you guys and regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 11:43:52 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50050CB8; Tue, 10 Jun 2014 11:43:52 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 71E032A9B; Tue, 10 Jun 2014 11:43:51 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id l4so3845599lbv.17 for ; Tue, 10 Jun 2014 04:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=q1K2dN0s2PNQSLofvWRY8FFgz38C/BLjjopc3TQBddI=; b=wovUoA9Wu9UuPxyHTRCMaIeokA7+Cn0FgGQlXY3x8acyMg5lFUtSevzxgiRMq/1qzx 0Lx52MxankEo9y33uPjAUtkXg/uYWlkVcdYybkfRiApSUMZD7J2B1hq8IYbXfCS5VC2Z Z8R1pj00wgp8KWWiSazwt09Mc/qmR8lMFcNxnyA+VnW7UFtAi4yeybhlooO8EPe5Yo0l fl2VdDgOz3g8Xbl9xQbDJgmtsBoYQO/tPckKc9xASOSJM+Rpp7b2f6WWZSrXnfdb3MKW IGBbRE189/BkARPDaEj/NCzEs3tYCAxIZr+InLBooanl65YbhF9ukammzVdMxfVToq9f hLbQ== X-Received: by 10.152.44.170 with SMTP id f10mr17291lam.87.1402400629311; Tue, 10 Jun 2014 04:43:49 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id b6sm18446425laa.20.2014.06.10.04.43.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 04:43:48 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 15:43:43 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610114342.GA11194@dev.san.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 11:43:52 -0000 Craig Rodrigues wrote: > Roman, >=20 > I have been reading your blog posts on using libvirt + bhyve > here: >=20 > http://empt1e.blogspot.com/ >=20 > and the libvirt bhyve documentation here: > http://libvirt.org/drvbhyve.html >=20 > but am having problems reproducing the steps. >=20 > I have a machine running: > FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #6 r266934M: Sat May 31 > 20:24:37 PDT 2014 > root@crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 >=20 >=20 > I applied the following patches to the devel/libvirt port to fix > compilation problems for bhyve support: >=20 > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/00258= 6.html >=20 > I then did the following: >=20 > BUILD AND CONFIGURE > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > cd /usr/ports/devel/libport > make config > (I made sure bhyve and QEMU support were enabled > make > make install >=20 >=20 > LIBVIRT CONFIG > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > I created the following /tmp/bhyve.xml file: >=20 >=20 > > bhyve > df3be7e7-a104-11e3-aeb0-50e5492bd3dc > 219136 > 219136 > 1 > > hvm > > > > > > > > > > > > > > > > > > >=20 >=20 > RUNNING > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > I did the following as root: >=20 > service libvirtd restart > virsh -c 'bhyve:///system' >=20 > Then I ran these commands from the virsh prompt: >=20 > virsh # define /tmp/bhyve.xml > Domain bhyve defined from /tmp/bhyve.xml >=20 > virsh # start bhyve > error: Failed to start domain bhyve > error: End of file while reading data: Input/output error > error: Failed to reconnect to the hypervisor >=20 >=20 > At this point libvirtd process crashed with a /libvirtd.core process , > but for some reason I could not get a valid stacktrace. >=20 > Can you provide assistance to get this working? >=20 > Ultimately, I want to get the Jenkins libvirt plugin ( > https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin ) > to work. >=20 > Thanks. Hi Craig, It's a little unfortunate that there's no stacktrace, though we can try to see what's going on without it. Could you please edit libvirtd.conf (should be something like /usr/local/etc/libvirtd.conf if installed from port) to add: log_level =3D 2 then run libvirtd by hand in foreground: # libvirtd=20 and then try to start the domain again and post a couple of lines it prints before crashing? Thanks, Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 13:59:54 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4FAF416 for ; Tue, 10 Jun 2014 13:59:54 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7351C290C for ; Tue, 10 Jun 2014 13:59:54 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 39DF81260C; Tue, 10 Jun 2014 23:59:52 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUW78341 (AUTH peterg@ptree32.com.au); Tue, 10 Jun 2014 23:59:50 +1000 Message-ID: <53970F5A.8060900@freebsd.org> Date: Tue, 10 Jun 2014 06:59:54 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <5396D18C.8000401@digiware.nl> In-Reply-To: <5396D18C.8000401@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 13:59:54 -0000 Hi Willem, > On 2014-06-09 23:04, Peter Grehan wrote: >>> Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, >>> but that is another minor challenge. >> >> Fixed in r267305 > > Ack. > > There is still a small difference in behaviour between Linux and > FreeBSD. But I'd call that negectable... > > > PID SIZE STATE C TIME WCPU COMMAND > 70250 2075M vmidle 5 0:17 2.69% bhyve: ubuntu-14.04 (bhyve){vcpu 1} > 70250 2075M vmidle 2 0:22 1.37% bhyve: ubuntu-14.04 (bhyve){vcpu 0} Thanks - saw similar behaviour on my tiny Sempron but wasn't sure if it was the slow CPU/clock aliasing etc. Time to bring out KTR and see what's going on. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 14:55:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2367CAD4; Tue, 10 Jun 2014 14:55:06 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D1C5E2EE5; Tue, 10 Jun 2014 14:55:05 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 5B77E153A9E; Tue, 10 Jun 2014 16:54:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DNyzxdVsF63j; Tue, 10 Jun 2014 16:54:53 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372] (unknown [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372]) by smtp.digiware.nl (Postfix) with ESMTP id E2A1A1534D1; Tue, 10 Jun 2014 16:54:53 +0200 (CEST) Message-ID: <53971C3A.30406@digiware.nl> Date: Tue, 10 Jun 2014 16:54:50 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <5396D18C.8000401@digiware.nl> <53970F5A.8060900@freebsd.org> In-Reply-To: <53970F5A.8060900@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 14:55:06 -0000 On 2014-06-10 15:59, Peter Grehan wrote: > Hi Willem, > >> On 2014-06-09 23:04, Peter Grehan wrote: >>>> Still seeing that a 2 CPU VM is using about 100% of 1 cpu when idleing, >>>> but that is another minor challenge. >>> >>> Fixed in r267305 >> >> Ack. >> >> There is still a small difference in behaviour between Linux and >> FreeBSD. But I'd call that negectable... >> >> >> PID SIZE STATE C TIME WCPU COMMAND >> 70250 2075M vmidle 5 0:17 2.69% bhyve: ubuntu-14.04 (bhyve){vcpu 1} >> 70250 2075M vmidle 2 0:22 1.37% bhyve: ubuntu-14.04 (bhyve){vcpu 0} > > Thanks - saw similar behaviour on my tiny Sempron but wasn't sure if > it was the slow CPU/clock aliasing etc. Time to bring out KTR and see > what's going on. I've got KTR compiled in, but last time I switched it on. I got swamped in traffic, and I sort of got locked out of the server... :( Could also be because I was writing it to a file as well. So you'll have to help/tell me what to do. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 15:34:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0062925 for ; Tue, 10 Jun 2014 15:34:10 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7E4AC2315 for ; Tue, 10 Jun 2014 15:34:10 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 58CCA122F2; Wed, 11 Jun 2014 01:34:08 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-69-181-164-196.hsd1.ca.comcast.net [69.181.164.196]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUW80013 (AUTH peterg@ptree32.com.au); Wed, 11 Jun 2014 01:34:07 +1000 Message-ID: <53972572.80902@freebsd.org> Date: Tue, 10 Jun 2014 08:34:10 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nils Beyer Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <20140610103233.4C31D6C4@hub.freebsd.org> In-Reply-To: <20140610103233.4C31D6C4@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 15:34:10 -0000 Hi Nils, > Confirmed. Running a bhyved 3-vCPU-"CentOS 6.5", the host CPU load for "vcpu 0" > is around 12% now. Doh, that's not good - haven't given Centos 6.5 a try; will now to investigate this. > Ping times to the VM are fluctuating - ranging from 0.185ms to 35ms. Hmmm, will look at that as well. > Now it's time to scrap all ESXi hosts here... ;-) Great to hear, but you may want to hold off a little bit - there are still some bugfixes in the pipe. Hoping to be mostly done in a week or so. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 15:58:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E0611A9; Tue, 10 Jun 2014 15:58:04 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41D5B24E6; Tue, 10 Jun 2014 15:58:03 +0000 (UTC) Received: by mail-la0-f46.google.com with SMTP id hz20so3889994lab.5 for ; Tue, 10 Jun 2014 08:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6mychWtN/91vFiEVBdrg28FvlRW00r3KQNg9JVWvQqk=; b=awLGWtyc3I8GuQ6jxP8sgp/m7CjFvmKRHGaI1fLB2td7HOKwAgEJfpMuLMO4l6uhOc N1TDg4siFdC/E37L/tyYFfxTs6nJg7tJBdBFpvhmpJaBAhPv0Mc201V641vz2uLURyRe SVnenltjF9x7JCkgF0fw/4efb0GGXPwnbb8McuLrfw58WYHw77w2hvxrqiG1gotWg2mA 821LVicVWP6VG+Eqf2ruzQ1C0uLs+fD3CUxMYs2fdcjVBz2fsVoAH2UGzZZMx5Ly7J7R j815JzqJLAUWTBs4g9oTDf5MyaS6bYlzLmo5I/W8ZoYKmlM3zsOqH9S7JFaGKy9olbZ6 COkw== MIME-Version: 1.0 X-Received: by 10.112.63.65 with SMTP id e1mr1763453lbs.81.1402415880970; Tue, 10 Jun 2014 08:58:00 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 08:58:00 -0700 (PDT) In-Reply-To: <20140610114342.GA11194@dev.san.ru> References: <20140610114342.GA11194@dev.san.ru> Date: Tue, 10 Jun 2014 08:58:00 -0700 X-Google-Sender-Auth: GZf-cv8HpjyZMTNLAFOetPKltFA Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 15:58:04 -0000 On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: > Hi Craig, > > It's a little unfortunate that there's no stacktrace, though we can try > to see what's going on without it. > > Could you please edit libvirtd.conf (should be something like > /usr/local/etc/libvirtd.conf if installed from port) to add: > > log_level = 2 > > then run libvirtd by hand in foreground: OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf and ran libvirtd in the foreground: 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 15:53:26.846+0000: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not accessible 2014-06-10 15:53:27.023+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 15:53:27.023+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 15:53:27.034+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 15:53:27.034+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 15:53:27.130+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 15:53:53.668+0000: 34485598208: error : virNetDevGetMTU:404 : Cannot get interface MTU on 'virbr0': Device not configured Segmentation fault (core dumped) I got that coredump after doing: virsh -c 'bhyve:///system' virsh # define /tmp/bhyve.xml virsh # start bhyve From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 16:53:44 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66A22EA9; Tue, 10 Jun 2014 16:53:44 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2124B2AB5; Tue, 10 Jun 2014 16:53:43 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 312291534C0; Tue, 10 Jun 2014 18:53:41 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZPyBiIfoLVxK; Tue, 10 Jun 2014 18:53:39 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372] (unknown [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372]) by smtp.digiware.nl (Postfix) with ESMTP id A68CA153448; Tue, 10 Jun 2014 18:53:39 +0200 (CEST) Message-ID: <53973810.6060503@digiware.nl> Date: Tue, 10 Jun 2014 18:53:36 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Peter Grehan , Nils Beyer Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <20140610103233.4C31D6C4@hub.freebsd.org> <53972572.80902@freebsd.org> In-Reply-To: <53972572.80902@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 16:53:44 -0000 On 2014-06-10 17:34, Peter Grehan wrote: > Hi Nils, > >> Confirmed. Running a bhyved 3-vCPU-"CentOS 6.5", the host CPU load for >> "vcpu 0" >> is around 12% now. > > Doh, that's not good - haven't given Centos 6.5 a try; will now to > investigate this. All Ubuntus I have installed in VMs give more or less perform the same... (13.10, 12.04 LTS) Although the overhead is just 5% and more evenly distibuted over both vPCUs. --WjW From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 16:55:51 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 001F1FF1 for ; Tue, 10 Jun 2014 16:55:50 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id B077F2ACB for ; Tue, 10 Jun 2014 16:55:50 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id 64BB31414809 for ; Tue, 10 Jun 2014 18:55:39 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 48D351A6590 for ; Tue, 10 Jun 2014 18:55:40 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Tue, 10 Jun 2014 18:55:40 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Bheve: Slow linux syscalls on AMD To: freebsd-virtualization@freebsd.org References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <20140610103233.4C31D6C4@hub.freebsd.org> <53972572.80902@freebsd.org> Lines: 46 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 16:55:51 -0000 Hi Peter, Peter Grehan wrote: >> Confirmed. Running a bhyved 3-vCPU-"CentOS 6.5", the host CPU load for >> "vcpu 0" is around 12% now. > > Doh, that's not good - haven't given Centos 6.5 a try; will now to > investigate this. CentOS is a bit bitchy about booting from harddisk. You'll have to provide a shorter "linux"-grub-line than what's written in the "grub.conf"-file; some- thing like this: linux /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root initrd /initramfs-2.6.32-431.el6.x86_64.img or else the LVM-groups won't get activated. >> Ping times to the VM are fluctuating - ranging from 0.185ms to 35ms. > > Hmmm, will look at that as well. For what it's worth, this is my "bhyve"-command line: =============================================================================== bhyve \ -w \ -c 3 \ -m 4096M \ -A \ -H \ -P \ -l com1,/dev/nmdm0A \ -s 0,hostbridge \ -s 1,lpc \ -s 2,ahci-cd,/mnt/iso/${ISO} \ -s 3,virtio-blk,lun0 \ -s 4,virtio-blk,lun1 \ -s 5,virtio-net,tap0 \ ${VM} =============================================================================== My host CPU is an "AMD Phenom(tm) II X6 1055T Processor"... Regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 17:11:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4275993E for ; Tue, 10 Jun 2014 17:11:48 +0000 (UTC) Received: from nijmegen.renzel.net (mx1.renzel.net [195.243.213.130]) by mx1.freebsd.org (Postfix) with ESMTP id F29DF2C8C for ; Tue, 10 Jun 2014 17:11:47 +0000 (UTC) Received: from dublin.vkf.isb.de.renzel.net (unknown [10.0.0.80]) by nijmegen.renzel.net (smtpd) with ESMTP id ADE20141480D for ; Tue, 10 Jun 2014 19:11:37 +0200 (CEST) Received: from asbach.renzel.net (unknown [10.2.0.7]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTP id 927311A668E for ; Tue, 10 Jun 2014 19:11:38 +0200 (CEST) Content-Type: text/plain; charset="ISO-8859-1" From: Nils Beyer Organization: VKF Renzel GmbH Date: Tue, 10 Jun 2014 19:11:38 +0200 User-Agent: KNode/4.12.5 Content-Transfer-Encoding: 7Bit Subject: Re: Bheve: Slow linux syscalls on AMD To: freebsd-virtualization@freebsd.org References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <20140610103233.4C31D6C4@hub.freebsd.org> <53972572.80902@freebsd.org> <53973810.6060503@digiware.nl> Lines: 15 MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.98 at nijmegen.renzel.net X-Virus-Status: Clean X-Spam-Status: No, score=-6.5 required=7.0 tests=BAYES_00,MISSING_MID, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on nijmegen.renzel.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:11:48 -0000 Hi Willem, Willem Jan Withagen wrote: > All Ubuntus I have installed in VMs give more or less perform the > same... (13.10, 12.04 LTS) > Although the overhead is just 5% and more evenly distibuted over both > vPCUs. I can confirm this. Just tested with an Ubuntu Server 12.04.3 ISO. Load on "vcpu 0" is now wobbling around 3% - 4%. Sum on all vcpus is around 5%. So, the Redhat clones do something different... Regards, Nils From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 17:28:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84AA6613 for ; Tue, 10 Jun 2014 17:28:30 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 427AC2DDC for ; Tue, 10 Jun 2014 17:28:30 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 2C526123DD; Wed, 11 Jun 2014 03:28:28 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUW82512 (AUTH peterg@ptree32.com.au); Wed, 11 Jun 2014 03:28:27 +1000 Message-ID: <5397403A.2090001@freebsd.org> Date: Tue, 10 Jun 2014 10:28:26 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Nils Beyer Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <20140610103233.4C31D6C4@hub.freebsd.org> <53972572.80902@freebsd.org> <20140610165553.3A29694@hub.freebsd.org> In-Reply-To: <20140610165553.3A29694@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:28:30 -0000 Hi Nils, > CentOS is a bit bitchy about booting from harddisk. You'll have to provide a > shorter "linux"-grub-line than what's written in the "grub.conf"-file; some- > thing like this: > > linux /vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root > initrd /initramfs-2.6.32-431.el6.x86_64.img > > or else the LVM-groups won't get activated. Yep, I think that's just the grubv1 versus grubv2 syntax differences :( Ubuntu/Debian are at least grubv2-native which makes things a bit easier. > For what it's worth, this is my "bhyve"-command line: That looks fine. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 17:39:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92FB8D07; Tue, 10 Jun 2014 17:39:41 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3F9F2F04; Tue, 10 Jun 2014 17:39:40 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so4106474lan.23 for ; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=iUsX0MPEBUdNQbHcyU/bQuifIQIBlZFm7mUAQvwOSdE=; b=xNtV/iynAY4mFbpHVKDNWmLuun5jvZUDwvMFdvqwsGjnPSTnf2vXQX2OqGY7L9yJ7A BfYHxX8ryqtB0xHOGJy2qO4AgaJ4uJP4oEPy7I0HKwvjlo4LNYzliqJ+NQM5pNroSEQT 2wfOYRVky/q9M7rOx0opwrgqgszqxWENSc4jYBeMDjsrWZ1Nx70L600ygNPG+WhspWlt X1DEqzXv0nxJ8DaRPo1q7b+nZCaPWNfBXFST4z8VxXnWof0G9t5dPnqWCx8O76sVm8ym hcLIuNWvjznwFF+yyTxqz9ewfNwRC9iDbViG5MrqG3wS7Kob4rV1952y3hk0fBPCe1V7 OQXg== MIME-Version: 1.0 X-Received: by 10.112.146.202 with SMTP id te10mr2292463lbb.75.1402421978558; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> Date: Tue, 10 Jun 2014 10:39:38 -0700 X-Google-Sender-Auth: R3ak9HTibl_6mmGOKIyEaRCddDE Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:39:41 -0000 On Tue, Jun 10, 2014 at 8:58 AM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: >> Hi Craig, >> >> It's a little unfortunate that there's no stacktrace, though we can try >> to see what's going on without it. >> >> Could you please edit libvirtd.conf (should be something like >> /usr/local/etc/libvirtd.conf if installed from port) to add: >> >> log_level = 2 >> >> then run libvirtd by hand in foreground: > > OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf > and ran libvirtd in the foreground: > > 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 > 2014-06-10 15:53:26.846+0000: 34485593088: warning : > virDriverLoadModule:63 : Module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not > accessible I saw the above warning, so I tried patching the port with this: --- pkg-plist.orig 2014-06-10 10:36:22.084757789 -0700 +++ pkg-plist 2014-06-10 09:27:25.280043239 -0700 @@ -274,6 +274,8 @@ lib/libvirt/connection-driver/libvirt_driver_secret.la lib/libvirt/connection-driver/libvirt_driver_storage.so lib/libvirt/connection-driver/libvirt_driver_storage.la +lib/libvirt/connection-driver/libvirt_driver_uml.so +lib/libvirt/connection-driver/libvirt_driver_uml.la lib/libvirt/connection-driver/libvirt_driver_vbox.so lib/libvirt/connection-driver/libvirt_driver_vbox.la However, when I restarted libvirtd, I got different errors: 2014-06-10 17:38:50.269+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 17:38:50.269+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 17:38:50.563+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 17:38:50.563+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 17:38:50.575+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 17:38:50.575+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 17:38:50.663+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 17:38:50.663+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 17:56:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FE6F578; Tue, 10 Jun 2014 17:56:32 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CBE820F4; Tue, 10 Jun 2014 17:56:31 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id ty20so4213951lab.12 for ; Tue, 10 Jun 2014 10:56:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=LybjP7zymltthPdonftR2oJF/ppY/zhjyo0rBp/UuFI=; b=vYDc/pz0CJz4JPNUbwqK1LRITuifOHOp85VzWwrH2Z/NDBdphFYnBb3fGpgx1ouIRs BQbwJHPBtnHnowoaryaaw08BO9lvHCUgkyOWeOjYDAl2hG7+jNwsQJNCQ6gnmqEnOZLT VHUGybC2dR1qV3wojydbkqBoqKbmxal0bxMsCUamOSaYaB2rL3oQoeW6VzIRhkXrOwLL hOP3jO/fAfiENiBtoSICMsp/hpGkUF96xTyBYvXywbtzRT5UkpAe5CF5+B8To0vCkUxt ScvqcFA55ose1wyvBtJQZ2zCgB/rEzfvTB/yzbawHS4/0sqqT0vEyGxImnNLMoC4C0rV J/Tw== X-Received: by 10.152.23.136 with SMTP id m8mr23843973laf.2.1402422989518; Tue, 10 Jun 2014 10:56:29 -0700 (PDT) Received: from kloomba ([77.94.196.164]) by mx.google.com with ESMTPSA id k1sm22170264lbb.9.2014.06.10.10.56.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 10:56:28 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 21:54:22 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610175420.GC61743@kloomba> References: <20140610114342.GA11194@dev.san.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="c3bfwLpm8qysLVxt" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:56:32 -0000 --c3bfwLpm8qysLVxt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 8:58 AM, Craig Rodrigues wr= ote: > > On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy = wrote: > >> Hi Craig, > >> > >> It's a little unfortunate that there's no stacktrace, though we can try > >> to see what's going on without it. > >> > >> Could you please edit libvirtd.conf (should be something like > >> /usr/local/etc/libvirtd.conf if installed from port) to add: > >> > >> log_level =3D 2 > >> > >> then run libvirtd by hand in foreground: > > > > OK, I added log_level =3D 2 to /usr/local/etc/libvirt/libvirtd.conf > > and ran libvirtd in the foreground: > > > > 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 > > 2014-06-10 15:53:26.846+0000: 34485593088: warning : > > virDriverLoadModule:63 : Module > > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not > > accessible >=20 > I saw the above warning, so I tried patching the port with this: >=20 > --- pkg-plist.orig 2014-06-10 10:36:22.084757789 -0700 > +++ pkg-plist 2014-06-10 09:27:25.280043239 -0700 > @@ -274,6 +274,8 @@ > lib/libvirt/connection-driver/libvirt_driver_secret.la > lib/libvirt/connection-driver/libvirt_driver_storage.so > lib/libvirt/connection-driver/libvirt_driver_storage.la > +lib/libvirt/connection-driver/libvirt_driver_uml.so > +lib/libvirt/connection-driver/libvirt_driver_uml.la > lib/libvirt/connection-driver/libvirt_driver_vbox.so > lib/libvirt/connection-driver/libvirt_driver_vbox.la This warning is not critical. Actually, UML should not be built on FreeBSD and I would need to disable it by default. For now I think the proper fix is to configure with '--without-uml' flag. As for the segfault, I think I have a guess. Could you please drop '--without-network' from CONFIGURE_ARGS in the port's Makefile and try to rebuild? Roman Bogorodskiy --c3bfwLpm8qysLVxt Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTl0ZMAAoJEMltX/4IwiJqyuIIAJymYJPw21w6t4UyMmJHSfBk 42AgLt7jGgH1CiGuXjguLgtX/NUaaPlcHoen4KBVq+d7pPjKZvyEjbQBXfYURIjw 22p+ZjIz/g4mkFcel0OCMc0CSHitR2m8TUnk89kg5vljbK52DM+lnYUE7oaXanxc 9s+GYouBmzfisCbdjOcc++zQqOWz0QqR/WSzAYULsye59a0SycjEChRUQ7ZLILfK eMeBpCVq+cU7YXA8yEbA5kYDorWsAKS/IOQhpB0kEwYxaTiYC1CppltDT8a8smmi 6U/wPGBHa97Owf3cWTLSxTNbF3Jqk1UIpQZZyGFK/lQ10oB5KnI1ncntF08f2v4= =hNbZ -----END PGP SIGNATURE----- --c3bfwLpm8qysLVxt-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 18:04:32 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EA1C8F0 for ; Tue, 10 Jun 2014 18:04:32 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 2C53A21F0 for ; Tue, 10 Jun 2014 18:04:32 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 6D8FF12677; Wed, 11 Jun 2014 04:04:30 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUW82934 (AUTH peterg@ptree32.com.au); Wed, 11 Jun 2014 04:04:29 +1000 Message-ID: <539748AA.5060005@freebsd.org> Date: Tue, 10 Jun 2014 11:04:26 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Willem Jan Withagen Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <5396D18C.8000401@digiware.nl> <53970F5A.8060900@freebsd.org> <53971C3A.30406@digiware.nl> In-Reply-To: <53971C3A.30406@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 18:04:32 -0000 Hi Willem, > I've got KTR compiled in, but last time I switched it on. > I got swamped in traffic, and I sort of got locked out of the server... :( > Could also be because I was writing it to a file as well. > > So you'll have to help/tell me what to do. For looking at this, I'd use one single-vCPU guest to avoid KTR noise from other vCPU/guests. The problem shows up with this so no issue there. First, boot the VM and get it sitting at a shell prompt idling. On the host, first see that there are enough entries in the KTR buffer # sysctl debug.ktr.entries <32/64K is a good number> If it's too small, it can be bumped up # sysctl debug.ktr.entries=65536 debug.ktr.entries: 8192 -> 65536 The bhyve kernel module uses the ktr mask value 0x01. This should be turned off by default: # sysctl debug.ktr.mask debug.ktr.mask: 0 To get some traces, turn this on for a few seconds # sysctl debug.ktr.mask=1 ; sleep 3 ; sysctl debug.ktr.mask=0 The ktr trace buffer can now be examined: # ktrdump -ct To repeat/rerun the process, make sure the ktr buffer is cleared with # sysctl debug.ktr.clear=1 debug.ktr.clear: 0 -> 0 later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 19:01:54 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 494FCA3E; Tue, 10 Jun 2014 19:01:54 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D87627F6; Tue, 10 Jun 2014 19:01:53 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id pv20so4222182lab.36 for ; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5769kDRIX79tWcyli7KRrjvXrsTrWOuF8qmznu6hi8A=; b=JdSsv9gBCv8zIPeu4nqEUyXm9J96+0q63ek0vfiH4lzx7q33oxMPxmpvlOwIbt+gbg pwfRT6Q+oPEjZgxpa6FvA+cvAjrLpwARmThLll32lANgQvNAlQ2ZjsI5zDMDLd8cfn04 bS8pOsjpNJ7uxk202/4EjqR3XBDKpYyBH0oTUgbSwVgQXkgL31qeLqhPwnQ0AIIbf/aL X77+RWe7NFSnqzmXSiUv/WNsVj3HMCnJzK9kErSRMcVjqPTwXYmY0a7RmSEtpfrQlDYW 7VTNN2fXWJJ9bG/ZowRtGtye2uMGNIeagcc/ouhFgBTKreTz/71efZ6/XBR349OfpllO aGDQ== MIME-Version: 1.0 X-Received: by 10.152.87.20 with SMTP id t20mr2397001laz.79.1402426911351; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) In-Reply-To: <20140610175420.GC61743@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> Date: Tue, 10 Jun 2014 12:01:51 -0700 X-Google-Sender-Auth: H-_EjKoONKpXFj6JhiWuajk2xZw Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:01:54 -0000 On Tue, Jun 10, 2014 at 10:54 AM, Roman Bogorodskiy wrote: > This warning is not critical. Actually, UML should not be built on > FreeBSD and I would need to disable it by default. For now I think the > proper fix is to configure with '--without-uml' flag. > > As for the segfault, I think I have a guess. > > Could you please drop '--without-network' from CONFIGURE_ARGS in the > port's Makefile and try to rebuild? > > Roman Bogorodskiy OK, I dropped --without-network from CONFIGURE_ARGS and rebuilt the port. I am still getting the coredump, but am getting different errors: 2014-06-10 18:59:22.713+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 18:59:22.713+0000: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_network.so not accessible 2014-06-10 18:59:22.715+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so: Undefined symbol "networkAllocateActualDevice" 2014-06-10 18:59:22.716+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 18:59:22.845+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 18:59:22.845+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-10 18:59:22.847+0000: 34485605376: error : do_open:1166 : no connection driver available for qemu:///system 2014-06-10 18:59:42.273+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet7: Invalid argument Segmentation fault (core dumped) From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 19:17:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0397B133; Tue, 10 Jun 2014 19:17:15 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19DE52941; Tue, 10 Jun 2014 19:17:13 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id u10so843366lbd.10 for ; Tue, 10 Jun 2014 12:17:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7XvX+ESCIjQFv1erfiOllgxNETthdLWgwB0S1pL3ROE=; b=PZFP/+e0W8ZI58cm0MRJxNzBeRVIH1kdLhG71K/xv26ZditEUB1z8Z5ANr1qu0gMDe 1YAEBVB9WHsudq0AkRMKO3CY+XUr3wF0+FSgNK2DJlY8SztEt4Va8PXZMmEC3QE8FNMq HdRlqiU7ZmlX4dOonUUtsLEWYxNVKQkIDwqrlsX52HeBGgIlIjP1Dymypau/XL5I3D/n tdV8T5m2qzeR4PojwbopObQxRp1LOv+GAVA9f1h908jj5bn4M/fqFE9VGdvaSFRrRuiH 3A3bSMfWYXR+NpNT60/Nn6Ivf/tErDVWLWTocFECDRcOouMllAFbzYDhow1mheVexTOV oFqg== X-Received: by 10.112.14.42 with SMTP id m10mr2328900lbc.84.1402427832003; Tue, 10 Jun 2014 12:17:12 -0700 (PDT) Received: from kloomba ([77.94.196.164]) by mx.google.com with ESMTPSA id lq20sm22351683lbb.24.2014.06.10.12.17.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 12:17:11 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 23:15:01 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610191439.GA220@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JgQwtEuHJzHdouWu" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:17:15 -0000 --JgQwtEuHJzHdouWu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 10:54 AM, Roman Bogorodskiy w= rote: > > This warning is not critical. Actually, UML should not be built on > > FreeBSD and I would need to disable it by default. For now I think the > > proper fix is to configure with '--without-uml' flag. > > > > As for the segfault, I think I have a guess. > > > > Could you please drop '--without-network' from CONFIGURE_ARGS in the > > port's Makefile and try to rebuild? > > > > Roman Bogorodskiy >=20 > OK, I dropped --without-network from CONFIGURE_ARGS and rebuilt the port. > I am still getting the coredump, but am getting different errors: >=20 > 2014-06-10 18:59:22.713+0000: 34485593088: info : libvirt version: 1.2.5 > 2014-06-10 18:59:22.713+0000: 34485593088: warning : > virDriverLoadModule:63 : Module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_network.so not > accessible ^^^^ Hmmm, this sounds weird. I think I need to try the port version myself to see if I could reproduce it this way (I'm usually using git version of libvirt instead of port). In the meantime, I'd really appreciate if you could build the port with WITH_DEBUG flag and try to get a stacktrace. > 2014-06-10 18:59:22.715+0000: 34485593088: error : > virDriverLoadModule:71 : failed to load module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so > /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so: > Undefined symbol "networkAllocateActualDevice" > 2014-06-10 18:59:22.716+0000: 34485593088: error : > virDriverLoadModule:71 : failed to load module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: > Undefined symbol "inotify_add_watch" > 2014-06-10 18:59:22.845+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-10 18:59:22.845+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-10 18:59:22.847+0000: 34485605376: error : do_open:1166 : no > connection driver available for qemu:///system > 2014-06-10 18:59:42.273+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet7: > Invalid argument This last line also sounds strange. I'm wondering if you execute 'net-list --all' will you see the 'default' network? And if it's there, will 'net-show default' should that Autostart is enabled for it? > Segmentation fault (core dumped) Thanks, Roman Bogorodskiy --JgQwtEuHJzHdouWu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTl1kwAAoJEMltX/4IwiJqmFwIAMFDi5JMJQLIEMY5dIRX422i yfyDgeejj0mOssN2+v0XnqDlBReRSGhx6BmpP2fh/eVxUVnZEWcsMmq9DKcmLIkr Z93j4BR+7CoEc6hc045LwYcJpgU7aH5lp0fBnthYZDIDDerep5kIT5DbQgFdTSdW yatRI/MIrX1EpQmYXczr4y6WNh/0ZffQOM4JrYzzPRwcsGNx6J6Eo1kJzNPVnJ9j oqV5pWNdKuil9yNoTaNXnCzUJ22qL79uwG71RIG4n8/f1mhi8h68IldnkFWwXSxa 2xLbqtJgvp2YjbM/bzXnNCU6Mfpm3j3dBXCrGq0iOAM2YtX3XFYLKFtKWtoPnJ0= =IA6e -----END PGP SIGNATURE----- --JgQwtEuHJzHdouWu-- From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 19:25:24 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0555499; Tue, 10 Jun 2014 19:25:23 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C3D72A04; Tue, 10 Jun 2014 19:25:22 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id 10so4234344lbg.18 for ; Tue, 10 Jun 2014 12:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=uAB/nAjMZ5ZUj4gVy+jkeVb7YvJ0e4/4pSpFKmDfpmc=; b=RGeuizuPF9VcPQRewei4WmWbwD5tAQFJA3+/S91aZLzKWMwnWWwlwYqrE+RysYsgme Opmasjlwi5CSrrqBcCGL2mMHWakKSMJmILgBF3UrZFE36rTMxfPDyrk1h2UL41VscjDM EhWGy4waqhz3ptkraDtSo+tUFa85Gi3KSt0BMugwebVoIDD96bDvOjElvlnYlr9zTyvh qus02H6nXFnGUfaek0dn2I6FKMVzs9XMS5qR0XrwxYduCKmzJtHDTQyxhndpJD26w9mM 531grJrMpvr18kfJ3Bo9RZiVYEYkanD+27Q9d57SRPOFCKtGHQ1oQhNe+B1u18oeheMw tPaw== MIME-Version: 1.0 X-Received: by 10.152.198.129 with SMTP id jc1mr2881226lac.63.1402428321065; Tue, 10 Jun 2014 12:25:21 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 12:25:20 -0700 (PDT) In-Reply-To: <20140610191439.GA220@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 12:25:20 -0700 X-Google-Sender-Auth: rd4x9ybya1csiRclI_62D45fDyI Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:25:24 -0000 On Tue, Jun 10, 2014 at 12:15 PM, Roman Bogorodskiy wrote: > > This last line also sounds strange. > > I'm wondering if you execute 'net-list --all' will you see the 'default' > network? And if it's there, will 'net-show default' should that > Autostart is enabled for it? If I do: # virsh -c 'bhyve:///system' Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # net-list --all error: Failed to get the number of active networks error: this function is not supported by the connection driver: virConnectNumOfNetworks The past few times, I have been compiling the port using WITH_DEBUG=yes. However, I am not getting useful stacktraces: /usr/ports/devel/libvirt/work/libvirt-1.2.5]# file /usr/local/sbin/libvirtd /usr/local/sbin/libvirtd: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 11.0 (1100022), not stripped # gdb /usr/local/sbin/libvirtd libvirtd.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `libvirtd'. Program terminated with signal 11, Segmentation fault. #0 0x0000000801aa8513 in ?? () (gdb) where #0 0x0000000801aa8513 in ?? () #1 0x0000001800000008 in ?? () #2 0x00007fffff3f9330 in ?? () #3 0x0000000809e5510c in ?? () #4 0x0000000000000007 in ?? () #5 0x0000000809e5531b in ?? () #6 0x0000000000000000 in ?? () BTW, can you give me: (1) exact git command to checkout libvirt (2) configure command you typed (3) make command you typed I want to get a working setup like you have. It looks like the port needs some fixing before it can be used fully. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 20:54:20 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BACE5253; Tue, 10 Jun 2014 20:54:20 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6F512268; Tue, 10 Jun 2014 20:54:19 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id el20so4349659lab.29 for ; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=n93JJS7xNnkWxz67beRPIDfMBc5EkcAJtsW6DXoiRu8=; b=JpCzkf1U1LkHSnGF3WR8Vco8dEbdF2MyNvmOQPng2/dRKnqmtZM2na940+BeaNn4Sb BztkDGPjLYiIMTxTyQ3qizzMo230os0NqFSpXDdN6wrKvJHbtb+o06DswcgHW8Zqaax2 2tQnqpyr7YvgQoI3JAXeUFpOTvxUbFDZnTh06Wb+4v96DPLg9j4MzfT9NYgj8jJ+6kpV JJdecymP2FoCAcq+QnujhoCkflGcswE/ib/b/mcxlABueI8+Zb3CinTCq9o8b6jRebo3 4pWUx8S2dcZcSnDfwXjNTetKkuqIf9h/sQbXA+3ZMFTgy6z9TJmKyy8oOOKuB1u+hbYL rJmQ== MIME-Version: 1.0 X-Received: by 10.152.36.38 with SMTP id n6mr24413266laj.0.1402433657667; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 13:54:17 -0700 X-Google-Sender-Auth: 1Lp3IvBfMmszNDsK1mN7lNIhWTM Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 20:54:20 -0000 On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote: > > > # gdb /usr/local/sbin/libvirtd libvirtd.core > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > Core was generated by `libvirtd'. > Program terminated with signal 11, Segmentation fault. > #0 0x0000000801aa8513 in ?? () > (gdb) where > #0 0x0000000801aa8513 in ?? () > #1 0x0000001800000008 in ?? () > #2 0x00007fffff3f9330 in ?? () > #3 0x0000000809e5510c in ?? () > #4 0x0000000000000007 in ?? () > #5 0x0000000809e5531b in ?? () > #6 0x0000000000000000 in ?? () OK, so it looks like in CURRENT, at least for the devel/libvirt port, the gdb in base, and also gdb in ports cannot debug libvirtd.core. I enabled WITH_LLDB=yes in /etc/src.conf and recompiled to install lldb. I am not so familiar with lldb, but at least I can get a stacktrace with it. lldb /usr/local/sbin/libvirtd libvirtd.core (lldb) bt * thread #1: tid = 0, 0x0000000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason = signal SIGSEGV * frame #0: 0x0000000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, flags=9) + 547 at virnetdevtap.c:583 frame #1: 0x0000000809e20569 libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x000000080a81b800, cmd=0x000000080a4db200, dryRun=false) + 841 at bhyve_command.c:82 frame #2: 0x0000000809e20196 libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x000000080a48a040, def=0x000000080a81b800, dryRun=false) + 310 at bhyve_command.c:248 frame #3: 0x0000000809e24d6a libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x000000080a4db100, driver=0x000000080a48a040, vm=0x000000080a48b0f0, reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at bhyve_process.c:134 frame #4: 0x0000000809e2334a libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x000000080a48a280, flags=0) + 490 at bhyve_driver.c:847 frame #5: 0x0000000809e2315a libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x000000080a48a280) + 26 at bhyve_driver.c:867 frame #6: 0x0000000801b96aad libvirt.so.1002`virDomainCreate(domain=0x000000080a48a280) + 637 at libvirt.c:8968 frame #7: 0x0000000001075427 libvirtd`remoteDispatchDomainCreate(server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0, rerr=0x00007fffff9fcc30, args=0x0000000807934340) + 231 at remote_dispatch.h:3048 frame #8: 0x000000000103b84e libvirtd`remoteDispatchDomainCreateHelper(server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0, rerr=0x00007fffff9fcc30, args=0x0000000807934340, ret=0x000000080a421058) + 254 at remote_dispatch.h:3026 frame #9: 0x0000000801c26b20 libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x000000080790f400, server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0) + 784 at virnetserverprogram.c:437 frame #10: 0x0000000801c26641 libvirt.so.1002`virNetServerProgramDispatch(prog=0x000000080790f400, server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0) + 705 at virnetserverprogram.c:307 frame #11: 0x000000000108f97c libvirtd`virNetServerProcessMsg(srv=0x000000080781c400, client=0x00000008078df480, prog=0x000000080790f400, msg=0x00000008078791d0) + 332 at virnetserver.c:172 frame #12: 0x000000000108cd9c libvirtd`virNetServerHandleJob(jobOpaque=0x0000000807934360, opaque=0x000000080781c400) + 204 at virnetserver.c:193 frame #13: 0x0000000801ac84b8 libvirt.so.1002`virThreadPoolWorker(opaque=0x0000000807901d60) + 680 at virthreadpool.c:145 frame #14: 0x0000000801ac7b9b libvirt.so.1002`virThreadHelper(data=0x000000080781b520) + 75 at virthread.c:197 frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 21:36:49 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA397D8A; Tue, 10 Jun 2014 21:36:49 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB92E25AF; Tue, 10 Jun 2014 21:36:48 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so4271905lan.23 for ; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UirTTnAYIlu17tjb4zygEEj05bq+JUIYUEyrVxnvvb8=; b=AVdHK0CmdzqV5tBjE3mmkBqMeFnSPnyPyV1kgzGwcVF9cTdkTgd1sakOOIJ3bFgBKh N+xMZTiAug9Wyn4smqjho8sp8XQIbPUe3yskoWXcpIdTHZZOYWdISM4Iij6WT3hpvEnC 0FIuSTZxRx5ubEUGs0UuyH85NfMszcsFMEHMw48C4CoUNbR6zpaDNtWtRF5Gm/YDRyHx sEYCku692xVc4aRfHH+FNr0C1cBdhzUDQtElBzTxf/cwDR9dcPOwSQ1ip4AyXgZqyJpb bdFuDA/D0KKrFO+868x0tKHw/P8Nc6COWtoNbnkKPNZ5AK96VkyLW8l/P9lCwOBFQcdw +SLQ== MIME-Version: 1.0 X-Received: by 10.152.29.168 with SMTP id l8mr24364001lah.35.1402436206546; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 14:36:46 -0700 X-Google-Sender-Auth: ODkdSs4u1jWrrW_ZwEaSZ3hbxz0 Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 21:36:49 -0000 On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote: > lldb /usr/local/sbin/libvirtd libvirtd.core > > (lldb) bt > * thread #1: tid = 0, 0x0000000801aa8513 > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, > ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, > vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, > virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, > flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason = > signal SIGSEGV > * frame #0: 0x0000000801aa8513 > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, > ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, > vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, > virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, > flags=9) + 547 at virnetdevtap.c:583 > frame #1: 0x0000000809e20569 > libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x000000080a81b800, > cmd=0x000000080a4db200, dryRun=false) + 841 at bhyve_command.c:82 > frame #2: 0x0000000809e20196 > libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x000000080a48a040, > def=0x000000080a81b800, dryRun=false) + 310 at bhyve_command.c:248 > frame #3: 0x0000000809e24d6a > libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x000000080a4db100, > driver=0x000000080a48a040, vm=0x000000080a48b0f0, > reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at > bhyve_process.c:134 > frame #4: 0x0000000809e2334a > libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x000000080a48a280, > flags=0) + 490 at bhyve_driver.c:847 > frame #5: 0x0000000809e2315a > libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x000000080a48a280) + 26 > at bhyve_driver.c:867 > frame #6: 0x0000000801b96aad > libvirt.so.1002`virDomainCreate(domain=0x000000080a48a280) + 637 at > libvirt.c:8968 > frame #7: 0x0000000001075427 > libvirtd`remoteDispatchDomainCreate(server=0x000000080781c400, > client=0x00000008078df480, msg=0x00000008078791d0, > rerr=0x00007fffff9fcc30, args=0x0000000807934340) + 231 at > remote_dispatch.h:3048 > frame #8: 0x000000000103b84e > libvirtd`remoteDispatchDomainCreateHelper(server=0x000000080781c400, > client=0x00000008078df480, msg=0x00000008078791d0, > rerr=0x00007fffff9fcc30, args=0x0000000807934340, > ret=0x000000080a421058) + 254 at remote_dispatch.h:3026 > frame #9: 0x0000000801c26b20 > libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x000000080790f400, > server=0x000000080781c400, client=0x00000008078df480, > msg=0x00000008078791d0) + 784 at virnetserverprogram.c:437 > frame #10: 0x0000000801c26641 > libvirt.so.1002`virNetServerProgramDispatch(prog=0x000000080790f400, > server=0x000000080781c400, client=0x00000008078df480, > msg=0x00000008078791d0) + 705 at virnetserverprogram.c:307 > frame #11: 0x000000000108f97c > libvirtd`virNetServerProcessMsg(srv=0x000000080781c400, > client=0x00000008078df480, prog=0x000000080790f400, > msg=0x00000008078791d0) + 332 at virnetserver.c:172 > frame #12: 0x000000000108cd9c > libvirtd`virNetServerHandleJob(jobOpaque=0x0000000807934360, > opaque=0x000000080781c400) + 204 at virnetserver.c:193 > frame #13: 0x0000000801ac84b8 > libvirt.so.1002`virThreadPoolWorker(opaque=0x0000000807901d60) + 680 > at virthreadpool.c:145 > frame #14: 0x0000000801ac7b9b > libvirt.so.1002`virThreadHelper(data=0x000000080781b520) + 75 at > virthread.c:197 > frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 > > -- > Craig OK, I think I found the problem. In this file: http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/bhyve/bhyve_command.c;hb=HEAD We have: static int bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryRun) { virDomainNetDefPtr net = NULL; char *brname = NULL; char *realifname = NULL; int *tapfd = NULL; tapfd is never initialized after this. In the same function, we have this: if (!dryRun) if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net->mac, def->uuid, tapfd, 1, virDomainNetGetActualVirtPortProfile(net), virDomainNetGetActualVlan(net), VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { VIR_FREE(net->ifname); VIR_FREE(brname); return -1; } virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the next parameter is 1 instead of 0. Then further along in this file: http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/util/virnetdevtap.c;hb=HEAD we have: error: for (i = 0; i < tapfdSize && tapfd[i] >= 0; i++) VIR_FORCE_CLOSE(tapfd[i]); tapfd is NULL, but tapfdSize is 1, so the coredump happens here. What is the best way to fix this? -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 18:23:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14E1797C; Wed, 11 Jun 2014 18:23:16 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 222192557; Wed, 11 Jun 2014 18:23:14 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id hr17so72994lab.31 for ; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=2bXKTQjGBs4PbPFOr1Va9Rjgi4hItZb44w0vqmOpPMo=; b=hk529SRVnDtGWRJUp6Gy9iv/hZ+NU9l2zdt1Vb+BzC1ritE08Bd9r+5OZwx2nO82+v je48n3X7iL3h+ldbhmzo62Svh1e9doNVTQOEIv8hkFry5FvlKc+0rfvrws/3SdUb87oh C+IT5V6MvNQ434fmp1IFTnUvnNN8Sc7ABHX2VRGi7ThLUhH6cQu5snbFTNIJEwtwzbxB r2hfku985FQDrCRJE1ZRgVtcWHig9rjt4y3m+HOeoKUC7CM8OxpWIF7N3K5TzvCd+wgn wpcr0wcDU9y0YJJD1ARsNkHnAqmDCfVSXuiWVdXFjFUEbk2LTwBEf/4tHxhcXBM1aG9D 5jMw== X-Received: by 10.152.26.99 with SMTP id k3mr2828239lag.68.1402510992576; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id d9sm22196463lag.19.2014.06.11.11.23.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 11:23:11 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 22:21:02 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611182101.GD2855@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W5WqUoFLvi1M7tJE" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:23:16 -0000 --W5WqUoFLvi1M7tJE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wr= ote: > > On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues = wrote: > > lldb /usr/local/sbin/libvirtd libvirtd.core > > > > (lldb) bt > > * thread #1: tid =3D 0, 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583, name =3D 'libvirtd', stop reaso= n =3D > > signal SIGSEGV > > * frame #0: 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583 > > frame #1: 0x0000000809e20569 > > libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=3D0x000000080a81b800, > > cmd=3D0x000000080a4db200, dryRun=3Dfalse) + 841 at bhyve_command.c:82 > > frame #2: 0x0000000809e20196 > > libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=3D0x0000000= 80a48a040, > > def=3D0x000000080a81b800, dryRun=3Dfalse) + 310 at bhyve_command.c:248 > > frame #3: 0x0000000809e24d6a > > libvirt_driver_bhyve.so`virBhyveProcessStart(conn=3D0x000000080a4db100, > > driver=3D0x000000080a48a040, vm=3D0x000000080a48b0f0, > > reason=3DVIR_DOMAIN_RUNNING_BOOTED, flags=3D0) + 746 at > > bhyve_process.c:134 > > frame #4: 0x0000000809e2334a > > libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=3D0x000000080a48= a280, > > flags=3D0) + 490 at bhyve_driver.c:847 > > frame #5: 0x0000000809e2315a > > libvirt_driver_bhyve.so`bhyveDomainCreate(dom=3D0x000000080a48a280) + 26 > > at bhyve_driver.c:867 > > frame #6: 0x0000000801b96aad > > libvirt.so.1002`virDomainCreate(domain=3D0x000000080a48a280) + 637 at > > libvirt.c:8968 > > frame #7: 0x0000000001075427 > > libvirtd`remoteDispatchDomainCreate(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340) + 231 at > > remote_dispatch.h:3048 > > frame #8: 0x000000000103b84e > > libvirtd`remoteDispatchDomainCreateHelper(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340, > > ret=3D0x000000080a421058) + 254 at remote_dispatch.h:3026 > > frame #9: 0x0000000801c26b20 > > libvirt.so.1002`virNetServerProgramDispatchCall(prog=3D0x000000080790f4= 00, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 784 at virnetserverprogram.c:437 > > frame #10: 0x0000000801c26641 > > libvirt.so.1002`virNetServerProgramDispatch(prog=3D0x000000080790f400, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 705 at virnetserverprogram.c:307 > > frame #11: 0x000000000108f97c > > libvirtd`virNetServerProcessMsg(srv=3D0x000000080781c400, > > client=3D0x00000008078df480, prog=3D0x000000080790f400, > > msg=3D0x00000008078791d0) + 332 at virnetserver.c:172 > > frame #12: 0x000000000108cd9c > > libvirtd`virNetServerHandleJob(jobOpaque=3D0x0000000807934360, > > opaque=3D0x000000080781c400) + 204 at virnetserver.c:193 > > frame #13: 0x0000000801ac84b8 > > libvirt.so.1002`virThreadPoolWorker(opaque=3D0x0000000807901d60) + 680 > > at virthreadpool.c:145 > > frame #14: 0x0000000801ac7b9b > > libvirt.so.1002`virThreadHelper(data=3D0x000000080781b520) + 75 at > > virthread.c:197 > > frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 > > > > -- > > Craig >=20 >=20 >=20 > OK, I think I found the problem. >=20 > In this file: http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3D= src/bhyve/bhyve_command.c;hb=3DHEAD >=20 > We have: >=20 > static int > bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryR= un) > { > virDomainNetDefPtr net =3D NULL; > char *brname =3D NULL; > char *realifname =3D NULL; > int *tapfd =3D NULL; >=20 >=20 > tapfd is never initialized after this. >=20 >=20 > In the same function, we have this: >=20 > if (!dryRun) > if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net= ->mac, > def->uuid, tapfd, 1, >=20 > virDomainNetGetActualVirtPortProfile(net), > virDomainNetGetActualVlan(= net), >=20 > VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { > VIR_FREE(net->ifname); > VIR_FREE(brname); > return -1; > } >=20 >=20 >=20 > virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the > next parameter is 1 instead of 0. >=20 >=20 >=20 > Then further along in this file: > http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3Dsrc/util/virne= tdevtap.c;hb=3DHEAD > we have: >=20 > error: > for (i =3D 0; i < tapfdSize && tapfd[i] >=3D 0; i++) > VIR_FORCE_CLOSE(tapfd[i]); >=20 >=20 > tapfd is NULL, but tapfdSize is 1, so the coredump happens here. >=20 > What is the best way to fix this? Thanks for the analysis! I'll think what would be a proper fix for that problem. Also, while thinking how to reproduce it, I did 'kldunload if_bridge' and started to see the same problem. So, as a temporary fix, could you check if you have bridge support available (in kernel or via module)? Bridges are needed to get networking anyway. Thanks, Roman Bogorodskiy --W5WqUoFLvi1M7tJE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmJ4NAAoJEMltX/4IwiJqXJUH/j+Oy+peyJse6WJYItF+CHnF DfQcz9YsuPxEOrFJIHMVz6MzTbfyXqXaxYLPtMfa0ly8qqgBuGWW3sCnQtYeZokG UxroKlFVKdgxBx5FlByNTHUn0Onj3sqK3oECp6lSKk/3e9HXTFUTsNF2q/5dueDN 66axiddJA4aKhb4RxxOsjBKU/jLSmDATfGp6d3GUentp7tqs4LGlgle4llNd+zNn LjFSN5ncXnCSKiQ3tCWqtGlBUQW5VfY8MhA5EeZe/8Hp65OxNurwDi7eWHdUQQD9 BfxysIiJiFWM1cJ2LBwTsdpbjxYv8JcuVvxLOB7epYQB+PJKQSxYJXNypn/WEWk= =lg/Q -----END PGP SIGNATURE----- --W5WqUoFLvi1M7tJE-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 18:52:44 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5596E7AD; Wed, 11 Jun 2014 18:52:44 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EFB552832; Wed, 11 Jun 2014 18:52:43 +0000 (UTC) Received: from meatwad.mouf.net (cpe-107-015-170-205.nc.res.rr.com [107.15.170.205]) by mouf.net (8.14.5/8.14.5) with ESMTP id s5BIqUdQ099750 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 11 Jun 2014 18:52:39 GMT (envelope-from swills@meatwad.mouf.net) Received: (from swills@localhost) by meatwad.mouf.net (8.14.9/8.14.5/Submit) id s5B9IH9B045237; Wed, 11 Jun 2014 09:18:17 GMT (envelope-from swills) Date: Wed, 11 Jun 2014 09:18:17 +0000 From: Steve Wills To: Neel Natu Subject: Re: interrupt storm on ahci Message-ID: <20140611091816.GB61572@meatwad.mouf.net> References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Delayed for 00:41:50 by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Wed, 11 Jun 2014 18:52:39 +0000 (UTC) X-Spam-Status: No, score=2.4 required=4.5 tests=DATE_IN_PAST_06_12, RCVD_IN_RP_RNBL autolearn=no version=3.3.2 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:52:44 -0000 On Sat, Jun 07, 2014 at 02:53:05PM -0700, Neel Natu wrote: > Hi Steve, > > On Sat, Jun 7, 2014 at 2:28 PM, Peter Grehan wrote: > > Hi Steve, > > > > > >> I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running > >> FreeBSD CURRENT, r266947. I've gotten this message about 12 times since > >> boot: > >> > >> interrupt storm detected on "irq268:"; throttling interrupt source > >> > >> vmstat -i shows: > >> > >> irq268: ahci1 236514222 839 > >> > >> ahci1 is the second disk connected to the system: > >> > >> ahci1: mem 0xc0002400-0xc00027ff irq 18 > >> at device 4.0 on pci0 > >> > >> The VM itself runs poudriere and was building a bunch of packages. At the > >> moment, the VM seems to be in a rather odd state. The poudriere jails are > >> running, but not doing anything. Ideas? > > > > > > Is this an 8.* host ? I don't believe AHCI has MSI support on that version, > > and AHCI legacy interrupts haven't had a huge amount of testing under load. > > > > If it is 8.*, I'd recommend using virtio-blk for the block device until we > > can work out what's going wrong. > > > > The KTR trace would be useful to figure out what's happening. > > To do that you can compile the host kernel and vmm.ko with the > following options: > options KTR > options KTR_MASK=(KTR_GEN) > options KTR_ENTRIES=(4*1024*1024) > > And when you see the interrupt storm message in the guest you can execute: > sudo ktrdump -cto /tmp/ktrdump.out > This was added to the kernel config, the kernel was built and installed and the host was rebooted. I started the VM and almost as soon as the workload was put back, the messages started again. So, I ran the ktrdump command above. I'll mail you privately with the location of the output. Thanks, Steve From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 18:56:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 031B4B63; Wed, 11 Jun 2014 18:56:36 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22170287B; Wed, 11 Jun 2014 18:56:34 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id u10so95405lbd.8 for ; Wed, 11 Jun 2014 11:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8pKE9HiuPvhKU2zujnAsjeRGxu/3KEypiPBsZRMqPTQ=; b=wx7jtQziIOBlGyF88xaooPwM/JnhyHgysXpwj+80dxr4WpJid3EQrtpO0xcCr08aIO P7+As9Rg3BZrXYn4fBZRIGOPYO//qBqS7KIeXXGVIE3qviAecvFsFdyuz4rIBqO+cmyG ZI5TXTlfuqS5ou4VojIdytN+2oAvRLPIgESy4BsKG7xErJKABcVeYFleZJD0pCAJ7Dst 6pOf7lFuFp1Xp6hFbdAfVN38wbphM2pCzzYzTTbCdl9yH8evXHroWnNYpl5v8T8b2LgY L2xXmdFVJhDtXtOEBNsL5OqXwGqnM2NwMdeneKuJrXhqZxjgrTrV66Q3N4yK0IoFtdgU bmkQ== MIME-Version: 1.0 X-Received: by 10.112.129.135 with SMTP id nw7mr51908lbb.89.1402512992942; Wed, 11 Jun 2014 11:56:32 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 11:56:32 -0700 (PDT) In-Reply-To: <20140611182101.GD2855@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> Date: Wed, 11 Jun 2014 11:56:32 -0700 X-Google-Sender-Auth: NswZkeS0f7_sp_agdBO02X2c80Y Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:56:36 -0000 On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy wrote: > > Thanks for the analysis! I'll think what would be a proper fix for that > problem. > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > and started to see the same problem. So, as a temporary fix, could you > check if you have bridge support available (in kernel or via module)? > > Bridges are needed to get networking anyway. Hi, My output of kldstat is: 1 29 0xffffffff80200000 19d6218 kernel 2 1 0xffffffff81bd7000 2e44b0 zfs.ko 3 2 0xffffffff81ebc000 8208 opensolaris.ko 4 1 0xffffffff81ec5000 4d68 nmdm.ko 5 1 0xffffffff81eca000 1a2110 vmm.ko 6 1 0xffffffff82211000 53e7 fdescfs.ko 7 1 0xffffffff82217000 9aff if_bridge.ko 8 1 0xffffffff82221000 5851 bridgestp.ko 9 1 0xffffffff82227000 5673 if_tap.ko 10 1 0xffffffff8222d000 2a94 uhid.ko 11 1 0xffffffff82230000 3592 ums.ko so I do have if_bridge.ko loaded. If you are doing development on CURRENT, make sure that you have WITH_LLDB=yes in /etc/src.conf when you rebuild the system. Looks like gdb in base cannot debug the cores. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 19:27:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84AA457D; Wed, 11 Jun 2014 19:27:30 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C2E82B96; Wed, 11 Jun 2014 19:27:29 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id pv20so120323lab.13 for ; Wed, 11 Jun 2014 12:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=TEvaFTXrP5NiWlUZ7wBDhy7D9Mhb3V+3Ga8Xm6sjeGY=; b=YraM6fIw6W4qVjOJM1l6SYIevPwEJHliqA5O7EWavc2+B4MAaBaanP1lifR2toqFHl rSmcRQOTrP9yBAZMvSS2MThHxn/ue7jYVYh9m14ksnOug4DENuBrLIbobPnnkoDNMj2v P7zp6/yNPFqFRJwh+KSz09LJWfJI7B5wuZ4ordcytBUpnteisngHoLHb1xiLhS5Pq4oU 3jiyHrUGQfFtjc0ql/2XyG5TeJNMasuYGKw2yj8Ro27Vjf3qZL7StfsvZmmBtZEvkuR/ InW1FQsDDEDT89FpWJZCFnGLAV1vSQH9T4Kpmv6kEziNb3Zoew41cYW31DrG5LW3UP8h mMKA== X-Received: by 10.152.23.136 with SMTP id m8mr29195227laf.2.1402514847517; Wed, 11 Jun 2014 12:27:27 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id bg4sm25730885lbb.6.2014.06.11.12.27.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 12:27:26 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 23:25:17 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611192514.GE2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lkTb+7nhmha7W+c3" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:27:30 -0000 --lkTb+7nhmha7W+c3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy w= rote: > > > > Thanks for the analysis! I'll think what would be a proper fix for that > > problem. > > > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > > and started to see the same problem. So, as a temporary fix, could you > > check if you have bridge support available (in kernel or via module)? > > > > Bridges are needed to get networking anyway. >=20 >=20 > Hi, >=20 > My output of kldstat is: >=20 > 1 29 0xffffffff80200000 19d6218 kernel > 2 1 0xffffffff81bd7000 2e44b0 zfs.ko > 3 2 0xffffffff81ebc000 8208 opensolaris.ko > 4 1 0xffffffff81ec5000 4d68 nmdm.ko > 5 1 0xffffffff81eca000 1a2110 vmm.ko > 6 1 0xffffffff82211000 53e7 fdescfs.ko > 7 1 0xffffffff82217000 9aff if_bridge.ko > 8 1 0xffffffff82221000 5851 bridgestp.ko > 9 1 0xffffffff82227000 5673 if_tap.ko > 10 1 0xffffffff8222d000 2a94 uhid.ko > 11 1 0xffffffff82230000 3592 ums.ko >=20 >=20 > so I do have if_bridge.ko loaded. I've attached a patch which should fix the segfault. Could you please let me know if it fixes the problem? However, we still need to figure out why networking is not being enabled. The root cause of that is that libvirt should start the default network when the daemon starts up. This network should create a bridge device (virbr0). Later on, when one starts a guest, it should create tap device and add it to the bridge. Obviously, it fails at this step as there's no bridge device. Could you please send me config.log file and also a port directory with your fixes so I could test it as well? As for your previous question on my setup, I'm using master branch of the libvirt git repo and my configure args are: $ ./configure --with-bhyve --without-polkit --with-hal \ CFLAGS=3D"-g -O0 -I/usr/local/include" LDFLAGS=3D-L/usr/local/lib \ --with-xml-catalog-file=3D/usr/local/share/xml/catalog I'm working on 10-STABLE. However, it's really bad if it works only this way. :-( I would love to fix the port. > If you are doing development on CURRENT, make sure that you have > WITH_LLDB=3Dyes in /etc/src.conf when you rebuild the system. > Looks like gdb in base cannot debug the cores. Thanks for the hint! Roman Bogorodskiy --lkTb+7nhmha7W+c3 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmK0aAAoJEMltX/4IwiJqlIcH/2yf8/YdVZY2SYbQy6dSkrKD ZK+b3hs0Q4ja6WFefdKbwpQvXq6d8O1+6bLtjNj9IV58o+7Nx/4ZLV+uZvhgfniX 6DekqwLYPtpbsSS0sfL2rnxwY0NQaIkxp6LzTbLp/kCiXiiG+FyEHqq0g3VImtuo xWJdgqck3JiFjmkOF/QfPJGoaavgNF3M729AqNkYgTTlbODUTcyoH3Mfg8kcZ8/m ksmLnFZghPjaONtiG1dCyanRewFaNI4RyRw2arGfZBl0uWtv0VY6cGyLocPMotAC j1nA1Z7iZVDlPqDNG5dHLBPVj7QccHTc7IyZS3kZFK5BozSwoz+8tOAjMyWrgGI= =tZZz -----END PGP SIGNATURE----- --lkTb+7nhmha7W+c3-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 19:28:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4FDA56CD; Wed, 11 Jun 2014 19:28:26 +0000 (UTC) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F5752BAC; Wed, 11 Jun 2014 19:28:25 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id n15so116493lbi.33 for ; Wed, 11 Jun 2014 12:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Dch8AmgtcJagUb5CBsQanX5oPCOCTV7Kqp0SjwF9/to=; b=mslAJMZBON3ABtLW3P0k0lrUd52pBdESMnw7UG6kCgdbZlCpkFjphiQOWv+grSHyyk uhsya2EJoIYFJSCJlr+h8L2qMiyQPR7GvV0tTqO7EJUmeicIojzA2ESnQlvx9NpDw+42 1AuZiLwBF7fszYLCpP3xNARj84EE6Ytet9EGs1THHKB0cSoxXigHrdKZhG1N4Bk2KSTS BKbER9conaz9kOJp/Ce3QDXD/l2qerwJIbBySYSubelRMuDv7krz96iCzIpPWMW9gnYw RXwwB+72pi34XiFS5PVJaanzTefFW2fVUeGko7sU7OwrxEvSneeEXE4a1pFM8JcEe9y6 zl6Q== X-Received: by 10.152.6.74 with SMTP id y10mr29012815lay.11.1402514903380; Wed, 11 Jun 2014 12:28:23 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id g3sm22329057lam.18.2014.06.11.12.28.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 12:28:21 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 23:26:13 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611192611.GF2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pE2VAHO2njSJCslu" Content-Disposition: inline In-Reply-To: <20140611192514.GE2855@kloomba> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:28:26 -0000 --pE2VAHO2njSJCslu Content-Type: multipart/mixed; boundary="JcvBIhDvR6w3jUPA" Content-Disposition: inline --JcvBIhDvR6w3jUPA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Roman Bogorodskiy wrote: > Craig Rodrigues wrote: >=20 > > On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy = wrote: > > > > > > Thanks for the analysis! I'll think what would be a proper fix for th= at > > > problem. > > > > > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > > > and started to see the same problem. So, as a temporary fix, could you > > > check if you have bridge support available (in kernel or via module)? > > > > > > Bridges are needed to get networking anyway. > >=20 > >=20 > > Hi, > >=20 > > My output of kldstat is: > >=20 > > 1 29 0xffffffff80200000 19d6218 kernel > > 2 1 0xffffffff81bd7000 2e44b0 zfs.ko > > 3 2 0xffffffff81ebc000 8208 opensolaris.ko > > 4 1 0xffffffff81ec5000 4d68 nmdm.ko > > 5 1 0xffffffff81eca000 1a2110 vmm.ko > > 6 1 0xffffffff82211000 53e7 fdescfs.ko > > 7 1 0xffffffff82217000 9aff if_bridge.ko > > 8 1 0xffffffff82221000 5851 bridgestp.ko > > 9 1 0xffffffff82227000 5673 if_tap.ko > > 10 1 0xffffffff8222d000 2a94 uhid.ko > > 11 1 0xffffffff82230000 3592 ums.ko > >=20 > >=20 > > so I do have if_bridge.ko loaded. >=20 > I've attached a patch which should fix the segfault. Could you please > let me know if it fixes the problem? >=20 > However, we still need to figure out why networking is not being > enabled. The root cause of that is that libvirt should start the default > network when the daemon starts up. This network should create a bridge > device (virbr0). >=20 > Later on, when one starts a guest, it should create tap device and add > it to the bridge. Obviously, it fails at this step as there's no bridge > device. >=20 > Could you please send me config.log file and also a port directory with > your fixes so I could test it as well? >=20 > As for your previous question on my setup, I'm using master branch of the > libvirt git repo and my configure args are: >=20 > $ ./configure --with-bhyve --without-polkit --with-hal \ > CFLAGS=3D"-g -O0 -I/usr/local/include" LDFLAGS=3D-L/usr/local/lib \ > --with-xml-catalog-file=3D/usr/local/share/xml/catalog >=20 > I'm working on 10-STABLE. >=20 > However, it's really bad if it works only this way. :-( I would love to > fix the port. >=20 > > If you are doing development on CURRENT, make sure that you have > > WITH_LLDB=3Dyes in /etc/src.conf when you rebuild the system. > > Looks like gdb in base cannot debug the cores. >=20 > Thanks for the hint! Sorry, forgot the actual attachment. :-( Roman Bogorodskiy --JcvBIhDvR6w3jUPA Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="libvirt_no_bridge.diff" Content-Transfer-Encoding: quoted-printable diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c index 91a8731..ba7a7ec 100644 --- a/src/bhyve/bhyve_command.c +++ b/src/bhyve/bhyve_command.c @@ -44,7 +44,7 @@ bhyveBuildNetArgStr(const virDomainDef *def, virCommandPt= r cmd, bool dryRun) virDomainNetDefPtr net =3D NULL; char *brname =3D NULL; char *realifname =3D NULL; - int *tapfd =3D NULL; + int tapfd =3D -1; char macaddr[VIR_MAC_STRING_BUFLEN]; =20 if (def->nnets !=3D 1) { @@ -80,7 +80,7 @@ bhyveBuildNetArgStr(const virDomainDef *def, virCommandPt= r cmd, bool dryRun) =20 if (!dryRun) if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net-= >mac, - def->uuid, tapfd, 1, + def->uuid, &tapfd, 1, virDomainNetGetActualVirtPo= rtProfile(net), virDomainNetGetActualVlan(n= et), VIR_NETDEV_TAP_CREATE_IFUP = | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { diff --git a/src/bhyve/bhyve_process.c b/src/bhyve/bhyve_process.c index 9f02a49..5e7d603 100644 --- a/src/bhyve/bhyve_process.c +++ b/src/bhyve/bhyve_process.c @@ -77,10 +77,12 @@ bhyveNetCleanup(virDomainObjPtr vm) int actualType =3D virDomainNetGetActualType(net); =20 if (actualType =3D=3D VIR_DOMAIN_NET_TYPE_BRIDGE) { - ignore_value(virNetDevBridgeRemovePort( - virDomainNetGetActualBridgeName(net), - net->ifname)); - ignore_value(virNetDevTapDelete(net->ifname)); + if (net->ifname) { + ignore_value(virNetDevBridgeRemovePort( + virDomainNetGetActualBridgeName(net), + net->ifname)); + ignore_value(virNetDevTapDelete(net->ifname)); + } } } } --JcvBIhDvR6w3jUPA-- --pE2VAHO2njSJCslu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmK1TAAoJEMltX/4IwiJqn8UIAIUivJv+T4xtI4WegKxiSITj oKCnk55jxrhO4maLg6mZcDNTp1U3zAFIpl1+YKMyk5D9Z4hJ1xTYUu6lyNxBZk/B H9eKiC51PO7Kn0RQE3/Bv79S3pLg5ltszbvWnemCLFX4v7+QLDXgL4+ldCN9+qLc AHZ3XlCqLIXg6NZ9C44pQbXYravoUdrTGCKHiYp4pEdRXRkUJwx/kbHXB0YBdLJH 9YZDcN7VhcC/o3ky33L2u3UjDnfl4+CnIvR6JSPGOurreyuhaHi906glZws2vQA+ 5fsdqSPeDXoaTnLRO+KlO8ECwl6x/VPZsFxXOMFNMq2K2pmIbkEBipQcOmH306c= =1S7U -----END PGP SIGNATURE----- --pE2VAHO2njSJCslu-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 22:49:55 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DFE54C8; Wed, 11 Jun 2014 22:49:55 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B5482034; Wed, 11 Jun 2014 22:49:53 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so238926lan.2 for ; Wed, 11 Jun 2014 15:49:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=srDrHyDOdSr4lgOvwHUesFw/nm/htOcTUD4zu1+na0M=; b=sv3LR2tww4V4mnWv8deo00sJjp0gmhBUWQItvQGplyRSf3FdFKinXBDWgaHzNiTjig Xc1xrcbKVXQgBXccCMfF3f4bc0wv4cwtiaezHMgCDztdcvvi3ZyuwXWJSx39ln9zFjs7 xicLRyDAU/53LN8o9Jlhb+9gT47oYkl32lD3L0rVG8DMoRfz3V086ufYR0aZPvS8Jrgs T9lByMqVH9e7dhkroauYDInKNI0UMD1QIuslz5tpy68w4af8wG1jBQtkfg6QITHl98fE 7WWwfxyEVV7Ttrb6niQXWM9ej82QBGKTibiBBC7BDIgssTrGJXCQKYxu0EHywwwFTFoN Gn5Q== MIME-Version: 1.0 X-Received: by 10.112.143.132 with SMTP id se4mr3847414lbb.53.1402526991435; Wed, 11 Jun 2014 15:49:51 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 15:49:51 -0700 (PDT) In-Reply-To: <20140611192514.GE2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> Date: Wed, 11 Jun 2014 15:49:51 -0700 X-Google-Sender-Auth: NivgK4Tr2vRCDeIfCncnV8vYRLw Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 22:49:55 -0000 On Wed, Jun 11, 2014 at 12:25 PM, Roman Bogorodskiy wrote: > I've attached a patch which should fix the segfault. Could you please > let me know if it fixes the problem? I have incorporated your patch into the libvirt port, and provided a tarball of the port here: http://people.freebsd.org/~rodrigc/libvirt_port.tgz I can confirm that your patch eliminates the coredump for me. So now I can do the following: (1) Start libvirtd: /usr/local/sbin/libvirtd (2) Start virsh: virsh -c "bhyve:///system" (3) Execute following inside virsh: define bhyve.xml start bhyve ( using bhyve.xml file from http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html ) In the libvirtd console, I see this: 2014-06-11 22:47:16.357+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-11 22:47:16.357+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-11 22:47:16.357+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-11 22:47:21.050+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-11 22:47:21.063+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-11 22:47:21.168+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-11 22:47:21.168+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-11 22:47:43.469+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet18: Invalid argument 2014-06-11 22:47:43.573+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 For the last line, it looks like it is trying to do bhyvectl --destroy --vm=bhyve on a VM which doesn't exist, i.e. there is no entry in /dev/vmm/bhyve It should ignore the error at this point and try to spawn the VM, but it fails. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 08:03:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC8FB4CC; Thu, 12 Jun 2014 08:03:16 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D39DB2CF6; Thu, 12 Jun 2014 08:03:15 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id q8so463816lbi.40 for ; Thu, 12 Jun 2014 01:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=0yq7A4cxJhqwVjEsUUsGX9VUhL2ycxxql7zvwxw8ji4=; b=ijvMpHd0io9j5o8QPYs19TlZmm8jsztvBqsCehN+yrz4SNb7V6K5fyJQY3jq32VdA4 iBEUTj73N3MgRfN+GefxMmcN3R7o/bDz/3PwTKHCzkjbdau8Hm9Wz3c+QeaWfmH7U2bQ Z0CxIFyJrSaSsJbEE8T9k/TxRQ1HA4gtTnPRKiZ27v6HQ75In3JL50D5ucnaCCDpns7z 6OMzqhTGv2I3bKpPdb3OdmqV4+lsdhIwgb2wQ/b8veazszYdbLOGzWi6cMctj8tzpyWs XpDmU6SRGT37LJAIjTrEv2ESPoT2D97+s2+WI1iQ+AWzxXAqPaM3yow5HcDUkB2RQaTc wDOg== X-Received: by 10.152.199.39 with SMTP id jh7mr31523168lac.18.1402560193797; Thu, 12 Jun 2014 01:03:13 -0700 (PDT) Received: from kloomba ([217.65.223.250]) by mx.google.com with ESMTPSA id 6sm1508783laz.6.2014.06.12.01.03.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jun 2014 01:03:12 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 12 Jun 2014 12:00:59 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140612080057.GA93486@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="x+6KMIRAuhnl3hBn" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 08:03:16 -0000 --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 12:25 PM, Roman Bogorodskiy w= rote: > > I've attached a patch which should fix the segfault. Could you please > > let me know if it fixes the problem? >=20 > I have incorporated your patch into the libvirt port, and provided a tarb= all > of the port here: >=20 > http://people.freebsd.org/~rodrigc/libvirt_port.tgz >=20 > I can confirm that your patch eliminates the coredump for me. >=20 > So now I can do the following: >=20 > (1) Start libvirtd: > /usr/local/sbin/libvirtd >=20 > (2) Start virsh: > virsh -c "bhyve:///system" >=20 > (3) Execute following inside virsh: > define bhyve.xml > start bhyve >=20 > ( using bhyve.xml file from > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/00258= 8.html > ) >=20 > In the libvirtd console, I see this: >=20 > 2014-06-11 22:47:16.357+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-11 22:47:16.357+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-11 22:47:16.357+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-11 22:47:21.050+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-11 22:47:21.063+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-11 22:47:21.168+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-11 22:47:21.168+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-11 22:47:43.469+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet18: > Invalid argument > 2014-06-11 22:47:43.573+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=3Dbhyve) unexpected exit status 255 >=20 >=20 >=20 > For the last line, it looks like it is trying to do bhyvectl --destroy > --vm=3Dbhyve > on a VM which doesn't exist, i.e. there is no entry in /dev/vmm/bhyve >=20 > It should ignore the error at this point and try to spawn the VM, but it = fails. This actually happens because it fails to spawn a VM (because there's no bridge) and tries to make sure to unload previously loaded VM in order not to leave it in a stale state. I have adjusted the port: http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz With this setup, I'm able to get networking (e.g. virsh net-list works) and updated the fix for the previous problem. I was able to start a VM with that setup. Roman Bogorodskiy --x+6KMIRAuhnl3hBn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmV45AAoJEMltX/4IwiJq9FgIAKHhXqh6lCsayHuBn3pWAFXL xqu9ND5I7CtQfKRtDFzMqGKPmca/mYUsvycw6GBiVPWiAWjKK6EPocZ6mr4joI3f v0Yt4w+d6HB1sFCVSg76sBKUSxIYW/wk7CO6MhqLcPvKi4XwBhXP59+2VdIP3hQJ Am8OysrgLOGhjJg0GjpY7hwHgJnKsgcvQztktg/4+dNtScs6cb498DY1WgnpJBdI 3CwefU1n8sHm8yxAQNHE8sD11kUeYAxDqn8YUaBUZH5GvKom3KsFLKYAkDdfdZoa 5FQFmmZtX1iwodIT6wHfK2aRJqH2sidQOf+xkakQOPEvaMUZPzrfScwjoLKj0Ho= =K62g -----END PGP SIGNATURE----- --x+6KMIRAuhnl3hBn-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 18:28:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 683B4B10; Thu, 12 Jun 2014 18:28:38 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8264029A9; Thu, 12 Jun 2014 18:28:37 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id w7so943038lbi.29 for ; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=KT5Xy5y8HC+5Aa4yiFtbfuon91wVek4GWFN3gJLJ99M=; b=Fy0+BCfaUgK2QzuLi6EpftnSxWJmRcI4RxAg29sKVn7xvl6YT9OCo9rn2hUZHla80t JLJU/UQYB055WaF65bc6VHJXp/D+HFnB9vzFk0eOtAlfn5aL5Cf+h9QczUcMK3uSUL0I O3I5AOmIjdjZRZHB2f4ErLQ4NnhNM/HPSi2KdJuqK1/blZrN82Noev/Nt+dXVxxtZVur MIxLfbMahzIW44KocBAo0kE9HRy8A1I2Zo0QbIgB67LIlRzrb45bAS4dt+yFI6Q4g+kv e4Bde0b80n9XwN72BpMdMQcQErRLoFkqw/0Lf9gKpfjqm+YMbvXhodkIxXBJmC90W8Zj KKwg== MIME-Version: 1.0 X-Received: by 10.112.50.2 with SMTP id y2mr2420479lbn.66.1402597715344; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) In-Reply-To: <20140612080057.GA93486@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> Date: Thu, 12 Jun 2014 11:28:35 -0700 X-Google-Sender-Auth: JDwrMbCpR8oomfCGRbjjyGPF47c Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 18:28:38 -0000 On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: > > http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz > > With this setup, I'm able to get networking (e.g. virsh net-list works) > and updated the fix for the previous problem. I was able to start a VM > with that setup. I deleted the old port from my system and took your modified port, built it, and installed it. I followed my previous steps in: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html and got this: 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-12 18:23:54.328+0000: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-12 18:23:54.328+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-12 18:23:54.328+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-12 18:23:54.439+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-12 18:23:54.460+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-12 18:23:54.461+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-12 18:23:54.560+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-12 18:23:54.560+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-12 18:24:17.940+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: Invalid argument 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 18:33:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 969D2D6D; Thu, 12 Jun 2014 18:33:58 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA5442A54; Thu, 12 Jun 2014 18:33:57 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so945752lbj.3 for ; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=MG7LfJTB2HGlnQ9xXGx8xtCy+EFBUJulGI93O1NfYWU=; b=lIVzMDUVetbHNCK5c0dQFQ7w0d0vnGbiEw9Q99cVqsxqxxB72RS0/c0zB6o1RpHyhZ 1XhQj3cgYcHAbHDw8J60otIOe/869lJGb350yOqd8UYgqV/Q5kOXSIFmHWFid+GMkozj eX8mXsz4Q+7qYViZqbROpGlXT8nkO5Olpc9QzOW74aA5kMw9/vW3dQZfi/GTa/1rb19n iX3NVEm7vpRl0+AflfEkcvVFHU/hLjvxlUwJ2Pzb8as5wy6/mD+Opv0JqW8meSUnwLh0 t6sqgmrqmh196wJEK4Q8aB3ZXftPRRO1W/sC1IR1vxE621w9erH3renknR1cNCk6A0wd e13w== MIME-Version: 1.0 X-Received: by 10.152.43.43 with SMTP id t11mr41769lal.89.1402598035459; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) In-Reply-To: References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> Date: Thu, 12 Jun 2014 11:33:55 -0700 X-Google-Sender-Auth: 47XyXvW-qtAqbzqWzwGzqzb467Y Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 18:33:58 -0000 On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: >> >> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz >> >> With this setup, I'm able to get networking (e.g. virsh net-list works) >> and updated the fix for the previous problem. I was able to start a VM >> with that setup. > > > I deleted the old port from my system and took your modified port, > built it, and installed it. > > I followed my previous steps in: > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html > and got this: > > 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-12 18:23:54.328+0000: 34485605376: error : > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > No such file or directory > 2014-06-12 18:23:54.328+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-12 18:23:54.328+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-12 18:23:54.439+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-12 18:23:54.460+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-12 18:23:54.461+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-12 18:23:54.560+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-12 18:23:54.560+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-12 18:24:17.940+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: > Invalid argument > 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=bhyve) unexpected exit status 255 > > > -- > Craig After the program fails, these are my tap and bridge devices, as shown by ifconfig: bridge0: flags=8843 metric 0 mtu 1500 ether 02:29:45:c7:8f:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 20000 member: tap2 flags=143 ifmaxaddr 0 port 7 priority 128 path cost 2000000 member: tap1 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap0 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 tap0: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:53:27:00:00 nd6 options=29 media: Ethernet autoselect status: no carrier tap1: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:59:27:00:01 nd6 options=29 media: Ethernet autoselect status: no carrier tap2: flags=8943 metric 0 mtu 1500 options=80000 ether 00:bd:5e:27:00:02 nd6 options=29 media: Ethernet autoselect status: active Opened by PID 1506 vnet0: flags=8802 metric 0 mtu 1500 options=80000 ether fe:54:00:33:20:8c nd6 options=21 media: Ethernet autoselect status: no carrier This is in my /etc/rc.conf for creating bridge and tap devices on bootup: ##################################################### # Create tap devices, one tap interface per BHyve VM. # Add the tap interfaces to bridge0 #################################################### cloned_interfaces="bridge0 tap0 tap1 tap2" autobridge_interfaces="bridge0" autobridge_bridge0="tap* em0" -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 19:03:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35247A7B; Thu, 12 Jun 2014 19:03:02 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 452B32DE8; Thu, 12 Jun 2014 19:03:01 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id u10so952375lbd.24 for ; Thu, 12 Jun 2014 12:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=kkn1bENRNXvAUpmFKYaAOigpj4IMxP8ifSDMNQd8a8I=; b=WFHazADP99dAYZQKwCEWfjLsCHWl9Pi4nuDZ9jnbnk9D2BV/c0taz7LQ0qjITUj4Rd TmBVi2caAqjOEZU+/JiOLRge4vUtMC4/YxZ/nFhy5JyfwkNAW9hFIPAAy5tcBi28thkR 0qHpwqhsdJyiY4RVTsB5tlboce2SsyFyI7ZctiW17uiqWRHavBbLB00VHzWa/pZsa9SG A90UPRpfGs2lfJphf4M3OEYs4QPTqP7zznKqUmzckrkK0MrbkIsCWloFTkBQ5lfYXYho SRkuJhRiukLd+zLZPbNyZV6H/cqEVWlz00hyV/ts663wOtXRiMMo9b3RZoz0cbET/6aq MAJA== X-Received: by 10.152.234.229 with SMTP id uh5mr2923590lac.56.1402599779011; Thu, 12 Jun 2014 12:02:59 -0700 (PDT) Received: from kloomba ([217.65.223.250]) by mx.google.com with ESMTPSA id d8sm25043689lah.12.2014.06.12.12.02.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jun 2014 12:02:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 12 Jun 2014 23:00:43 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140612190042.GA2858@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 19:03:02 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues w= rote: > > On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy = wrote: > >> > >> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz > >> > >> With this setup, I'm able to get networking (e.g. virsh net-list works) > >> and updated the fix for the previous problem. I was able to start a VM > >> with that setup. > > > > > > I deleted the old port from my system and took your modified port, > > built it, and installed it. > > > > I followed my previous steps in: > > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002= 588.html > > and got this: > > > > 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 > > 2014-06-12 18:23:54.328+0000: 34485605376: error : > > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > > No such file or directory > > 2014-06-12 18:23:54.328+0000: 34485605376: info : > > networkReloadFirewallRules:1750 : Reloading iptables rules > > 2014-06-12 18:23:54.328+0000: 34485605376: info : > > networkRefreshDaemons:1722 : Refreshing network daemons > > 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : > > Cannot find 'pm-is-supported' in path: No such file or directory > > 2014-06-12 18:23:54.439+0000: 34485605376: warning : > > virQEMUCapsInit:948 : Failed to get host power management capabilities > > 2014-06-12 18:23:54.460+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/var/run/libvirt/qemu > > 2014-06-12 18:23:54.461+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/etc/libvirt/qemu > > 2014-06-12 18:23:54.560+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/etc/libvirt/bhyve > > 2014-06-12 18:23:54.560+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > > 2014-06-12 18:24:17.940+0000: 34485598208: error : > > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: > > Invalid argument > > 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 > > : internal error: Child process (/usr/sbin/bhyvectl --destroy > > --vm=3Dbhyve) unexpected exit status 255 > > > > > > -- > > Craig >=20 >=20 > After the program fails, these are my tap and bridge devices, as shown > by ifconfig: >=20 > bridge0: flags=3D8843 metric 0 mt= u 1500 > ether 02:29:45:c7:8f:00 > nd6 options=3D9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: em0 flags=3D143 > ifmaxaddr 0 port 1 priority 128 path cost 20000 > member: tap2 flags=3D143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: tap1 flags=3D143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap0 flags=3D143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > tap0: flags=3D8902 metric 0 mtu 1500 > options=3D80000 > ether 00:bd:53:27:00:00 > nd6 options=3D29 > media: Ethernet autoselect > status: no carrier > tap1: flags=3D8902 metric 0 mtu > 1500 > options=3D80000 > ether 00:bd:59:27:00:01 > nd6 options=3D29 > media: Ethernet autoselect > status: no carrier > tap2: flags=3D8943 > metric 0 mtu 1500 > options=3D80000 > ether 00:bd:5e:27:00:02 > nd6 options=3D29 > media: Ethernet autoselect > status: active > Opened by PID 1506 > vnet0: flags=3D8802 metric 0 mtu 1500 > options=3D80000 > ether fe:54:00:33:20:8c > nd6 options=3D21 > media: Ethernet autoselect > status: no carrier >=20 >=20 > This is in my /etc/rc.conf for creating bridge and tap devices on bootup: >=20 > ##################################################### > # Create tap devices, one tap interface per BHyve VM. > # Add the tap interfaces to bridge0 > #################################################### > cloned_interfaces=3D"bridge0 tap0 tap1 tap2" >=20 > autobridge_interfaces=3D"bridge0" > autobridge_bridge0=3D"tap* em0" Ah. Actually, libvirt manages bridges and taps on its own. So, the preferred flow is that libvirt starts its networks on startup automatically. By default it has a network with the 'virbr0' device which it creates on its own. Later on it creates vnet devices (which are tap devices) automatically as well on VM startup and plugs them to the bridge. The configuration you have probably confuses it, could you consider trying without that? Also, do you have working 'net-list' command in virsh with my version of port? PS If you're available for some real-time conversation such as on IRC, I think some kind of things would be faster to discuss there. Thanks, Roman Bogorodskiy --fdj2RfSjLxBAspz7 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmfjaAAoJEMltX/4IwiJq4K8H/1o5UX77Kw3FO+CN/cCOt4H7 Aa06bB9vCSJrRK8qlc2HJgiXvQgC2IeczkcCJ4EVkc6tLSnSdSTtvJt6u1PKaAz1 varEsL5KMHhP9KSXfqZ7VJC4/EY/mnJRkWtUErZDnFA51i4wDEFZa7KFm19TxZHI oP3DLkG5PwkaB+9WccuPWyNKYNwtvzYVAXsjyZKlfmC0bw5/ebWE9VWppXNoC2Ui guDLQGo8HYa+tpxSZg11xs4Obn9aQufn7idPs6/2FV+aTiPmI7UCYs2HA3WaO3fh BPGpQiEcG7rNQbjQ6xfsquQNv689zxm0RPK7ZeMvbtvn4dsDI9Xl7uANGYWv5Q0= =Bs7F -----END PGP SIGNATURE----- --fdj2RfSjLxBAspz7-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 20:49:39 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 327B836D for ; Thu, 12 Jun 2014 20:49:39 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E2E2B273F for ; Thu, 12 Jun 2014 20:49:38 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s5CKnSj8024606 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 12 Jun 2014 20:49:33 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s5CKnSH3024605 for virtualization@freebsd.org; Thu, 12 Jun 2014 20:49:28 GMT (envelope-from swills) Date: Thu, 12 Jun 2014 20:49:28 +0000 From: Steve Wills To: virtualization@freebsd.org Subject: Re: interrupt storm on ahci Message-ID: <20140612204924.GA20784@mouf.net> References: <20140607212440.GB3163@mouf.net> <539383F3.2060307@freebsd.org> <20140611091816.GB61572@meatwad.mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140611091816.GB61572@meatwad.mouf.net> User-Agent: Mutt/1.5.22 (2013-10-16) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Thu, 12 Jun 2014 20:49:33 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 20:49:39 -0000 On Wed, Jun 11, 2014 at 09:18:17AM +0000, Steve Wills wrote: > On Sat, Jun 07, 2014 at 02:53:05PM -0700, Neel Natu wrote: > > Hi Steve, > > > > On Sat, Jun 7, 2014 at 2:28 PM, Peter Grehan wrote: > > > Hi Steve, > > > > > > > > >> I'm running a FreeBSD guest in bhyve on a FreeBSD host. Both are running > > >> FreeBSD CURRENT, r266947. I've gotten this message about 12 times since > > >> boot: > > >> > > >> interrupt storm detected on "irq268:"; throttling interrupt source > > >> > > >> vmstat -i shows: > > >> > > >> irq268: ahci1 236514222 839 > > >> > > >> ahci1 is the second disk connected to the system: > > >> > > >> ahci1: mem 0xc0002400-0xc00027ff irq 18 > > >> at device 4.0 on pci0 > > >> > > >> The VM itself runs poudriere and was building a bunch of packages. At the > > >> moment, the VM seems to be in a rather odd state. The poudriere jails are > > >> running, but not doing anything. Ideas? > > > > > > > > > Is this an 8.* host ? I don't believe AHCI has MSI support on that version, > > > and AHCI legacy interrupts haven't had a huge amount of testing under load. > > > > > > If it is 8.*, I'd recommend using virtio-blk for the block device until we > > > can work out what's going wrong. > > > > > > > The KTR trace would be useful to figure out what's happening. > > > > To do that you can compile the host kernel and vmm.ko with the > > following options: > > options KTR > > options KTR_MASK=(KTR_GEN) > > options KTR_ENTRIES=(4*1024*1024) > > > > And when you see the interrupt storm message in the guest you can execute: > > sudo ktrdump -cto /tmp/ktrdump.out > > > > This was added to the kernel config, the kernel was built and installed and the > host was rebooted. I started the VM and almost as soon as the workload was put > back, the messages started again. So, I ran the ktrdump command above. I'll > mail you privately with the location of the output. Replying to myself just to note for the record that the interrupt storm was harmless, because the throttling only reduced performance (I thought another issue I was having at the same time was a result, but it was just a coincidence) and that the default threshold for interrupt storms of 1000 interrupts per second may be too low in some cases, such as this one. I disabled the throttling by setting the hw.intr_storm_threshold sysctl to 0 and haven't had any issues. Steve From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 13 16:21:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14D471AE; Fri, 13 Jun 2014 16:21:09 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 382AC2C9D; Fri, 13 Jun 2014 16:21:08 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 65D5F1534D2; Fri, 13 Jun 2014 18:21:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nAHDWD0NCu0l; Fri, 13 Jun 2014 18:20:57 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372] (unknown [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372]) by smtp.digiware.nl (Postfix) with ESMTP id E7C521534C0; Fri, 13 Jun 2014 18:20:57 +0200 (CEST) Message-ID: <539B24E1.7040006@digiware.nl> Date: Fri, 13 Jun 2014 18:20:49 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Craig Rodrigues , Roman Bogorodskiy Subject: Re: libvirt and bhyve problems References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 16:21:09 -0000 On 2014-06-12 20:33, Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues wrote: >> On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: >>> >>> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz >>> >>> With this setup, I'm able to get networking (e.g. virsh net-list works) >>> and updated the fix for the previous problem. I was able to start a VM >>> with that setup. >> >> >> I deleted the old port from my system and took your modified port, >> built it, and installed it. >> >> I followed my previous steps in: >> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html >> and got this: >> >> 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 >> 2014-06-12 18:23:54.328+0000: 34485605376: error : >> dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: >> No such file or directory >> 2014-06-12 18:23:54.328+0000: 34485605376: info : >> networkReloadFirewallRules:1750 : Reloading iptables rules >> 2014-06-12 18:23:54.328+0000: 34485605376: info : >> networkRefreshDaemons:1722 : Refreshing network daemons >> 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : >> Cannot find 'pm-is-supported' in path: No such file or directory >> 2014-06-12 18:23:54.439+0000: 34485605376: warning : >> virQEMUCapsInit:948 : Failed to get host power management capabilities >> 2014-06-12 18:23:54.460+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/var/run/libvirt/qemu >> 2014-06-12 18:23:54.461+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/etc/libvirt/qemu >> 2014-06-12 18:23:54.560+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/etc/libvirt/bhyve >> 2014-06-12 18:23:54.560+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >> 2014-06-12 18:24:17.940+0000: 34485598208: error : >> virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: >> Invalid argument >> 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 >> : internal error: Child process (/usr/sbin/bhyvectl --destroy >> --vm=bhyve) unexpected exit status 255 >> >> >> -- >> Craig > > > After the program fails, these are my tap and bridge devices, as shown > by ifconfig: > > bridge0: flags=8843 metric 0 mtu 1500 > ether 02:29:45:c7:8f:00 > nd6 options=9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: em0 flags=143 > ifmaxaddr 0 port 1 priority 128 path cost 20000 > member: tap2 flags=143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: tap1 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap0 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > tap0: flags=8902 metric 0 mtu 1500 > options=80000 > ether 00:bd:53:27:00:00 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > tap1: flags=8902 metric 0 mtu > 1500 > options=80000 > ether 00:bd:59:27:00:01 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > tap2: flags=8943 > metric 0 mtu 1500 > options=80000 > ether 00:bd:5e:27:00:02 > nd6 options=29 > media: Ethernet autoselect > status: active > Opened by PID 1506 > vnet0: flags=8802 metric 0 mtu 1500 > options=80000 > ether fe:54:00:33:20:8c > nd6 options=21 > media: Ethernet autoselect > status: no carrier > > > This is in my /etc/rc.conf for creating bridge and tap devices on bootup: > > ##################################################### > # Create tap devices, one tap interface per BHyve VM. > # Add the tap interfaces to bridge0 > #################################################### > cloned_interfaces="bridge0 tap0 tap1 tap2" > > autobridge_interfaces="bridge0" > autobridge_bridge0="tap* em0" I claim to know nothing about libvirt..... But in my scripts I need to delay using the tap?? interfaces for 1-2 secs with a sleep. Because otherwise my bhyve VM fails. Could be the same here? --WjW From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 13 19:18:39 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 56C1BD0B; Fri, 13 Jun 2014 19:18:39 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75C422C8D; Fri, 13 Jun 2014 19:18:38 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id s18so1716539lam.4 for ; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OlXaqaK53DBy4/q7wAQFn3f6qRPnikclnCpqFCq6oK4=; b=0/UVCzYwp7eAx7xSkM7McqNSMu0+fK8EufiOGYBt6jHaT7LV3CJPx6ex+nGTxhfV4a FZZnIvHvmTBagodNpNhIbQre9G4Fj7cGKEP6TXF0Hwa5FbV0TFYw3mQVkJUDt5ZlAvL4 K/WEpcnClrk1iHMU1bkE7uKIMf0+WIV/YgK0bsQNsxZBA1DeWxIrsEALk05jSStZLtUO 8PPg0tHmHk1tHFX05dk8RGTwuQ4/E+OyMU0ExqaGFtmOCvKbFbw57eX6APxrnisTCepp 6gn+yRb+ETWrfnCy5XvuQ7p/aSY6NabdettPvazmhO1KnI+XFrzyiej0GcjMLQNIVwVu j1EQ== MIME-Version: 1.0 X-Received: by 10.112.217.10 with SMTP id ou10mr2769615lbc.20.1402687116441; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) In-Reply-To: <20140612190042.GA2858@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> Date: Fri, 13 Jun 2014 12:18:36 -0700 X-Google-Sender-Auth: 8oqqzSOMKxju4urHTSRMYuXeI5A Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 19:18:39 -0000 On Thu, Jun 12, 2014 at 12:00 PM, Roman Bogorodskiy wrote: > > Ah. Actually, libvirt manages bridges and taps on its own. So, the > preferred flow is that libvirt starts its networks on startup > automatically. By default it has a network with the 'virbr0' device > which it creates on its own. > > Later on it creates vnet devices (which are tap devices) automatically > as well on VM startup and plugs them to the bridge. > > The configuration you have probably confuses it, could you consider > trying without that? OK, I removed all autobridge and cloned_interfaces entries from my rc.conf and rebooted. I am still using your version of the libvirt port. I tried the steps again and got this: 2014-06-13 19:09:10.482+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-13 19:09:10.482+0000: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-13 19:09:10.482+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-13 19:09:10.482+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-13 19:09:10.593+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-13 19:09:10.593+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-13 19:09:10.615+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-13 19:09:10.615+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-13 19:09:10.716+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-13 19:09:10.716+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-13 19:09:25.938+0000: 34485598208: error : virNetDevTapCreate:385 : Unable to create tap device: Invalid argument 2014-06-13 19:09:26.063+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 If I do ifconfig, I have no bridge or tap devices. > > Also, do you have working 'net-list' command in virsh with my version of > port? I get this: virsh # net-list Name State Autostart Persistent ---------------------------------------------------------- > > PS If you're available for some real-time conversation such as on IRC, I > think some kind of things would be faster to discuss there. You can private mail me some times when you are available. I am busy the next few days, but might have a few hours on Sunday. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 13 22:31:13 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0EE22B7; Fri, 13 Jun 2014 22:31:13 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1A952D1F; Fri, 13 Jun 2014 22:31:12 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s5DMNBR8098589 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 13 Jun 2014 18:23:12 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: consistent VM hang during reboot From: John Nielsen In-Reply-To: <83DA2398-0004-49EC-8AC1-9AA64F33A194@jnielsen.net> Date: Fri, 13 Jun 2014 16:23:13 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0238084D-FD0F-42A5-85F5-597A590E666C@jnielsen.net> References: <201405081303.17079.jhb@freebsd.org> <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> <83DA2398-0004-49EC-8AC1-9AA64F33A194@jnielsen.net> To: "freebsd-hackers@freebsd.org" , "freebsd-virtualization@freebsd.org" X-Mailer: Apple Mail (2.1878.2) X-DCC-MGTINTERNET-Metrics: ns1.jnielsen.net 1170; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 22:31:13 -0000 On May 13, 2014, at 9:50 AM, John Nielsen wrote: > On May 9, 2014, at 12:41 PM, John Nielsen wrote: >=20 >> On May 8, 2014, at 12:42 PM, Andrew Duane wrote: >>=20 >>> From: owner-freebsd-hackers@freebsd.org = [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of John Nielsen >>>=20 >>>> On May 8, 2014, at 11:03 AM, John Baldwin wrote: >>>>=20 >>>>> On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >>>>>> I am trying to solve a problem with amd64 FreeBSD virtual = machines running on a Linux+KVM hypervisor. To be honest I'm not sure if = the problem is in FreeBSD or=20 >>>>> the hypervisor, but I'm trying to rule out the OS first. >>>>>>=20 >>>>>> The _second_ time FreeBSD boots in a virtual machine with more = than one core, the boot hangs just before the kernel would normally = print e.g. "SMP: AP CPU #1=20 >>>>> Launched!" (The last line on the console is "usbus0: 12Mbps Full = Speed USB v1.0", but the problem persists even without USB). The VM will = boot fine a first time,=20 >>>>> but running either "shutdown -r now" OR "reboot" will lead to a = hung second boot. Stopping and starting the host qemu-kvm process is the = only way to continue. >>>>>>=20 >>>>>> The problem seems to be triggered by something in the SMP portion = of cpu_reset() (from sys/amd64/amd64/vm_machdep.c). If I hit the virtual = "reset" button the next=20 >>>>> boot is fine. If I have 'kern.smp.disabled=3D"1"' set for the = initial boot then subsequent boots are fine (but I can only use one CPU = core, of course). However, if I=20 >>>>> boot normally the first time then set 'kern.smp.disabled=3D"1"' = for the second (re)boot, the problem is triggered. Apparently something = in the shutdown code is=20 >>>>> "poisoning the well" for the next boot. >>>>>>=20 >>>>>> The problem is present in FreeBSD 8.4, 9.2, 10.0 and 11-CURRENT = as of yesterday. >>>>>>=20 >>>>>> This (heavy-handed and wrong) patch (to HEAD) lets me avoid the = issue: >>>>>>=20 >>>>>> --- sys/amd64/amd64/vm_machdep.c.orig 2014-05-07 = 13:19:07.400981580 -0600 >>>>>> +++ sys/amd64/amd64/vm_machdep.c 2014-05-07 17:02:52.416783795 = -0600 >>>>>> @@ -593,7 +593,7 @@ >>>>>> void >>>>>> cpu_reset() >>>>>> { >>>>>> -#ifdef SMP >>>>>> +#if 0 >>>>>> cpuset_t map; >>>>>> u_int cnt; >>>>>>=20 >>>>>> I've tried skipping or disabling smaller chunks of code within = the #if block but haven't found a consistent winner yet. >>>>>>=20 >>>>>> I'm hoping the list will have suggestions on how I can further = narrow down the problem, or theories on what might be going on. >>>>>=20 >>>>> Can you try forcing the reboot to occur on the BSP (via 'cpuset -l = 0 reboot') >>>>> or a non-BSP ('cpuset -l 1 reboot') to see if that has any effect? = It might >>>>> not, but if it does it would help narrow down the code to = consider. >>>>=20 >>>> Hello jhb, thanks for responding. >>>>=20 >>>> I tried your suggestion but unfortunately it does not make any = difference. The reboot hangs regardless of which CPU I assign the = command to. >>>>=20 >>>> Any other suggestions? >>>=20 >>> When I was doing some early work on some of the Octeon multi-core = chips, I encountered something similar. If I remember correctly, there = was an issue in the shutdown sequence that did not properly halt the = cores and set up the "start jump" vector. So the first core would start, = and when it tried to start the next ones it would hang waiting for the = ACK that they were running (since they didn't have a start vector and = hence never started). I know MIPS, not AMD, so I can't say what the = equivalent would be, but I'm sure there is one. Check that part, setting = up the early state. >>>=20 >>> If Juli and/or Adrian are reading this: do you remember anything = about that, something like 2 years ago? >>=20 >> That does sound promising, would love more details if anyone can = provide them. >>=20 >> Here's another wrinkle: >>=20 >> The KVM machine in question is part of a cluster of identical servers = (hardware, OS, software revisions). The problem is present on all = servers in the cluster. >>=20 >> I also have access to a second homogenous cluster. The OS and = software revisions on this cluster are identical to the first. The = hardware is _nearly_ identical--slightly different mainboards from the = same manufacturer and slightly older CPUs. The same VMs (identical disk = image and definition, including CPU flags passed to the guest) that have = a problem on the first cluster work flawlessly on this one. >>=20 >> Not sure if that means the bad behavior only appears on certain CPUs = or if it's timing-related or something else entirely. I'd welcome = speculation at this point. >>=20 >> CPU details below in case it makes a difference. >>=20 >> =3D=3D Problem Host =3D=3D >> model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr = pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand = lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi = flexpriority ept vpid fsgsbase smep erms >>=20 >> =3D=3D Good Host =3D=3D >> model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr = pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe = syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good = nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 = monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 = sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat = epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid >=20 > Still haven't found a solution but I did learn something else = interesting: an ACPI reboot allows the system to come back up = successfully. What is different from the system or CPU point of view = about an ACPI reboot versus running "reboot" or "shutdown" from = userland? Following up on the off chance anyone else is interested. I installed = -HEAD on a host that was having the problem ("v2" Xeon CPU) and ran a = FreeBSD 9 VM under bhyve. The problem did _not_ persist. That's not = entirely conclusive but it does point the finger at Qemu a bit more = strongly. I have filed a bug with them: https://bugs.launchpad.net/qemu/+bug/1329956 Still, if anyone has any ideas on what could be going on I'd love to = hear them. JN From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 14 16:35:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D4B0984; Sat, 14 Jun 2014 16:35:18 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1E1A2CD0; Sat, 14 Jun 2014 16:35:16 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id E0EB91534D2; Sat, 14 Jun 2014 18:35:07 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cVgYOY419Bs7; Sat, 14 Jun 2014 18:35:04 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:11c2:bed9:c648:c447] (unknown [IPv6:2001:4cb8:3:1:11c2:bed9:c648:c447]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 329DB153434; Sat, 14 Jun 2014 18:35:04 +0200 (CEST) Message-ID: <539C79B8.6040104@digiware.nl> Date: Sat, 14 Jun 2014 18:35:04 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Peter Grehan Subject: Re: Bheve: Slow linux syscalls on AMD References: <5388B56D.1000501@digiware.nl> <20140530212900.6B74B6BF@hub.freebsd.org> <53891BC1.9050701@digiware.nl> <53891EAA.8040105@freebsd.org> <538927FC.3000903@digiware.nl> <53939AE2.80804@freebsd.org> <539434B6.6020804@digiware.nl> <5394817F.4030203@digiware.nl> <53948DA5.4040000@digiware.nl> <53962141.8070702@freebsd.org> <5396D18C.8000401@digiware.nl> <53970F5A.8060900@freebsd.org> <53971C3A.30406@digiware.nl> <539748AA.5060005@freebsd.org> In-Reply-To: <539748AA.5060005@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2014 16:35:18 -0000 On 10-6-2014 20:04, Peter Grehan wrote: > Hi Willem, > >> I've got KTR compiled in, but last time I switched it on. >> I got swamped in traffic, and I sort of got locked out of the >> server... :( >> Could also be because I was writing it to a file as well. >> >> So you'll have to help/tell me what to do. > > For looking at this, I'd use one single-vCPU guest to avoid KTR noise > from other vCPU/guests. The problem shows up with this so no issue there. The trace consists mainly of large number of sequeces like: (ktrdump -trc) ---- 204 1 37052 vm ubuntu-14.04[0]: pending intr 65 203 1 39489 vm ubuntu-14.04[0]: VMEXIT halted CPU. 202 1 53836 vm ubuntu-14.04[0]: SVM:Guest in interrupt shadow. 201 1 69072 vm ubuntu-14.04[0]: SVM:Enter vmrun RIP:0xffffffff8104f596 inst len=0/15 200 1 37218 vm ubuntu-14.04[0]: pending intr 65 199 1 39534 vm ubuntu-14.04[0]: VMEXIT halted CPU. ---- intermixed with blocks: ---- 13485 1 65347 vm ubuntu-14.04[0]: SVM:event injected,vector=65. 13484 1 41457 vm ubuntu-14.04[0]: vlapic_update_ppr 0x40 13483 1 53987 vm ubuntu-14.04[0]: vlapic_intr_accepted isr7 0x00000000 13482 1 52734 vm ubuntu-14.04[0]: vlapic_intr_accepted isr6 0x00000000 13481 1 52729 vm ubuntu-14.04[0]: vlapic_intr_accepted isr5 0x00000000 13480 1 52784 vm ubuntu-14.04[0]: vlapic_intr_accepted isr4 0x00000000 13479 1 52755 vm ubuntu-14.04[0]: vlapic_intr_accepted isr3 0x00000000 13478 1 52840 vm ubuntu-14.04[0]: vlapic_intr_accepted isr2 0x00000002 13477 1 52709 vm ubuntu-14.04[0]: vlapic_intr_accepted isr1 0x00000000 13476 1 52669 vm ubuntu-14.04[0]: vlapic_intr_accepted isr0 0x00000000 13475 1 52722 vm ubuntu-14.04[0]: vlapic_intr_accepted irr7 0x00000000 13474 1 52694 vm ubuntu-14.04[0]: vlapic_intr_accepted irr6 0x00000000 13473 1 52701 vm ubuntu-14.04[0]: vlapic_intr_accepted irr5 0x00000000 13472 1 52705 vm ubuntu-14.04[0]: vlapic_intr_accepted irr4 0x00000000 13471 1 52692 vm ubuntu-14.04[0]: vlapic_intr_accepted irr3 0x00000000 13470 1 52884 vm ubuntu-14.04[0]: vlapic_intr_accepted irr2 0x00000000 13469 1 52839 vm ubuntu-14.04[0]: vlapic_intr_accepted irr1 0x00000000 13468 1 53055 vm ubuntu-14.04[0]: vlapic_intr_accepted irr0 0x00000000 ---- So it looks like the VM goes halted, but interrupt 65 tries to wake it. Dmesg however does not give a irq 65 in the guest. wjw@ubuntu-14:~$ dmesg | grep irq [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.000000] nr_irqs_gsi: 40 [ 0.000000] NR_IRQS:16640 nr_irqs:256 16 [ 0.321489] hpet: hpet2 irq 40 for MSI [ 0.748675] 00:01: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 0.770813] 00:02: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 0.778181] virtio-pci 0000:00:02.0: irq 41 for MSI/MSI-X [ 0.778203] virtio-pci 0000:00:02.0: irq 42 for MSI/MSI-X [ 0.778224] virtio-pci 0000:00:02.0: irq 43 for MSI/MSI-X [ 1.393123] ahci 0000:00:03.0: irq 44 for MSI/MSI-X [ 1.424971] ata1: SATA max UDMA/133 abar m1024@0xc0002000 port 0xc0002100 irq 44 [ 1.435304] ahci 0000:00:04.0: irq 45 for MSI/MSI-X [ 1.456767] ata7: SATA max UDMA/133 abar m1024@0xc0002400 port 0xc0002500 irq 45 A larger ktrdump is at: http://smtp.digiware.nl/tmp/ktrdump.log --WjW From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 15 15:22:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71920AEF; Sun, 15 Jun 2014 15:22:33 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CF552387; Sun, 15 Jun 2014 15:22:32 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id 10so1390934lbg.1 for ; Sun, 15 Jun 2014 08:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7+0oghu3OKdcsfe2AkLRQsU4DA8kCON5PeA+yZy0RM8=; b=QjT+FiQz1Gk5cHLRJWSlqimyGYl2LQxvigwkGO7re2P1R0vDtMfIiYyf9z2E8p1WSm ZEg0+DUCSnMeQnxl48OS1oU3dAnT6Y1QyjcRGRkBU2F6E+erf/fN+QPxyxaLO6eXQbUg 8+RJNSRUxIAmNQ4I7i9ktBFPZMBES3se5Dejx4Mq3DMipDWpQJFUIfoD+jHMgmNFv9ta i/EacWdxw1gZ34/PxwXSJij7iZDwpMPc53UYfUmEUzVLOZErDaMThMnhzXfkOn4wV7O8 FcEOBksD0pda5qlFma5ASb5gn/MYhNIllH6ghNBQN8zhOJI9dWx2tUujXd3e+S1Ay3OW rY4Q== X-Received: by 10.152.36.134 with SMTP id q6mr10507893laj.29.1402845750233; Sun, 15 Jun 2014 08:22:30 -0700 (PDT) Received: from kloomba ([95.104.132.224]) by mx.google.com with ESMTPSA id bg4sm6414133lbb.6.2014.06.15.08.22.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jun 2014 08:22:29 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 15 Jun 2014 19:20:09 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wac7ysb48OaltWcw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 15:22:33 -0000 --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 12:00 PM, Roman Bogorodskiy w= rote: > > > > Ah. Actually, libvirt manages bridges and taps on its own. So, the > > preferred flow is that libvirt starts its networks on startup > > automatically. By default it has a network with the 'virbr0' device > > which it creates on its own. > > > > Later on it creates vnet devices (which are tap devices) automatically > > as well on VM startup and plugs them to the bridge. > > > > The configuration you have probably confuses it, could you consider > > trying without that? >=20 > OK, I removed all autobridge and cloned_interfaces entries > from my rc.conf and rebooted. I am still using your version of the > libvirt port. >=20 > I tried the steps again and got this: >=20 > 2014-06-13 19:09:10.482+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-13 19:09:10.482+0000: 34485605376: error : > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > No such file or directory > 2014-06-13 19:09:10.482+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-13 19:09:10.482+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-13 19:09:10.593+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-13 19:09:10.593+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-13 19:09:10.615+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-13 19:09:10.615+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-13 19:09:10.716+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-13 19:09:10.716+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-13 19:09:25.938+0000: 34485598208: error : > virNetDevTapCreate:385 : Unable to create tap device: Invalid argument > 2014-06-13 19:09:26.063+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=3Dbhyve) unexpected exit status 255 >=20 >=20 > If I do ifconfig, I have no bridge or tap devices. >=20 >=20 > > > > Also, do you have working 'net-list' command in virsh with my version of > > port? >=20 > I get this: > virsh # net-list > Name State Autostart Persistent > ---------------------------------------------------------- >=20 I have updated the port: http://people.freebsd.org/~novel/misc/libvirt_port2.tgz Changes are: * added RUN_DEPENDS on dnsmasq (required for networking) * added installation on network definition files Hopefully this will allow to bring up network on start. When the daemon starts, virsh should show the following: virsh # net-list Name State Autostart Persistent ---------------------------------------------------------- default active yes yes virsh #=20 If that's not the case, then something is still going wrong. And yes, qemu support should be enabled (networks defs seem to be installed with qemu support). Even if it doesn't work, and 'net-list --all' is empty, we could try to do it manually (but still, if it's the case, then it's something wrong with the port that needs to be fixed): in virsh # net-define /usr/local/etc/libvirt/qemu/networks/default.xml # net-start default Roman Bogorodskiy --wac7ysb48OaltWcw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTnbmoAAoJEMltX/4IwiJqJSYH/iB/li8gPsiD4tnzO7Wi7TAp hVxaStC3I2cMYfPxnKp/76dwDKzJDqveNTFSXr02gh6khQ4YilZ0h+Vi1jGcPfgH JKMWEKT7I/mcl6gkEOZr9Dwb9NKJQci+6RSaMMBkV7cjAweOByFlOz68tPJ/hV2O oiZbJj8uNA+KkbKEUYLxb/Digpo+mvdFlPWDXxtTMb02vW3J8eHnUUTy4EpvNo0r wWReT7ri57yG/LHcO/oGiK1QZ4OjeUBibD1iEKAXltX9kiN6XOK3+J/IZGKx0Ch5 hpBW9RK9g7adosdr4AZSVMqmsw6hi1mvkRJGcb+TnNym5pSEqwhQfx1OA1CWlj8= =TAba -----END PGP SIGNATURE----- --wac7ysb48OaltWcw-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 15 19:04:34 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CE55488; Sun, 15 Jun 2014 19:04:34 +0000 (UTC) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 816512822; Sun, 15 Jun 2014 19:04:33 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id w7so645812lbi.11 for ; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JoHVFedMMb5TE8byogy1mGIBDKWk30bv8MAw/jhTjNo=; b=hTkwuONQY9hzV3YFy5c8ygrdPv1HHeOr+f8w/pgon+bIludv9XrzXQJmrhueE8CUWz 8NVrghmTgPtg9WWsnC2+SSCdQslLQPNg4N7rTu6u2sUeLfYLWXduvBwbgQox2CHhv0el /1M4RcS4e0oqCPqN7Yya99rrlj+iF/QyQ+dQpGHaw4hbLmxY7h39Fj3u3iAB0B8dPECZ 22Zup23MAvU3AIeB3KCp7QIs5tSbhkh1+j3UaQ2csM3Rf2irbENps9H+8V6YQd7EsSV5 t59ZwsKbctKLE7MSw7Wyvl0UOm3ERwbsKCclsgDuIpbbgIx+IOe1JjD0k+qQNVzN8KFI RxGg== MIME-Version: 1.0 X-Received: by 10.112.132.233 with SMTP id ox9mr10274187lbb.35.1402859071356; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) In-Reply-To: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> Date: Sun, 15 Jun 2014 12:04:31 -0700 X-Google-Sender-Auth: ilov8IrBjf_fJXAV9aETyXwMMbE Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 19:04:34 -0000 On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz OK, I deinstall the old port, took your port, built it, and installed it. When I started libvirtd, I got this: 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+0000: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' Trying to start bhyve did not work. After some investigation, I did the following: service local_unbound stop Then I started libvirtd again: 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+0000: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' At this point, I could see the virbr0 device: virbr0: flags=8843 metric 0 mtu 1500 ether 02:b8:18:5f:0f:00 inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 I was then able to do: virsh -c 'bhyve:///system' start bhyve and start a VM. This is good progress! Once thing I observed is that if I killed the libvirtd process, it did not clean up the dnsmasq process that it started: /usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf and a subsequent restart of libvirtd would fail. The devel/libvirt port is looking a lot better with your changes. Can you push these changes into the port? I filed bugs 190767 and 190783, but did not get any response from the port maintainer, jgh@. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 16 06:44:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26A20A34; Mon, 16 Jun 2014 06:44:19 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DEC12CD8; Mon, 16 Jun 2014 06:44:18 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so2124367lab.22 for ; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=60b/mZEKfhsOeWiaBPKnwOQRRossbb9/NKT4QBNRVbM=; b=lC1DlHTis9GiJ5Aqztei3sxY7ju6VwYJ13dBa8jhgJ0CqAEJc/7DmPEsQ48wMO4bEw xHUaRI8sKWBmaZly9Y1gJg321G6WcijlFndMZzfBPeV/ht5NWckrV3i3Zh3q30hrFXy3 6tMqejLkA7RMEtchwgvpDpil9Q3LTdOlwZqWfB10+ZCRvXrvfF4no5M/3Zm3kyEBkFxr DdH2THIK7CXJFQO89tKbgSe+hBGE3S8ARaKuaYWfB1rT8IHIhihjVQc2ny24KJlbHylt 4yb8Jqjhb6BP1EWeQGn/eEf++8J3LQmrx1DSIMsWbqy3922gDldB/zSWrLmD1SGt8g7c Y2DA== X-Received: by 10.112.17.102 with SMTP id n6mr12120948lbd.39.1402901056011; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id s15sm7882189lbp.42.2014.06.15.23.44.14 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jun 2014 23:44:15 -0700 (PDT) Sender: Roman Bogorodskiy Date: Mon, 16 Jun 2014 10:44:07 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140616064406.GA41585@dev.san.ru> References: <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 06:44:19 -0000 Craig Rodrigues wrote: > On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wr= ote: > > > > I have updated the port: > > > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz >=20 >=20 > OK, I deinstall the old port, took your port, built it, and installed it. >=20 > When I started libvirtd, I got this: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 >=20 > Trying to start bhyve did not work. >=20 > After some investigation, I did the following: >=20 > service local_unbound stop >=20 >=20 > Then I started libvirtd again: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 > At this point, I could see the virbr0 device: >=20 > virbr0: flags=3D8843 metric 0 mtu= 1500 > ether 02:b8:18:5f:0f:00 > inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255 > nd6 options=3D1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >=20 >=20 > I was then able to do: >=20 > virsh -c 'bhyve:///system' > start bhyve >=20 > and start a VM. > This is good progress! >=20 > Once thing I observed is that if I killed the libvirtd process, > it did not clean up the dnsmasq process that it started: >=20 > /usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf >=20 > and a subsequent restart of libvirtd would fail. I think it's intentional not to kill dnsmasq. There are persistent and transient objects in libvirt (such as domains and networks), and persistent objects live forever, while transient objects live until manually destroyed or host is restarted. So libvirt doesn't kill persistent VMs and, I guess, it doesn't stop dnsmasq to keep persistent networks functional. It should not affect restart. I'll fail to start the network, but as it's here already, we're fine. BTW, I'm still yet to write the code to catch up previously started bhyve VMs between libvirtd restarts... > The devel/libvirt port is looking a lot better with your changes. > Can you push these changes into the port? >=20 > I filed bugs 190767 and 190783, but did not get any response > from the port maintainer, jgh@. I'll send the patch, but I need to fix some issues first. I noticed that check-orphans complains about the orphaned libvirtd.conf file. Also, I need to have a poudriere run as well to make sure I didn't miss anything else. Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 17 08:40:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EECAD2; Tue, 17 Jun 2014 08:40:19 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D34C23C8; Tue, 17 Jun 2014 08:40:18 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so3747014lan.30 for ; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Yg18n1LWiug4JZ6goXoPxkFrwkRNFaqMht8aFi4lHr4=; b=bn0XBcbiohMmE4HAcHny28hMDbc7zceHkwszONHwgQVYGs761C2ncRp5PReNodwVZD xA5D9oRRYtXVvH5XybUqQsNJRo1hLRjvM/mkc6X/bkErH/1bZKft495sIsf6zAH07KXA JdhJB1FtcO/UdZSqv/c/+oEkjwBVvPCwz5BY6Orj1I+3mZNrCMVQpVIiqQikPt7H929w MizjfkPLZ2DVVHa+Fo68gI5VSCdAXF4CUsl/a45mTSPcgm21eCHTWsRzJJfKIQA5vqCt JqiNkiiCqdtdS+pM39s9+VzW8KUam+I7lc6nYJVyRL++zlkmnXNyf9IJ5scmf9CNa7ol KuCw== MIME-Version: 1.0 X-Received: by 10.152.87.20 with SMTP id t20mr297582laz.79.1402994416383; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) In-Reply-To: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> Date: Tue, 17 Jun 2014 01:40:16 -0700 X-Google-Sender-Auth: LzSnJe6ArS6YQVDTBYwfNUGeucA Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 08:40:19 -0000 On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz > > Changes are: > > * added RUN_DEPENDS on dnsmasq (required for networking) > * added installation on network definition files Hi, dnsmasq needs to be in BUILD_DEPENDS *and* RUN_DEPENDS for this port. I did a test on my system where I deleted the dnsmasq port, and then built the libvirt port. During the configure stage of this port, the #define DNSMASQ variable is expanded and written to a config.h. If the dnsmasq port is not around during the build stage of the libvirt port, then this variable is not expanded properly, and when the libvirt port is fully built, running libvirtd will not start dnsmasq, because the path to dnsmasq was not written properly to config.h. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 17 16:59:17 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76FB3D5A; Tue, 17 Jun 2014 16:59:17 +0000 (UTC) Received: from secure.freebsdsolutions.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3B413246A; Tue, 17 Jun 2014 16:59:16 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by secure.freebsdsolutions.net (8.14.4/8.14.4) with ESMTP id s5HGx6rx056955 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 17 Jun 2014 12:59:07 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: consistent VM hang during reboot From: John Nielsen In-Reply-To: <0238084D-FD0F-42A5-85F5-597A590E666C@jnielsen.net> Date: Tue, 17 Jun 2014 10:59:14 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <7D960087-1839-4295-AEE3-5D8F60D9D710@jnielsen.net> References: <201405081303.17079.jhb@freebsd.org> <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> <83DA2398-0004-49EC-8AC1-9AA64F33A194@jnielsen.net> <0238084D-FD0F-42A5-85F5-597A590E666C@jnielsen.net> To: "freebsd-hackers@freebsd.org" , "freebsd-virtualization@freebsd.org" X-Mailer: Apple Mail (2.1878.2) X-DCC-sonic.net-Metrics: ns1.jnielsen.net 1117; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 16:59:17 -0000 On Jun 13, 2014, at 4:23 PM, John Nielsen wrote: > On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote: >> I am trying to solve a problem with amd64 FreeBSD virtual machines = running on a Linux+KVM hypervisor. To be honest I'm not sure if the = problem is in FreeBSD or the hypervisor, but I'm trying to rule out the = OS first. >>=20 >> The _second_ time FreeBSD boots in a virtual machine with more than = one core, the boot hangs just before the kernel would normally print = e.g. "SMP: AP CPU #1 Launched!" (The last line on the console is = "usbus0: 12Mbps Full Speed USB v1.0", but the problem persists even = without USB). The VM will boot fine a first time, but running either = "shutdown -r now" OR "reboot" will lead to a hung second boot. Stopping = and starting the host qemu-kvm process is the only way to continue. ... > Following up on the off chance anyone else is interested. I installed = -HEAD on a host that was having the problem ("v2" Xeon CPU) and ran a = FreeBSD 9 VM under bhyve. The problem did _not_ persist. That's not = entirely conclusive but it does point the finger at Qemu a bit more = strongly. I have filed a bug with them: > https://bugs.launchpad.net/qemu/+bug/1329956 With some help from the Qemu and KVM folks I've finally made some = headway. The salient difference between the working and non-working CPUs = above seems to be support for APIC virtualization. Loading the intel_kvm = module (on the Linux host) with "enable_apicv=3DN" works around the = reboot problem I've been having. Since this now looks like a Linux KVM bug I won't follow up here any = more, but I wanted to wrap up the thread for the archives. JN From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 18 18:30:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7BA9206; Wed, 18 Jun 2014 18:30:00 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1860C2236; Wed, 18 Jun 2014 18:29:59 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id s7so782572lbd.18 for ; Wed, 18 Jun 2014 11:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=6ZVE1+gzi/k2Xm0oQYwd5JPwNgHhTs8pUIi2XeMbftQ=; b=QamOCtHWxKEmADmRCLhmY8KD8q7L3KDbVn/qIevsdFBrLPCP9oTz7T9RzytI/mvJCX sBFNJwI8xLWF7EWmXnRfA1OwYWiwhEqxInYfzFa6mdrSrPjDfbRPlIbQn1SxOawu3tHN V9HvhqeNGifu8fpAc5C0vSBSTmiI8CQ4I+85evur20GUMvXD1D+mu+WApy59mQpmlWC3 t3wJq8X/JUtG/nkYR8d9xbApCiHsOi9rAlcMzZsHIvlwf0iX7iJk60FZ/O5R6hYudPij 70aCAImhd7kvXy/5HqumofP3beGT/vh8IWVv9UIeRI4udeyDfbfp5axsIgIznCoGF6SC eS1g== MIME-Version: 1.0 X-Received: by 10.112.35.14 with SMTP id d14mr2226385lbj.43.1403116197889; Wed, 18 Jun 2014 11:29:57 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 18 Jun 2014 11:29:57 -0700 (PDT) Date: Wed, 18 Jun 2014 11:29:57 -0700 X-Google-Sender-Auth: zuHHGwTQuAdzNWrP7RtdFPdxHrY Message-ID: Subject: Request for Testers: libvirt and bhyve From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: Roman Bogorodskiy X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 18:30:00 -0000 Hi, I've had some success with getting libvirt and bhyve to work. I wrote up a list of the steps that I followed here: http://people.freebsd.org/~rodrigc/libvirt-bhyve/libvirt-bhyve.html If anyone is interested, please try it out, and provide feedback to this list. It would be great to get more testers. Roman, when do you think you will be able to commit all your latest fixes into the devel/libvirt port Makefile? It would greatly increase the usability of the port. Thanks for doing all the work to add bhyve support to libvirt!! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 18 19:46:55 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F187303 for ; Wed, 18 Jun 2014 19:46:55 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BFA5F29A4 for ; Wed, 18 Jun 2014 19:46:54 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id pv20so864888lab.9 for ; Wed, 18 Jun 2014 12:46:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=KJWhwpy3JkVM6CeKUEz1DlSXnWgqKGkmaWhw+y9XgQI=; b=Jg1rkmjK/bz6cyS8IUU1Xe84cSbrwBDcC8dr7jH2EW+engCJ9zbUbipN6JI2eCxweZ TVS7hTVK/uQ5ptYKhTfvu37sGEyCsKTYNYgQE2ZQn9Qsgv1O74oD5zFgL01PyYIZ6QN3 ZUO4Sy52V3XiN0U/5YoRbz0jg8oxqJXTkBRDyvIItu3g90itg0DGp6DRL1PXLPdk5zSJ a51HyL14BAyA4ozK+61LDdyp486i9zN6ugdjeuzSFd9BlMM/R57X2SKrhymQjmODzAZp Z9F3ndn4XaFkSq7CMXlytt3N8UBCbTNvGiQ/tqqXxFdFsxEhtlka1VCHVg77hfzPlOVi O7cw== MIME-Version: 1.0 X-Received: by 10.152.183.193 with SMTP id eo1mr45756lac.95.1403120812411; Wed, 18 Jun 2014 12:46:52 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 18 Jun 2014 12:46:52 -0700 (PDT) Date: Wed, 18 Jun 2014 12:46:52 -0700 X-Google-Sender-Auth: 8-QNo3qX-G34_8K3xs06cS2Ba8E Message-ID: Subject: Request for testers: virt-manager, libvirt, and bhyve From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 19:46:55 -0000 Hi, I created a patch for the deskutils/virt-manager port to add support for bhyve. virt-manager depends on libvirt, so the work that Roman has done with libvirt is critical for it to work. I have instructions for setting it up here, plus a screenshot of the virt-manager UI: http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html Is there anyone interested in testing this and providing feedback? Also, since virt-manager is in Python, is there anyone out there interested in improving this patch, and sending it upstream to the virt-manager developers? -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 00:29:08 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0BABBDB for ; Thu, 19 Jun 2014 00:29:08 +0000 (UTC) Received: from mail.bmfms.ro (mail.anotr.ro [194.176.115.131]) by mx1.freebsd.org (Postfix) with ESMTP id 7AE802183 for ; Thu, 19 Jun 2014 00:29:07 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.bmfms.ro (Postfix) with ESMTP id C16F2C8AA5C for ; Thu, 19 Jun 2014 03:19:24 +0300 (EEST) X-Virus-Scanned: amavisd-new at bmfms.ro Received: from mail.bmfms.ro ([127.0.0.1]) by localhost (mail.bmfms.ro [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ensU1-+1XO0i for ; Thu, 19 Jun 2014 03:19:21 +0300 (EEST) Received: from [192.168.0.104] (unknown [108.61.43.200]) by mail.bmfms.ro (Postfix) with ESMTP id B57A9C8AAEF for ; Thu, 19 Jun 2014 03:17:39 +0300 (EEST) Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Important Notice!!! To: freebsd-virtualization@freebsd.org From: "Help Desk" Date: Thu, 19 Jun 2014 08:17:32 +0800 Reply-To: help.desk014@tech-center.com Message-Id: <20140619001739.B57A9C8AAEF@mail.bmfms.ro> X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 00:29:08 -0000 Help Desk Attention Account User, Scheduled Maintenance & Upgrade Your account is in the process of being upgraded to a newest = Windows-based servers and an enhanced online email interface inline with in= ternet infrastructure Maintenance. The new servers will provide better anti= -spam and anti-virus functions, along with IMAP Support for mobile devices = to enhance your usage. To ensure that your account is not disrupted but active during and after th= is upgrade, you are required to kindly confirm your account by stating the = details below: * Domain\user name: = * Password: = This will prompt the upgrade of your account. Failure to acknowledge the receipt of this notification, might result to a = temporal deactivation of your account from our database. Your account shall= remain active upon your confirmation of your login details. We do apologize for any inconvenience caused. Sincerely, Your Customer Care Team (c) Copyright 2014, All Rights Reserved. From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 05:47:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55DBD402; Thu, 19 Jun 2014 05:47:30 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3A6A2912; Thu, 19 Jun 2014 05:47:29 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id mc6so1127290lab.38 for ; Wed, 18 Jun 2014 22:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=v7/pFFFKGlJtcJ1QaATGq1LWHIn9mevRIbaNLCl//CA=; b=TefalwkAH5BKUSXG2PZy8qUR9PcVyYeCJZw9+yLRKFQX1V+PXsKN8rwupY/Llk9aap VpiSq01VX4LAzsE63R2sGnth1xbYl+fma24VxnEg1aHjcxM6fef/jPaVl0mAFArORL5r fQOMjFwAE3Plug1bpUwr7IpVlAFxKu/4fmA7yjO9z1EWKoI/dak55MFqENPTOU4/vEdV QsMQGPYKPreTGt2DhrqVIucxt8LlKEqU+dXGHwnySkNHCpvj7mf2fWXKULaOZDeZRFGo YiovwQvb6uPQbVQrQDDM8by3NC+MsCRmy1Oc8LecIGrcMU1mu4tM94CWEf6mK32xvQva m/LA== X-Received: by 10.112.168.227 with SMTP id zz3mr1893732lbb.0.1403156847439; Wed, 18 Jun 2014 22:47:27 -0700 (PDT) Received: from kloomba ([95.104.138.66]) by mx.google.com with ESMTPSA id to10sm3323944lbb.11.2014.06.18.22.47.26 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Jun 2014 22:47:26 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 19 Jun 2014 09:44:58 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: Request for testers: virt-manager, libvirt, and bhyve Message-ID: <20140619054456.GA1851@kloomba> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 05:47:30 -0000 --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > Hi, >=20 > I created a patch for the deskutils/virt-manager port to add support > for bhyve. virt-manager depends on libvirt, so the work > that Roman has done with libvirt is critical for it to work. >=20 > I have instructions for setting it up here, plus a screenshot of the > virt-manager UI: >=20 > http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html >=20 > Is there anyone interested in testing this and providing feedback? >=20 > Also, since virt-manager is in Python, is there anyone out there interest= ed > in improving this patch, and sending it upstream to the virt-manager > developers? Hi Craig, Back in April bhyve support was added into virt-manager master [1]. So it should be there in the next release. I've looked at updating the desktuils/virt-manager port to 1.0.x, but at that time it was blocked by some outdated dependency. 1: https://git.fedorahosted.org/cgit/virt-manager.git/commit/?id=3D05df5a64= 843f2bd4e9a5197d97608d41b2e6dc43 Roman Bogorodskiy --ew6BAiZeqk4r7MaW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTonjYAAoJEMltX/4IwiJquCYH/0I4sQjMGIFcmrBhG9KOKOuq c7OD1N3H2G39Fzh19rULvCI61TycIonTQ9ccHWVdPM7kZx2ZyNe/ScPXhB6+cOHn XwM4wFsS03oeYgnuBbvwQn1dgMmtRvkibIGdVLkdmBDO+L4pNQ0toRomy3kbDOON TGcbCaj1ZEmRtbClhoa+m7+S+7sd2zSBgIYTGkhXIA/u0oYlDpW0VENAHK2oampX JX3xiAuWhaK9/912DbEY6sMskwNnXY5Qs6xgO1RwMzMKjrjnF3/seuo/c7vfwlos lkq47tkQH9+ILmmtQzAzjCO97Cvr1twD5awQ9A24YP49XVCxvft3mWMZGrLV5hc= =vxvi -----END PGP SIGNATURE----- --ew6BAiZeqk4r7MaW-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 05:52:03 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0382E48E; Thu, 19 Jun 2014 05:52:03 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 511692997; Thu, 19 Jun 2014 05:52:02 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id el20so1084201lab.7 for ; Wed, 18 Jun 2014 22:52:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=jMJElDUebUnbwSwPbgjPtMZX2/1tQbBM712FgTEP+Fs=; b=BFEZaQ7hlhO3+XK0URNEYjXCmW8PhB6R1WXv5yQoeZzo0pQVoQde4eeczRUPrSvjQi +dF158Xy5ZE84yBWl01TCo15kw/la4uqOKpxY4P7zdHksQMnYXaiRzW5Ynah9TQwoI0+ 4tvnSrnJdqlxqc+G/ZBR/w3AH10Lb+qT7BVqjxtTeQ35duP2qWrgi2TuEmc6pGPT9Ki5 yRjqcpWyO1IPOmGRDNuMB8vMRY2w80dbEQbchgOeMkkp8ABBfU4xddMhDUUTqbts9ERZ p7AT2acVOFRb7YZdeUE3YzmK8HLtHV7foHdRr+0uzeVVopDK0ZW9VxJs4Fx4HTwIYgDq VQHQ== X-Received: by 10.112.50.2 with SMTP id y2mr331010lbn.66.1403157120118; Wed, 18 Jun 2014 22:52:00 -0700 (PDT) Received: from kloomba ([95.104.138.66]) by mx.google.com with ESMTPSA id 8sm1760829lao.5.2014.06.18.22.51.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Jun 2014 22:51:59 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 19 Jun 2014 09:49:36 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: Request for Testers: libvirt and bhyve Message-ID: <20140619054935.GB1851@kloomba> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zx4FCpZtqtKETZ7O" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 05:52:03 -0000 --zx4FCpZtqtKETZ7O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > Hi, >=20 > I've had some success with getting libvirt and bhyve to work. >=20 > I wrote up a list of the steps that I followed here: >=20 > http://people.freebsd.org/~rodrigc/libvirt-bhyve/libvirt-bhyve.html >=20 > If anyone is interested, please try it out, and provide feedback to this > list. It would be great to get more testers. >=20 > Roman, when do you think you will be able to commit all your latest > fixes into the devel/libvirt port Makefile? It would greatly increase > the usability of the port. Thanks for doing all the work to add bhyve > support to libvirt!! Hi! I hope to get a clean and ready patch for the port this weekend and will send it to jgh@. Thanks a lot for your testing and the feedback! Roman Bogorodskiy --zx4FCpZtqtKETZ7O Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTonnvAAoJEMltX/4IwiJqehAIAIauI8nc2xbxE7SkrMCurR/2 MMNklyL52yZ0WCdUyCW2OIhbxnO9n3Ri84mbxuF1LxEcklwoKWkvwtSbFQO7Tm5w ShshQa4+oFvq0OpPbBJxDWztVae/QHMBFQfej+D/ptg6EWglN8xn67MSxedS2gcY JJe8YlD2RkOwoc4mwHtpJPYHKd+dEIL+G/6st+vAF4DG/tjCK9aXP9rkdM0rn1+y BhJs2RcK+5VB2tPHwdmtlvS3ppsXD6Q/N+pLhtgREchHFQ9mSVsXCZS7cis2oSCm dR6mBjAVv6YONvbZnJs8jUKbQbdj03ZyyjBf/xu3H3U4whh9O12qHVx6STy82N0= =vAHB -----END PGP SIGNATURE----- --zx4FCpZtqtKETZ7O-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 14:01:45 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F8A43DA for ; Thu, 19 Jun 2014 14:01:45 +0000 (UTC) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C0E6323FA for ; Thu, 19 Jun 2014 14:01:44 +0000 (UTC) Received: from john by potato.growveg.org with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1Wxcbj-0002Gl-44 for freebsd-virtualization@freebsd.org; Thu, 19 Jun 2014 14:42:27 +0100 Date: Thu, 19 Jun 2014 14:42:27 +0100 From: John To: freebsd-virtualization@freebsd.org Subject: various virtualisation questions Message-ID: <20140619134227.GA7523@potato.growveg.org> Reply-To: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: john@potato.growveg.org X-SA-Exim-Scanned: No (on potato.growveg.org); SAEximRunCond expanded to false X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 14:01:45 -0000 Hello list, I've looked and looked for bhyve info and worked examples all in one place but my google-fu seems to be lacking today, so I'm asking here. I'm running stable-10 r262917 host. Can one now have the following bhyve instances? 1. openbsd (which versions?) 2. ubuntu latest 3. opensuse latest Can these be installed from the console? If so, how? I'm using virtualbox right now and it is very very very slow with freebsd guest. Are there optimisations I need for bhyve? The host runs zfs. There's lots of memory (192GB but lots of that seems swallowed by zfs!) 4. Can a virtualbox instance be converted to a bhyve instance? thanks! -- John From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 16:16:52 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29B24A45 for ; Thu, 19 Jun 2014 16:16:52 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 0345120A9 for ; Thu, 19 Jun 2014 16:16:51 +0000 (UTC) Received: from [10.1.1.2] (S01060001abad1dea.hm.shawcable.net [50.70.146.73]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 470A414506 for ; Thu, 19 Jun 2014 16:16:45 +0000 (UTC) Message-ID: <53A30D00.6020604@freebsd.org> Date: Thu, 19 Jun 2014 12:17:04 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: various virtualisation questions References: <20140619134227.GA7523@potato.growveg.org> In-Reply-To: <20140619134227.GA7523@potato.growveg.org> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7fDUdJC238XiFd1uSxrK5o6v8c9FbsAGM" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 16:16:52 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7fDUdJC238XiFd1uSxrK5o6v8c9FbsAGM Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-06-19 09:42, John wrote: > Hello list, >=20 > I've looked and looked for bhyve info and worked examples all in one pl= ace > but my google-fu seems to be lacking today, so I'm asking here. > I'm running stable-10 r262917 host. Can one now have the following bhyv= e=20 > instances? >=20 > 1. openbsd (which versions?) > 2. ubuntu latest > 3. opensuse latest >=20 > Can these be installed from the console? If so, how?=20 >=20 > I'm using virtualbox right now and it is very very very slow with freeb= sd=20 > guest. Are there optimisations I need for bhyve? The host runs zfs. The= re's=20 > lots of memory (192GB but lots of that seems swallowed by zfs!) >=20 > 4. Can a virtualbox instance be converted to a bhyve instance? >=20 > thanks! >=20 Firstly, for running FreeBSD, Ubuntu and Redhat, I have entries in the handbook that walk you through it: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization-= host-bhyve.html OpenSuse would just be a matter of changing the grub config as required. If the guest happens to use grub2, grub-bhyve can usually just pickup the config file and work without any heavy lifting on your part, compared to CentOS 6 which still uses grub1 and requires injecting a bunch of kernel boot parameters. You can run some versions of OpenBSD, but you need to use a 'flashrd' style image. However, recent work in -CURRENT now allows running all modern versions of OpenBSD using the standard ISO to do the install etc. If you install the qemu-img tool, you should be able to convert your VirtualBox image into a raw disk image, and then you would be able to boot that in bhyve. --=20 Allan Jude --7fDUdJC238XiFd1uSxrK5o6v8c9FbsAGM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTow0CAAoJEJrBFpNRJZKfdhoP/3jZZa5wIo5TyGmwi6dTmPnq NEGFvQrCvCL4FIbmdVefu6dIlUhJX2pihvTwgBtgOZqtpjlPuJDwUw7n9DVpN31L U+JMm6lrOP3EZaRlMe9aU59F2+C0yqV2CNDBTN34AGxiXc7PsRnDS464V48ca/rs 3LdkZhOpXqpTTdQLZr3xxFvBcUzcDlG+sk0acshU5JEhs+psTBJdSRebuy3STZi6 1d9W/6HdnQCa21GJdcgttHckQ+lvrClrlx3ZaiNrRGiGgXDDXHGuQHr9oMNQWLav cC1PrJToO9WOURv7WUi9/RfocQm+5ICowe2LNtmalvR372wnBQB8EnXacDsKcrzq PNSXrVHIBzgmJNynZbmSeDIT98WkSc54B4jWjqhZACcXhSGa0ruEHM4JUoO21dao xXH5jp8meTn6+ZjHInjZYaEOVsi1Be8s1Eex+jx8oz3bgf4o6vZyHUwR3V2RtvGl /d+BKKEbZ8LLzBNdvJzaQIsGtmfKqxc0AutAc/NvzhEar7p32mgpZdI2R4N8wOVy lFzKhXuPzfw0xok/cfCnNeKbqS1VC2v80SNdkInYv7afbmOTDPRMF/y0sxFQiomR nl8kI6sn39vLzGt6q0AQSzawl35l57bR/gfUL7Pm2ZuzvqBvJ05ER9uUL5kK3l2F cLYqehz9KTJXa4qCOJT1 =R4KZ -----END PGP SIGNATURE----- --7fDUdJC238XiFd1uSxrK5o6v8c9FbsAGM-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 16:24:53 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2ACADDD2; Thu, 19 Jun 2014 16:24:53 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5057A2162; Thu, 19 Jun 2014 16:24:52 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id s7so1610029lbd.16 for ; Thu, 19 Jun 2014 09:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VP+s9h/RkaR/zVQR4Png2POYiNdyNrrejHezAlyTZVs=; b=epnwugr8Pb/YwQQ0xEDmndSM/1HPQjXCa4pUgBuwSantbS8M65mdLpYhK+rOBgparg gDVAlLAdV4n72UKYHtrg+vOA6h378FTsY5KcYlAl17oiPLzjpfUgnAOtSv4ov4yJHAb6 qbCmywPNcvPiqLhtWG+koIQXomjrWFPv3CnzNpogomDmEtyT1pf8IDKam2zUN7uBJYHV 7MScUZEIXztFtpGDRA2mh4SejOh/O03CTHy/1vX0deewP6IgJ1JOAnXF1H1atapDIA3C FDd875Vid2wjmVM7CkArIEAKNk/EpM6b1RKmm9eP+O/aeoUfMRi27BnPpWwSrfmfbc7R lAgg== MIME-Version: 1.0 X-Received: by 10.152.2.225 with SMTP id 1mr1853424lax.93.1403195090187; Thu, 19 Jun 2014 09:24:50 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 19 Jun 2014 09:24:50 -0700 (PDT) In-Reply-To: <20140619054456.GA1851@kloomba> References: <20140619054456.GA1851@kloomba> Date: Thu, 19 Jun 2014 09:24:50 -0700 X-Google-Sender-Auth: WpZU68pO2vRS2NlGHfYf0kOWdA4 Message-ID: Subject: Re: Request for testers: virt-manager, libvirt, and bhyve From: Craig Rodrigues To: Roman Bogorodskiy , =?ISO-8859-2?Q?Bart=B3omiej_Rutkowski?= Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 16:24:53 -0000 On Wed, Jun 18, 2014 at 10:44 PM, Roman Bogorodskiy wrote: > Craig Rodrigues wrote: > >> Hi, >> >> I created a patch for the deskutils/virt-manager port to add support >> for bhyve. virt-manager depends on libvirt, so the work >> that Roman has done with libvirt is critical for it to work. >> >> I have instructions for setting it up here, plus a screenshot of the >> virt-manager UI: >> >> http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html >> >> Is there anyone interested in testing this and providing feedback? >> >> Also, since virt-manager is in Python, is there anyone out there interested >> in improving this patch, and sending it upstream to the virt-manager >> developers? > > Hi Craig, > > Back in April bhyve support was added into virt-manager master [1]. So > it should be there in the next release. > > I've looked at updating the desktuils/virt-manager port to 1.0.x, but at > that time it was blocked by some outdated dependency. > > 1: https://git.fedorahosted.org/cgit/virt-manager.git/commit/?id=05df5a64843f2bd4e9a5197d97608d41b2e6dc43 Hi, What ports dependency is blocking the update of the deskutils/virt-manager port to 1.0.x? We can get other ports developers such as Bartlomiej to help. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 16:50:03 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BD225E3 for ; Thu, 19 Jun 2014 16:50:03 +0000 (UTC) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE00C2388 for ; Thu, 19 Jun 2014 16:50:02 +0000 (UTC) Received: from john by potato.growveg.org with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WxfX0-0002SD-Gw for freebsd-virtualization@freebsd.org; Thu, 19 Jun 2014 17:49:46 +0100 Date: Thu, 19 Jun 2014 17:49:46 +0100 From: John To: freebsd-virtualization@freebsd.org Subject: Re: various virtualisation questions Message-ID: <20140619164946.GA9277@potato.growveg.org> Reply-To: freebsd-virtualization@freebsd.org References: <20140619134227.GA7523@potato.growveg.org> <53A30D00.6020604@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53A30D00.6020604@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: john@potato.growveg.org X-SA-Exim-Scanned: No (on potato.growveg.org); SAEximRunCond expanded to false X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 16:50:03 -0000 On Thu, Jun 19, 2014 at 12:17:04PM -0400, Allan Jude wrote: [...] thanks for this, quick response, too :D -- John From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 17:27:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BEEB2D2; Thu, 19 Jun 2014 17:27:26 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E38892777; Thu, 19 Jun 2014 17:27:25 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.9/8.14.9) with ESMTP id s5JHRNrA057040 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Jun 2014 11:27:23 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.9/8.14.9/Submit) with ESMTP id s5JHRNpE057037; Thu, 19 Jun 2014 11:27:23 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Thu, 19 Jun 2014 11:27:23 -0600 (MDT) From: Warren Block To: Allan Jude Subject: Re: various virtualisation questions In-Reply-To: <53A30D00.6020604@freebsd.org> Message-ID: References: <20140619134227.GA7523@potato.growveg.org> <53A30D00.6020604@freebsd.org> User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Thu, 19 Jun 2014 11:27:23 -0600 (MDT) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 17:27:26 -0000 On Thu, 19 Jun 2014, Allan Jude wrote: > If you install the qemu-img tool, you should be able to convert your > VirtualBox image into a raw disk image, and then you would be able to > boot that in bhyve. VirtualBox can do that: VBoxManage internalcommands converttoraw mydisk.vdi mydisk.raw From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 19 18:44:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B213C491; Thu, 19 Jun 2014 18:44:18 +0000 (UTC) Received: from mail-qg0-x235.google.com (mail-qg0-x235.google.com [IPv6:2607:f8b0:400d:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 515492E3D; Thu, 19 Jun 2014 18:44:18 +0000 (UTC) Received: by mail-qg0-f53.google.com with SMTP id i50so2461537qgf.26 for ; Thu, 19 Jun 2014 11:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XwHPpzDxZYjcZq8Syp8viXjED3sAgYOPOAzEouTTE9A=; b=UphqD3+RlxA9PmXF1tn8/CWvFlGEdQ2dIE20/EUZotPb48njT87P12viLWoky+2uHS g1E0rCCsuJr3BGk037qMJPQBKhauxOFwh0WnkgWUWpmz/xg2cOyxrurSYCP43Ly9muzL pZoUOM/m2K/9XbAJAQbDiC95g7dmSoUAwXitHI5YTlZtJNLf8qRB/Nkq61pAtu8MTmyO 8VknYuMgTUrhefLiafg4O19TT8KWjI3MEIjQn9vHC5Q4DHh09KZTZ3f1Qp3wirF0q/7/ ECGZWtu+gNL8ouUirt5PppE6MlBqM9TW7iyE1dBTv7h0d1ns5Qjls4Hi6r+8/yFwPUiJ zaYA== MIME-Version: 1.0 X-Received: by 10.224.147.80 with SMTP id k16mr10137829qav.40.1403203457478; Thu, 19 Jun 2014 11:44:17 -0700 (PDT) Sender: spankthespam@gmail.com Received: by 10.229.96.132 with HTTP; Thu, 19 Jun 2014 11:44:17 -0700 (PDT) In-Reply-To: References: <20140619054456.GA1851@kloomba> Date: Thu, 19 Jun 2014 20:44:17 +0200 X-Google-Sender-Auth: x4CYWk7pdcAgVzGo3Gw-bmZOuZc Message-ID: Subject: Re: Request for testers: virt-manager, libvirt, and bhyve From: Bartek Rutkowski To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 18:44:18 -0000 On Thu, Jun 19, 2014 at 6:24 PM, Craig Rodrigues wrote: > On Wed, Jun 18, 2014 at 10:44 PM, Roman Bogorodskiy wrote: >> Craig Rodrigues wrote: >> >>> Hi, >>> >>> I created a patch for the deskutils/virt-manager port to add support >>> for bhyve. virt-manager depends on libvirt, so the work >>> that Roman has done with libvirt is critical for it to work. >>> >>> I have instructions for setting it up here, plus a screenshot of the >>> virt-manager UI: >>> >>> http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html >>> >>> Is there anyone interested in testing this and providing feedback? >>> >>> Also, since virt-manager is in Python, is there anyone out there interested >>> in improving this patch, and sending it upstream to the virt-manager >>> developers? >> >> Hi Craig, >> >> Back in April bhyve support was added into virt-manager master [1]. So >> it should be there in the next release. >> >> I've looked at updating the desktuils/virt-manager port to 1.0.x, but at >> that time it was blocked by some outdated dependency. >> >> 1: https://git.fedorahosted.org/cgit/virt-manager.git/commit/?id=05df5a64843f2bd4e9a5197d97608d41b2e6dc43 > > > > Hi, > > What ports dependency is blocking the update of the > deskutils/virt-manager port to 1.0.x? We can get other ports > developers such as Bartlomiej to help. > > -- > Craig I am happy to look at updating the virt-manager - any chance you could point me to what from the below list was the dependency causing issues? dbus>=0.83.2 libxml2>=2.7.8 virtinst>=0.600.4 vte>=0.26.2 gtk-vnc>=0.3.10 librsvg2>=2.32.1 libvirt>=0 Kind regards, Bartek Rutkowski From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 01:10:21 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF88672A for ; Fri, 20 Jun 2014 01:10:21 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BD402DC7 for ; Fri, 20 Jun 2014 01:10:21 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so1960742lan.30 for ; Thu, 19 Jun 2014 18:10:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=4wiQdz5nPTb/HGT3LdZiaIHsmpY4ywDFdwqrf38mAg4=; b=ezpACvo2vT6LBxDRuUt0ojTM2+S2xDNBmO2XHpPtXki1pODmwVbtHTNybDzquTjjC9 3yDkMAfo0VtTrwVqOXIieJSRzHAtn2Dk9wOcavK+2DsXy9POLIsdIO12fRP/bEFSl3VO GwRblTd+zdD5gTRtVxqT0H69LEKUA7iD9ynv2eAxQay18nsLrndeVps32S4FEijfOubA sLDnaiH1MCbSY9/5768X5gR5O6eqYkhekq12B7rSS1YoBZxprh9WwE+hgJveilEuTnZf LpE1hBiZbusMOKATcWtuKaQIt40/ka1vdbGHvpGc8AUnj7HDDO5su7vpw2U4T2JXyAmk 4idw== MIME-Version: 1.0 X-Received: by 10.112.143.132 with SMTP id se4mr55773lbb.53.1403226619180; Thu, 19 Jun 2014 18:10:19 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 19 Jun 2014 18:10:19 -0700 (PDT) Date: Thu, 19 Jun 2014 18:10:19 -0700 X-Google-Sender-Auth: 2_o9KDzg0NoiF6T-W8hsyFyXqCQ Message-ID: Subject: libvirt, bhyve, and Jenkins configuration problems From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:10:21 -0000 Hi, I am trying to configure Jenkins to spawn bhyve VM's on demand using libvirt. I did the following: (1) Looked at Roman's blog article: http://empt1e.blogspot.com/2014/03/using-jenkins-libvirt-slave-plugin-with.html (2) Followed the libvirt configuration steps here: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html Here are the screenshots of my Jenkins setp for libvirt: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html However, after going through all that, when I tried to configure a job to run on the libvirt node, it did not start. Any ideas what I am doing wrong? Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 01:11:41 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12ECA77F for ; Fri, 20 Jun 2014 01:11:41 +0000 (UTC) Received: from mailer.gewt.net (mailer.gewt.net [198.52.200.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E6DCA2E3A for ; Fri, 20 Jun 2014 01:11:40 +0000 (UTC) Received: from adsl-75-49-4-163.dsl.wotnoh.sbcglobal.net ([75.49.4.163] helo=melanie.gimme-sympathy.org) by mailer.gewt.net with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WxqpX-000NQH-FY for freebsd-virtualization@freebsd.org; Fri, 20 Jun 2014 00:53:39 -0400 Date: Thu, 19 Jun 2014 20:51:01 -0400 (EDT) From: Cory Smelosky X-X-Sender: b4@melanie To: freebsd-virtualization@freebsd.org Subject: bhyve: vde2/openvswitch Message-ID: User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Sender: b4@mailer.gewt.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:11:41 -0000 Hey all, Is there any current support for vde2 aside from using vde2_plug2tap? (Does that even work? I believe vde2 likes to open the tap interface exclusively. I need to bring up a bhyve test environment to check fully.) How exactly does the bhyve network stack work through bridging? I'm imaging it's tying in to bridging APIs? I'm assuming bhyve-to-bhyve network communication is also possible? I can't imagine it'd be too hard to implement vde2 support for networking...it'd give me more control over my VMs and give easier integration with SIMH VMs. OpenVSwitch I know less about...but I'm aware it is another virtual switch and it works on FreeBSD...not sure if it has a kernel module or if it's in userspace. With vde2 I know for sure it can be attached to a tap interface and therefore a bridge. A virtual switch for bhyve would be a useful feature if there aren't any plans for one. Get up there and have it in base. ;) Unrelated: any plans for a special bhyve-to-bhyve communication bus? Not too useful of a feature...I can't think of any use for it off-hand aside from dedicating a VM to administrating VMs without exposing much to the host beyond configurations. -- Cory Smelosky http://gewt.net Personal stuff From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 02:00:04 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3B0BE8F for ; Fri, 20 Jun 2014 02:00:04 +0000 (UTC) Received: from ims7bld5.solarus.net (smtp5.solarus.net [209.94.171.125]) by mx1.freebsd.org (Postfix) with ESMTP id 7D4B02230 for ; Fri, 20 Jun 2014 02:00:03 +0000 (UTC) Received: from mail.solarus.net (mail.tznet.com [209.94.172.150]) by ims7bld5.solarus.net (8.14.5/8.14.5) with ESMTP id s5K1VJCi019657 for ; Thu, 19 Jun 2014 20:31:19 -0500 (CDT) Date: Thu, 19 Jun 2014 20:31:19 -0500 (CDT) From: Scott Pilz To: freebsd-virtualization@freebsd.org Subject: Bhyve: CARP Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Solarus-VirusScanner: Clean X-Spam-Status: No X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 02:00:04 -0000 I know VTNET does not support CARP just yet - but has anyone tried to run a trunk to the vtnet and VLAN off of say vtnet0, setup CARP on that vlan? If not, I'll try it - just looking to save myself some time. Also as I send very few emails to this list - I'd like to say that Bhyve is an amazing feat. In the last few days we have established four production bhyves (now serving our customers) and planning for 20 additional installations this year. I built a front-end to handle starting/stopping/configuration/etc (as I did not have much luck with vmrc - just didn't do what I wanted it to do). Thanks for all the hard work in this amazing project. Scott From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 02:45:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7BB2D13 for ; Fri, 20 Jun 2014 02:45:02 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BB7642702 for ; Fri, 20 Jun 2014 02:45:02 +0000 (UTC) Received: from jre-mbp.elischer.org (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5K2it5m077479 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 19 Jun 2014 19:44:59 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53A3A022.3090004@freebsd.org> Date: Fri, 20 Jun 2014 10:44:50 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Cory Smelosky , freebsd-virtualization@freebsd.org Subject: Re: bhyve: vde2/openvswitch References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 02:45:03 -0000 On 6/20/14, 8:51 AM, Cory Smelosky wrote: > Hey all, > > Is there any current support for vde2 aside from using > vde2_plug2tap? (Does that even work? I believe vde2 likes to open > the tap interface exclusively. I need to bring up a bhyve test > environment to check fully.) > > How exactly does the bhyve network stack work through bridging? I'm > imaging it's tying in to bridging APIs? > > I'm assuming bhyve-to-bhyve network communication is also possible? > > I can't imagine it'd be too hard to implement vde2 support for > networking...it'd give me more control over my VMs and give easier > integration with SIMH VMs. > > OpenVSwitch I know less about...but I'm aware it is another virtual > switch and it works on FreeBSD...not sure if it has a kernel module > or if it's in userspace. With vde2 I know for sure it can be > attached to a tap interface and therefore a bridge. > > A virtual switch for bhyve would be a useful feature if there aren't > any plans for one. Get up there and have it in base. ;) > > Unrelated: any plans for a special bhyve-to-bhyve communication > bus? Not too useful of a feature...I can't think of any use for it > off-hand aside from dedicating a VM to administrating VMs without > exposing much to the host beyond configurations. > I believe it should be possible to plumb up arbitrary vm to vm network topologies using several different schemes, such as running different vms in different vimage jails, and using netgraph as a linkage medium, but I haven't tried them. From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 09:36:10 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B95DAEA6; Fri, 20 Jun 2014 09:36:10 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A41112989; Fri, 20 Jun 2014 09:36:09 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,513,1400025600"; d="scan'208";a="145670617" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 20 Jun 2014 09:36:00 +0000 Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Fri, 20 Jun 2014 05:35:59 -0400 Message-ID: <53A40079.9000804@citrix.com> Date: Fri, 20 Jun 2014 11:35:53 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: , "freebsd-xen@freebsd.org" , Subject: FreeBSD and memory balloon drivers X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 09:36:10 -0000 Hello, I've been looking into the Xen balloon driver, because I've experienced problems when ballooning memory down which AFAICT are also present in the VirtIO balloon driver. The problem I've experienced is that when ballooning memory down, we basically allocate a bunch of memory as WIRED, to make sure nobody tries to swap it do disk, since it will crash the kernel because the memory is not populated. Due to this massive amount of memory allocated as WIRED, user-space programs that try to use mlock will fail because we hit the limit in vm.max_wired. I'm not sure what's the best way to deal with this limitation, should vm.max_wired be changed from the balloon drivers when ballooning down/up? Is there anyway to remove the pages ballooned down from the memory accounting of wired pages? Thanks, Roger. From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 12:56:20 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F6BCCCF; Fri, 20 Jun 2014 12:56:20 +0000 (UTC) Received: from unsane.co.uk (unsane-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:110::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "unsane.co.uk", Issuer "unsane.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EC9B32E8D; Fri, 20 Jun 2014 12:56:19 +0000 (UTC) Received: from vhoffman.lon.namesco.net (lon.namesco.net [195.7.254.102]) (authenticated bits=0) by unsane.co.uk (8.14.8/8.14.8) with ESMTP id s5KCuGoW060936 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 20 Jun 2014 13:56:17 +0100 (BST) (envelope-from vince@unsane.co.uk) Message-ID: <53A42F70.7040707@unsane.co.uk> Date: Fri, 20 Jun 2014 13:56:16 +0100 From: Vincent Hoffman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Craig Rodrigues , "freebsd-virtualization@freebsd.org" Subject: Re: Request for Testers: libvirt and bhyve References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Roman Bogorodskiy X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 12:56:20 -0000 Hi Craig, I've given it a try and its looking good. I already had some bhyve vms i wanted to import rather than creating new. This worked fine. I didnt ever get the default network which isnt an issue as its the first thing I undefine on any KVM server I've use libvirt on. Also I created the bridges myself as I was unsure how libvirt was supposed to do this and I've always created them manually for linux libvirt/kvm instances, just letting libvirt create the vnet interfaces. I'm using zvols rather than file backed but thats not causing any issues. I'd suggest the port have a pkg-msg to suggest you add uri_default = "bhyve:///system" to /usr/local/etc/libvirt/libvirt.conf so you can simply use virsh list etc instead of needing to specify the uri each time. Thanks for all the work on this. Vince On 18/06/2014 19:29, Craig Rodrigues wrote: > Hi, > > I've had some success with getting libvirt and bhyve to work. > > I wrote up a list of the steps that I followed here: > > http://people.freebsd.org/~rodrigc/libvirt-bhyve/libvirt-bhyve.html > > If anyone is interested, please try it out, and provide feedback to this > list. It would be great to get more testers. > > Roman, when do you think you will be able to commit all your latest > fixes into the devel/libvirt port Makefile? It would greatly increase > the usability of the port. Thanks for doing all the work to add bhyve > support to libvirt!! > > -- > Craig > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 13:13:37 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90B5C33F for ; Fri, 20 Jun 2014 13:13:37 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 50F2C2041 for ; Fri, 20 Jun 2014 13:13:37 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s5KDDRIH007689 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 20 Jun 2014 13:13:32 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s5KDDRVE007688 for virtualization@FreeBSD.org; Fri, 20 Jun 2014 13:13:27 GMT (envelope-from swills) Date: Fri, 20 Jun 2014 13:13:27 +0000 From: Steve Wills To: virtualization@FreeBSD.org Subject: bhyve crash Message-ID: <20140620131323.GC83357@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kfjH4zxOES6UT95V" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Fri, 20 Jun 2014 13:13:32 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 13:13:37 -0000 --kfjH4zxOES6UT95V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, Just had a bhyve crash on 11-CURRENT. The host is r267362. The guest is r266947. The crash was: pid 38256 (bhyve), uid 0: exited on signal 6 (core dumped) (The formatting of this may be off a bit) vmexit[12] reason VMX rip 0xffffffff80d9e11d inst_length 3 status 0 exit_reason 2 qualification 0x0000000000000000 inst_type 0 inst_error 0 I can provide the core dump file. Ideas? Thanks, Steve --kfjH4zxOES6UT95V Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTpDNyAAoJEPXPYrMgexuhTrsH/ROy4cZktRXQW7xe/9HZTiNe kiZrRE/8N061Rk+Vq9upwsl3+kkgUxw6NqeVBu3MYXLUhCxHzGZ3CkEBu0bzAHPC RTKYKOE+xyqdWzJqNrp8MN0YgHb8v+m5IPwlSqvit8TuEQMvNOBL4nh6y0gVtFtl ESChNzVOtdI08sH6YtlvM6XfkR3R7+Ea9wsS29siQBvtqKbr91TerEVmIBU11jAd gr+qZbtZ1pn3ZBNEHDJWi75qzxNlx0lPij/qJmgowrCOsOi7a6T5S+bjrQGKsWwv YBB+k5xOku9asnN0NffuA5uP4LhzYzjZ4zlZc0ftuvJAZ0MLLYP7OlFZHjC0RQk= =xcuQ -----END PGP SIGNATURE----- --kfjH4zxOES6UT95V-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 13:28:33 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED9D49CD; Fri, 20 Jun 2014 13:28:33 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77303216D; Fri, 20 Jun 2014 13:28:33 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5KDSG66027116; Fri, 20 Jun 2014 16:28:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5KDSG66027116 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.8/Submit) id s5KDSGDQ027115; Fri, 20 Jun 2014 16:28:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 20 Jun 2014 16:28:16 +0300 From: Konstantin Belousov To: Roger Pau Monn? Subject: Re: FreeBSD and memory balloon drivers Message-ID: <20140620132816.GH3991@kib.kiev.ua> References: <53A40079.9000804@citrix.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i+R2Fnv74iOC5F2I" Content-Disposition: inline In-Reply-To: <53A40079.9000804@citrix.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: virtualization@FreeBSD.org, "freebsd-xen@freebsd.org" , bryanv@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 13:28:34 -0000 --i+R2Fnv74iOC5F2I Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 20, 2014 at 11:35:53AM +0200, Roger Pau Monn? wrote: > Hello, >=20 > I've been looking into the Xen balloon driver, because I've experienced > problems when ballooning memory down which AFAICT are also present in > the VirtIO balloon driver. The problem I've experienced is that when > ballooning memory down, we basically allocate a bunch of memory as > WIRED, to make sure nobody tries to swap it do disk, since it will crash > the kernel because the memory is not populated. Due to this massive > amount of memory allocated as WIRED, user-space programs that try to use > mlock will fail because we hit the limit in vm.max_wired. >=20 > I'm not sure what's the best way to deal with this limitation, should > vm.max_wired be changed from the balloon drivers when ballooning > down/up? Is there anyway to remove the pages ballooned down from the > memory accounting of wired pages? You could change the type of pages the ballon driver is allocating. Instead of wired pages, you may request unmanaged, by passing NULL object to vm_page_alloc(). This would also save on the trie nodes for managing the radix trie for the object. There are still plinks or listq to keep track of the allocated pages. --i+R2Fnv74iOC5F2I Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTpDbvAAoJEJDCuSvBvK1BgikP+gPdzLt8XtsevQkU8vfpCtJj ZnRdVneYSdHC6JBe+wUI0eT4/gQ9bIUD8eHYeVrVSW4JG0Br4SrpU0fTigfHi14j udk2+tcTmUFw7Ksx2l8L8qKGatWacZXZwF9adZfazvulwmMKmzou2bOcO9V9bBMb 8XgamaNFDldAl2KYyFP2k93ozuSTjgww1WhjooFcbhdlVgDDYzK+SNpk17BgDD+b 9FXjkiIKBNtH+xTznmL5zYAmrr4pCAy0MaOElawnR7oRlm0h9/9E49efm6jAcXrn ntnkfzlScTSRBtk/ZxIMpkK87LUfY80VkGnSentkSAUBqqEr217Ifnqf/Kbn8Y0v Zh0HGP2nF1J1TwqhEbsgQMGEBHjwzjx/vX39NX5pJfnCQ9Okb6mJMkzCR/8iIysI mNgYa0EOzg5wo022GiO18iWinAhklYFhIc+rc65Ot9qOSq5Y2lhp1xks1HTEnuC6 ircTeK5do0II/boxvv5j3pefjnUEc39iwSuKuKx4enFDErN3tQjJ/G2NqdqfYPH7 rhl4UMX1X0vIxLraOT1ZNIlJupj1gSfuAxgcEMdl54Upx9nE8IeSfy0F8UN8Kj5n js6brUwv06sHX9FUN5Ixo10ifZRqktzV9YcEEzaVYrTsM9+IFC3UgEt8QQ3CQubW sphwPTjgF7JM8TWS+fWf =dJ1f -----END PGP SIGNATURE----- --i+R2Fnv74iOC5F2I-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 15:15:50 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41C71B2E; Fri, 20 Jun 2014 15:15:50 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 326532B84; Fri, 20 Jun 2014 15:15:48 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,514,1400025600"; d="scan'208,223";a="145480682" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 20 Jun 2014 15:15:45 +0000 Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Fri, 20 Jun 2014 11:15:45 -0400 Message-ID: <53A4501F.4020201@citrix.com> Date: Fri, 20 Jun 2014 17:15:43 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: FreeBSD and memory balloon drivers References: <53A40079.9000804@citrix.com> <20140620132816.GH3991@kib.kiev.ua> In-Reply-To: <20140620132816.GH3991@kib.kiev.ua> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------080308010102010404060706" X-DLP: MIA1 Cc: virtualization@FreeBSD.org, "freebsd-xen@freebsd.org" , bryanv@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 15:15:50 -0000 --------------080308010102010404060706 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/06/14 15:28, Konstantin Belousov wrote: > On Fri, Jun 20, 2014 at 11:35:53AM +0200, Roger Pau Monn? wrote: >> Hello, >> >> I've been looking into the Xen balloon driver, because I've >> experienced problems when ballooning memory down which AFAICT are >> also present in the VirtIO balloon driver. The problem I've >> experienced is that when ballooning memory down, we basically >> allocate a bunch of memory as WIRED, to make sure nobody tries to >> swap it do disk, since it will crash the kernel because the >> memory is not populated. Due to this massive amount of memory >> allocated as WIRED, user-space programs that try to use mlock >> will fail because we hit the limit in vm.max_wired. >> >> I'm not sure what's the best way to deal with this limitation, >> should vm.max_wired be changed from the balloon drivers when >> ballooning down/up? Is there anyway to remove the pages ballooned >> down from the memory accounting of wired pages? > > You could change the type of pages the ballon driver is > allocating. Instead of wired pages, you may request unmanaged, by > passing NULL object to vm_page_alloc(). This would also save on > the trie nodes for managing the radix trie for the object. There > are still plinks or listq to keep track of the allocated pages. Thanks for the info, I have the following patch which fixes the usage of WIRED for both the Xen and the VirtIO balloon drivers, could someone please test the VirtIO side? Roger. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) iQEcBAEBAgAGBQJTpFAfAAoJEKXZdqUyumTA1TQH/22YpAGCQ8oa0hmpfE8oovxz q8EDRyfDoogEswNYwboI8cBP7GSbuBbe1Z0MTiMHtwyHzqGhJM5B7jKioqFsqxvc /Qfld8z3vDD94/5iaMX64dV2/VKkLwypR2uU5PkN018FTAJ0FFycC336xVjD8eUz /DCRQIZRUzNlcrZYlOtSALR2M9bM1/f2++e2C6L7kbSsF4BAH2wmRcdtM1uBMO6/ CnD7ctZsnxxdS05eLWMpv6jfcRH8yDM3vPaHgXa223q74TU1Rh7AFw/TPsyBvBY2 cDUnYLZdUx0NQJfPM9MKGLe8P5o3WLVO1hfGmrZHdmjE2B18mNfzg4SS0CUQhQ0= =hctw -----END PGP SIGNATURE----- --------------080308010102010404060706 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="0001-xen-virtio-fix-balloon-drivers-to-not-mark-pages-as-.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-xen-virtio-fix-balloon-drivers-to-not-mark-pages-as-.pa"; filename*1="tch" >From 2ed0f82b16753c96e96acc1e26a75a0fd2ee7d34 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Fri, 20 Jun 2014 16:34:31 +0200 Subject: [PATCH] xen/virtio: fix balloon drivers to not mark pages as WIRED Prevent the Xen and VirtIO balloon drivers from marking pages as wired. This prevents them from increasing the system wired page count, which can lead to mlock failing because of hitting the limit in vm.max_wired. Also, in the Xen case make sure pages are zeroed before giving them back to the hypervisor, or else we might be leaking data. Sponsored by: Citrix Systems R&D Reviewed by: xxx Approved by: xxx dev/virtio/balloon/virtio_balloon.c: - Don't allocate pages with VM_ALLOC_WIRED. dev/xen/balloon/balloon.c: - Don't allocate pages with VM_ALLOC_WIRED. - Make sure pages are zeroed before giving them back to the hypervisor. --- sys/dev/virtio/balloon/virtio_balloon.c | 4 +--- sys/dev/xen/balloon/balloon.c | 13 ++++++++++--- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/sys/dev/virtio/balloon/virtio_balloon.c b/sys/dev/virtio/balloon/virtio_balloon.c index d540099..6d00ef3 100644 --- a/sys/dev/virtio/balloon/virtio_balloon.c +++ b/sys/dev/virtio/balloon/virtio_balloon.c @@ -438,8 +438,7 @@ vtballoon_alloc_page(struct vtballoon_softc *sc) { vm_page_t m; - m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_WIRED | - VM_ALLOC_NOOBJ); + m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ); if (m != NULL) sc->vtballoon_current_npages++; @@ -450,7 +449,6 @@ static void vtballoon_free_page(struct vtballoon_softc *sc, vm_page_t m) { - vm_page_unwire(m, PQ_INACTIVE); vm_page_free(m); sc->vtballoon_current_npages--; } diff --git a/sys/dev/xen/balloon/balloon.c b/sys/dev/xen/balloon/balloon.c index fa56c86..a7ca1e4 100644 --- a/sys/dev/xen/balloon/balloon.c +++ b/sys/dev/xen/balloon/balloon.c @@ -255,7 +255,6 @@ increase_reservation(unsigned long nr_pages) set_phys_to_machine(pfn, frame_list[i]); - vm_page_unwire(page, PQ_INACTIVE); vm_page_free(page); } @@ -286,18 +285,26 @@ decrease_reservation(unsigned long nr_pages) for (i = 0; i < nr_pages; i++) { if ((page = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | - VM_ALLOC_WIRED | VM_ALLOC_ZERO)) == NULL) { + VM_ALLOC_ZERO)) == NULL) { nr_pages = i; need_sleep = 1; break; } + if ((page->flags & PG_ZERO) == 0) { + /* + * Zero the page, or else we might be leaking + * important data to other domains on the same + * host. + */ + pmap_zero_page(page); + } + pfn = (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); frame_list[i] = PFNTOMFN(pfn); set_phys_to_machine(pfn, INVALID_P2M_ENTRY); if (balloon_append(page) != 0) { - vm_page_unwire(page, PQ_INACTIVE); vm_page_free(page); nr_pages = i; -- 1.7.7.5 (Apple Git-26) --------------080308010102010404060706-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 15:26:20 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69FBDF8A; Fri, 20 Jun 2014 15:26:20 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EF5242C76; Fri, 20 Jun 2014 15:26:19 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5KFQAMZ054571; Fri, 20 Jun 2014 18:26:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5KFQAMZ054571 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.8/Submit) id s5KFQABo054570; Fri, 20 Jun 2014 18:26:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 20 Jun 2014 18:26:10 +0300 From: Konstantin Belousov To: Roger Pau Monn? Subject: Re: FreeBSD and memory balloon drivers Message-ID: <20140620152610.GI3991@kib.kiev.ua> References: <53A40079.9000804@citrix.com> <20140620132816.GH3991@kib.kiev.ua> <53A4501F.4020201@citrix.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OrKNV1jnpyJzuZNm" Content-Disposition: inline In-Reply-To: <53A4501F.4020201@citrix.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: virtualization@FreeBSD.org, "freebsd-xen@freebsd.org" , bryanv@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 15:26:20 -0000 --OrKNV1jnpyJzuZNm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 20, 2014 at 05:15:43PM +0200, Roger Pau Monn? wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > On 20/06/14 15:28, Konstantin Belousov wrote: > > On Fri, Jun 20, 2014 at 11:35:53AM +0200, Roger Pau Monn? wrote: > >> Hello, > >>=20 > >> I've been looking into the Xen balloon driver, because I've > >> experienced problems when ballooning memory down which AFAICT are > >> also present in the VirtIO balloon driver. The problem I've > >> experienced is that when ballooning memory down, we basically > >> allocate a bunch of memory as WIRED, to make sure nobody tries to > >> swap it do disk, since it will crash the kernel because the > >> memory is not populated. Due to this massive amount of memory > >> allocated as WIRED, user-space programs that try to use mlock > >> will fail because we hit the limit in vm.max_wired. > >>=20 > >> I'm not sure what's the best way to deal with this limitation, > >> should vm.max_wired be changed from the balloon drivers when > >> ballooning down/up? Is there anyway to remove the pages ballooned > >> down from the memory accounting of wired pages? > >=20 > > You could change the type of pages the ballon driver is > > allocating. Instead of wired pages, you may request unmanaged, by > > passing NULL object to vm_page_alloc(). This would also save on > > the trie nodes for managing the radix trie for the object. There > > are still plinks or listq to keep track of the allocated pages. >=20 > Thanks for the info, I have the following patch which fixes the usage > of WIRED for both the Xen and the VirtIO balloon drivers, could > someone please test the VirtIO side? I briefly looked at the xen balloon. You do not need balloon_append(). Use struct vm_page plinks field to link the pages. --OrKNV1jnpyJzuZNm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTpFKRAAoJEJDCuSvBvK1Bbu4QAIt2ojo9JMTih/B2Y4S5vveD GRqLa3YUfOetBMt4xKbGKTmWcIzCcHBe1MIZ1qY8iufQErYwC5Kw0F0AxnSV2BEV TJReZbrWzlyKKIQZTTqgHf6ZUOIKI2rO1tSt5fj9fMBDSrbteqQOKdSd5saWfd1y g/xOFYjdB6iLtKhS+Q/1/CtwYmxTZKbzGf0D0LSa9dDsAEuL/i3SyNvNSUqXN6D3 uj6w78L0sTDx24zI31oEWbxWI5eII4tdTgMkDFZpa6c5/VRNAHo8dmCTDBTSy9M2 IcvgMWkOhPHNe+GG8IO1nupPcyby7vhlAymo0ubuH+QrnfWZNaTzpZgfE+bNtDkD wHwz4GRY/fheslBDkHf8KkQIASFpeE3uvgt2bktkJCTG53ER5Sr1CmLCjfic9xkF E2h6KoiK+5QsMBOy1UYO47La7apSEoPB8QrLrXf9V7Jjg3ZFLNNG0FbcGvfdD1Xp Fu8Fl+1ZIjuWrzA/UKjLaUzk5YrplWhWKbN1W9sndz7snFv+I3y2KVyi/AR2dcsq A50ctrx+qMrI4sDtjH3LpTRqI1xhAtIU3fveAKGWN19EYiJkgYCVaPmwTnz1L1c/ u5/szDWuegfaYQGjOr5yt/Zy/CBSiQwE1BiWUM4CXAZWXFxa9yRJASif+CmdILSM nMyMR7xeCqFadeLZc6Hh =+WSD -----END PGP SIGNATURE----- --OrKNV1jnpyJzuZNm-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 16:56:30 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8FCEFE9; Fri, 20 Jun 2014 16:56:30 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E4A32496; Fri, 20 Jun 2014 16:56:28 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,514,1400025600"; d="scan'208,223";a="145521367" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 20 Jun 2014 16:56:24 +0000 Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Fri, 20 Jun 2014 12:56:24 -0400 Message-ID: <53A467B4.8070501@citrix.com> Date: Fri, 20 Jun 2014 18:56:20 +0200 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: FreeBSD and memory balloon drivers References: <53A40079.9000804@citrix.com> <20140620132816.GH3991@kib.kiev.ua> <53A4501F.4020201@citrix.com> <20140620152610.GI3991@kib.kiev.ua> In-Reply-To: <20140620152610.GI3991@kib.kiev.ua> X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------090506020608050903030305" X-DLP: MIA1 Cc: virtualization@FreeBSD.org, "freebsd-xen@freebsd.org" , bryanv@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 16:56:30 -0000 --------------090506020608050903030305 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/06/14 17:26, Konstantin Belousov wrote: > On Fri, Jun 20, 2014 at 05:15:43PM +0200, Roger Pau Monn? wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> >> On 20/06/14 15:28, Konstantin Belousov wrote: >>> On Fri, Jun 20, 2014 at 11:35:53AM +0200, Roger Pau Monn? >>> wrote: >>>> Hello, >>>> >>>> I've been looking into the Xen balloon driver, because I've >>>> experienced problems when ballooning memory down which AFAICT >>>> are also present in the VirtIO balloon driver. The problem >>>> I've experienced is that when ballooning memory down, we >>>> basically allocate a bunch of memory as WIRED, to make sure >>>> nobody tries to swap it do disk, since it will crash the >>>> kernel because the memory is not populated. Due to this >>>> massive amount of memory allocated as WIRED, user-space >>>> programs that try to use mlock will fail because we hit the >>>> limit in vm.max_wired. >>>> >>>> I'm not sure what's the best way to deal with this >>>> limitation, should vm.max_wired be changed from the balloon >>>> drivers when ballooning down/up? Is there anyway to remove >>>> the pages ballooned down from the memory accounting of wired >>>> pages? >>> >>> You could change the type of pages the ballon driver is >>> allocating. Instead of wired pages, you may request unmanaged, >>> by passing NULL object to vm_page_alloc(). This would also >>> save on the trie nodes for managing the radix trie for the >>> object. There are still plinks or listq to keep track of the >>> allocated pages. >> >> Thanks for the info, I have the following patch which fixes the >> usage of WIRED for both the Xen and the VirtIO balloon drivers, >> could someone please test the VirtIO side? > I briefly looked at the xen balloon. You do not need > balloon_append(). Use struct vm_page plinks field to link the > pages. Sure, thanks for the review, here is an updated version. Roger. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) iQEcBAEBAgAGBQJTpGe0AAoJEKXZdqUyumTA5acH/jQ+Iwjw+QTwUtsh9ZLs7Gm0 h7XaCwa/gASEjzcxK2smIBuKA10LUSoulcYkC3cUWXqaPwFega14JcbySBRI06Z1 1bU50DL8TPLuQIzrVWZgtA+QsZkwEu/jhijr0AMxTcnm6Wq1LV5QRDoqhD0kRiHu kEW1ez832y/0j+oeQAq0aR67zVw7hoIouH9eLaWXS4Vgxz5YQ2Pal1BMAY/OCgua xyF7BHia9KsGTKZ9pPUQfQAW5eJrwAxR0AitQjmOwRKtWyRqymZxhYHjLYOgOKQJ w93aLRVeG2oo0NNC6a9JD/c64sGHLABg37mnSTRL/v9gTj9I1DcFoid2q0iDGbM= =8nDn -----END PGP SIGNATURE----- --------------090506020608050903030305 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="0001-xen-virtio-fix-balloon-drivers-to-not-mark-pages-as-.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-xen-virtio-fix-balloon-drivers-to-not-mark-pages-as-.pa"; filename*1="tch" >From e5c898b1b2b734fdc4aa2acf645efd481f09b71d Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Fri, 20 Jun 2014 16:34:31 +0200 Subject: [PATCH] xen/virtio: fix balloon drivers to not mark pages as WIRED Prevent the Xen and VirtIO balloon drivers from marking pages as wired. This prevents them from increasing the system wired page count, which can lead to mlock failing because of hitting the limit in vm.max_wired. In the Xen case make sure pages are zeroed before giving them back to the hypervisor, or else we might be leaking data. Also remove the balloon_{append/retrieve} and link pages directly into the ballooned_pages queue using the plinks.q field in the page struct. Sponsored by: Citrix Systems R&D Reviewed by: xxx Approved by: xxx dev/virtio/balloon/virtio_balloon.c: - Don't allocate pages with VM_ALLOC_WIRED. dev/xen/balloon/balloon.c: - Don't allocate pages with VM_ALLOC_WIRED. - Make sure pages are zeroed before giving them back to the hypervisor. - Remove the balloon_entry struct and the balloon_{append/retrieve} functions and use the page plinks.q entry to link the pages directly into the ballooned_pages queue. --- sys/dev/virtio/balloon/virtio_balloon.c | 4 +- sys/dev/xen/balloon/balloon.c | 87 ++++++++----------------------- 2 files changed, 23 insertions(+), 68 deletions(-) diff --git a/sys/dev/virtio/balloon/virtio_balloon.c b/sys/dev/virtio/balloon/virtio_balloon.c index d540099..6d00ef3 100644 --- a/sys/dev/virtio/balloon/virtio_balloon.c +++ b/sys/dev/virtio/balloon/virtio_balloon.c @@ -438,8 +438,7 @@ vtballoon_alloc_page(struct vtballoon_softc *sc) { vm_page_t m; - m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_WIRED | - VM_ALLOC_NOOBJ); + m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ); if (m != NULL) sc->vtballoon_current_npages++; @@ -450,7 +449,6 @@ static void vtballoon_free_page(struct vtballoon_softc *sc, vm_page_t m) { - vm_page_unwire(m, PQ_INACTIVE); vm_page_free(m); sc->vtballoon_current_npages--; } diff --git a/sys/dev/xen/balloon/balloon.c b/sys/dev/xen/balloon/balloon.c index fa56c86..0d2bba2 100644 --- a/sys/dev/xen/balloon/balloon.c +++ b/sys/dev/xen/balloon/balloon.c @@ -94,13 +94,8 @@ SYSCTL_ULONG(_dev_xen_balloon, OID_AUTO, low_mem, CTLFLAG_RD, SYSCTL_ULONG(_dev_xen_balloon, OID_AUTO, high_mem, CTLFLAG_RD, &bs.balloon_high, 0, "High-mem balloon"); -struct balloon_entry { - vm_page_t page; - STAILQ_ENTRY(balloon_entry) list; -}; - /* List of ballooned pages, threaded through the mem_map array. */ -static STAILQ_HEAD(,balloon_entry) ballooned_pages; +static TAILQ_HEAD(,vm_page) ballooned_pages; /* Main work function, always executed in process context. */ static void balloon_process(void *unused); @@ -110,47 +105,6 @@ static void balloon_process(void *unused); #define WPRINTK(fmt, args...) \ printk(KERN_WARNING "xen_mem: " fmt, ##args) -/* balloon_append: add the given page to the balloon. */ -static int -balloon_append(vm_page_t page) -{ - struct balloon_entry *entry; - - mtx_assert(&balloon_mutex, MA_OWNED); - - entry = malloc(sizeof(struct balloon_entry), M_BALLOON, M_NOWAIT); - if (!entry) - return (ENOMEM); - entry->page = page; - STAILQ_INSERT_HEAD(&ballooned_pages, entry, list); - bs.balloon_low++; - - return (0); -} - -/* balloon_retrieve: rescue a page from the balloon, if it is not empty. */ -static vm_page_t -balloon_retrieve(void) -{ - vm_page_t page; - struct balloon_entry *entry; - - mtx_assert(&balloon_mutex, MA_OWNED); - - if (STAILQ_EMPTY(&ballooned_pages)) - return (NULL); - - entry = STAILQ_FIRST(&ballooned_pages); - STAILQ_REMOVE_HEAD(&ballooned_pages, list); - - page = entry->page; - free(entry, M_BALLOON); - - bs.balloon_low--; - - return (page); -} - static unsigned long current_target(void) { @@ -203,7 +157,6 @@ static int increase_reservation(unsigned long nr_pages) { unsigned long pfn, i; - struct balloon_entry *entry; vm_page_t page; long rc; struct xen_memory_reservation reservation = { @@ -217,10 +170,9 @@ increase_reservation(unsigned long nr_pages) if (nr_pages > nitems(frame_list)) nr_pages = nitems(frame_list); - for (entry = STAILQ_FIRST(&ballooned_pages), i = 0; - i < nr_pages; i++, entry = STAILQ_NEXT(entry, list)) { - KASSERT(entry, ("ballooned_pages list corrupt")); - page = entry->page; + for (page = TAILQ_FIRST(&ballooned_pages), i = 0; + i < nr_pages; i++, page = TAILQ_NEXT(page, plinks.q)) { + KASSERT(page != NULL, ("ballooned_pages list corrupt")); frame_list[i] = (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); } @@ -245,8 +197,10 @@ increase_reservation(unsigned long nr_pages) } for (i = 0; i < nr_pages; i++) { - page = balloon_retrieve(); - KASSERT(page, ("balloon_retrieve failed")); + page = TAILQ_FIRST(&ballooned_pages); + KASSERT(page != NULL, ("Unable to get ballooned page")); + TAILQ_REMOVE(&ballooned_pages, page, plinks.q); + bs.balloon_low--; pfn = (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); KASSERT((xen_feature(XENFEAT_auto_translated_physmap) || @@ -255,7 +209,6 @@ increase_reservation(unsigned long nr_pages) set_phys_to_machine(pfn, frame_list[i]); - vm_page_unwire(page, PQ_INACTIVE); vm_page_free(page); } @@ -286,24 +239,27 @@ decrease_reservation(unsigned long nr_pages) for (i = 0; i < nr_pages; i++) { if ((page = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | - VM_ALLOC_WIRED | VM_ALLOC_ZERO)) == NULL) { + VM_ALLOC_ZERO)) == NULL) { nr_pages = i; need_sleep = 1; break; } + if ((page->flags & PG_ZERO) == 0) { + /* + * Zero the page, or else we might be leaking + * important data to other domains on the same + * host. + */ + pmap_zero_page(page); + } + pfn = (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); frame_list[i] = PFNTOMFN(pfn); set_phys_to_machine(pfn, INVALID_P2M_ENTRY); - if (balloon_append(page) != 0) { - vm_page_unwire(page, PQ_INACTIVE); - vm_page_free(page); - - nr_pages = i; - need_sleep = 1; - break; - } + TAILQ_INSERT_HEAD(&ballooned_pages, page, plinks.q); + bs.balloon_low++; } set_xen_guest_handle(reservation.extent_start, frame_list); @@ -438,7 +394,8 @@ balloon_init(void *arg) /* Initialise the balloon with excess memory space. */ for (pfn = xen_start_info->nr_pages; pfn < max_pfn; pfn++) { page = PHYS_TO_VM_PAGE(pfn << PAGE_SHIFT); - balloon_append(page); + TAILQ_INSERT_HEAD(&ballooned_pages, page, plinks.q); + bs.balloon_low++; } #undef max_pfn #endif -- 1.7.7.5 (Apple Git-26) --------------090506020608050903030305-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 17:48:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4F0EFD8 for ; Fri, 20 Jun 2014 17:48:15 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD3F2973 for ; Fri, 20 Jun 2014 17:48:15 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so2641528lab.27 for ; Fri, 20 Jun 2014 10:48:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pXA/u4OOLH7nlM5RupmdSySA5ipyndnZVNmW0uN7cvc=; b=Wj6ld2KgQdAbcwLrk6nN4CBuLB8YDpbqIjvX7kdNJHPsZXX159yon0p9XfpfpRnx79 xo7ESAA54Ue3ug9h39k4A6nZ/nG1gLpCgsyFgK6sgdN66zf2HvlW5K2HMTm6cDxDHUU3 lgEmMc4bqIC8abXmNtZZ7E61InCBfmIVDfL9Roe4Gl8MALE8GkLI3TSX0kotgdPnr9Jr 8PMzxJxZqXkUVzC6+a3ZJwvIZFErjJYhCr/R88Httk8Sl0d3htbOW6R+x8MdYjY0Pz/p pDrUoj0689USmn0v8I2kXwwUkeTZPn318rvTUcDATqhaal3ugkyEqXHC/MbfmM6WTxao w3ZA== MIME-Version: 1.0 X-Received: by 10.152.179.131 with SMTP id dg3mr3495682lac.21.1403286493062; Fri, 20 Jun 2014 10:48:13 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 20 Jun 2014 10:48:13 -0700 (PDT) In-Reply-To: <53A42F70.7040707@unsane.co.uk> References: <53A42F70.7040707@unsane.co.uk> Date: Fri, 20 Jun 2014 10:48:13 -0700 X-Google-Sender-Auth: i6wAI1F5X4Pa-JykUdxVpS1-cMA Message-ID: Subject: Re: Request for Testers: libvirt and bhyve From: Craig Rodrigues To: Vincent Hoffman Content-Type: text/plain; charset=ISO-8859-1 Cc: Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 17:48:15 -0000 On Fri, Jun 20, 2014 at 5:56 AM, Vincent Hoffman wrote: > Hi Craig, > I've given it a try and its looking good. Nice! Do you have time to try virt-manager: http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html > > Thanks for all the work on this. Roman did all the hard work to get bhyve support into libvirt, and upstream his changes to the libvirt project. All the thanks go to him! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 20 17:52:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A499F1E1 for ; Fri, 20 Jun 2014 17:52:19 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 67BCF2A27 for ; Fri, 20 Jun 2014 17:52:19 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 7BBD412420; Sat, 21 Jun 2014 03:52:11 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BVC07087 (AUTH peterg@ptree32.com.au); Sat, 21 Jun 2014 03:52:09 +1000 Message-ID: <53A474C7.9030909@freebsd.org> Date: Fri, 20 Jun 2014 10:52:07 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Cory Smelosky Subject: Re: bhyve: vde2/openvswitch References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 17:52:19 -0000 Hi Cory, > Is there any current support for vde2 None yet. > aside from using vde2_plug2tap? > (Does that even work? I believe vde2 likes to open the tap interface > exclusively. I need to bring up a bhyve test environment to check fully.) ... > How exactly does the bhyve network stack work through bridging? I'm > imaging it's tying in to bridging APIs? bhyve uses a tap(4) device. Since that shows up as an ifnet to the rest of the system, it can be used in any way an ethernet interface can be used. For bridging, the bridge(4) device is used to join tap/ethernet interfaces into an L2 bridge. > I'm assuming bhyve-to-bhyve network communication is also possible? Yes, though currently that is through tap/bridge devices. > I can't imagine it'd be too hard to implement vde2 support for > networking...it'd give me more control over my VMs and give easier > integration with SIMH VMs. > A virtual switch for bhyve would be a useful feature if there aren't any > plans for one. Get up there and have it in base. ;) This has been talked about quite a bit, though nothing concrete. > Unrelated: any plans for a special bhyve-to-bhyve communication bus? > Not too useful of a feature...I can't think of any use for it off-hand > aside from dedicating a VM to administrating VMs without exposing much > to the host beyond configurations. There should be code going in soon that allows the netmap VALE switch to be used for high-speed VM-VM communication. This work also allows different network back-ends to be written, so it should be a simple task to drop in a vde2 driver. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 21 05:31:31 2014 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C40B42EB; Sat, 21 Jun 2014 05:31:31 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4AD20218C; Sat, 21 Jun 2014 05:31:31 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5L5VLb5073107; Sat, 21 Jun 2014 08:31:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5L5VLb5073107 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.8/Submit) id s5L5VLdo073106; Sat, 21 Jun 2014 08:31:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 21 Jun 2014 08:31:21 +0300 From: Konstantin Belousov To: Roger Pau Monn? Subject: Re: FreeBSD and memory balloon drivers Message-ID: <20140621053121.GK3991@kib.kiev.ua> References: <53A40079.9000804@citrix.com> <20140620132816.GH3991@kib.kiev.ua> <53A4501F.4020201@citrix.com> <20140620152610.GI3991@kib.kiev.ua> <53A467B4.8070501@citrix.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="N7OK10pvBj1bDp4V" Content-Disposition: inline In-Reply-To: <53A467B4.8070501@citrix.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: virtualization@FreeBSD.org, "freebsd-xen@freebsd.org" , bryanv@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 05:31:31 -0000 --N7OK10pvBj1bDp4V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 20, 2014 at 06:56:20PM +0200, Roger Pau Monn? wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > On 20/06/14 17:26, Konstantin Belousov wrote: > > On Fri, Jun 20, 2014 at 05:15:43PM +0200, Roger Pau Monn? wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > >>=20 > >> On 20/06/14 15:28, Konstantin Belousov wrote: > >>> On Fri, Jun 20, 2014 at 11:35:53AM +0200, Roger Pau Monn? > >>> wrote: > >>>> Hello, > >>>>=20 > >>>> I've been looking into the Xen balloon driver, because I've=20 > >>>> experienced problems when ballooning memory down which AFAICT > >>>> are also present in the VirtIO balloon driver. The problem > >>>> I've experienced is that when ballooning memory down, we > >>>> basically allocate a bunch of memory as WIRED, to make sure > >>>> nobody tries to swap it do disk, since it will crash the > >>>> kernel because the memory is not populated. Due to this > >>>> massive amount of memory allocated as WIRED, user-space > >>>> programs that try to use mlock will fail because we hit the > >>>> limit in vm.max_wired. > >>>>=20 > >>>> I'm not sure what's the best way to deal with this > >>>> limitation, should vm.max_wired be changed from the balloon > >>>> drivers when ballooning down/up? Is there anyway to remove > >>>> the pages ballooned down from the memory accounting of wired > >>>> pages? > >>>=20 > >>> You could change the type of pages the ballon driver is=20 > >>> allocating. Instead of wired pages, you may request unmanaged, > >>> by passing NULL object to vm_page_alloc(). This would also > >>> save on the trie nodes for managing the radix trie for the > >>> object. There are still plinks or listq to keep track of the > >>> allocated pages. > >>=20 > >> Thanks for the info, I have the following patch which fixes the > >> usage of WIRED for both the Xen and the VirtIO balloon drivers, > >> could someone please test the VirtIO side? > > I briefly looked at the xen balloon. You do not need > > balloon_append(). Use struct vm_page plinks field to link the > > pages. >=20 > Sure, thanks for the review, here is an updated version. It looks fine to me. >=20 > Roger. >=20 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (Darwin) >=20 > iQEcBAEBAgAGBQJTpGe0AAoJEKXZdqUyumTA5acH/jQ+Iwjw+QTwUtsh9ZLs7Gm0 > h7XaCwa/gASEjzcxK2smIBuKA10LUSoulcYkC3cUWXqaPwFega14JcbySBRI06Z1 > 1bU50DL8TPLuQIzrVWZgtA+QsZkwEu/jhijr0AMxTcnm6Wq1LV5QRDoqhD0kRiHu > kEW1ez832y/0j+oeQAq0aR67zVw7hoIouH9eLaWXS4Vgxz5YQ2Pal1BMAY/OCgua > xyF7BHia9KsGTKZ9pPUQfQAW5eJrwAxR0AitQjmOwRKtWyRqymZxhYHjLYOgOKQJ > w93aLRVeG2oo0NNC6a9JD/c64sGHLABg37mnSTRL/v9gTj9I1DcFoid2q0iDGbM=3D > =3D8nDn > -----END PGP SIGNATURE----- > >From e5c898b1b2b734fdc4aa2acf645efd481f09b71d Mon Sep 17 00:00:00 2001 > From: Roger Pau Monne > Date: Fri, 20 Jun 2014 16:34:31 +0200 > Subject: [PATCH] xen/virtio: fix balloon drivers to not mark pages as WIR= ED >=20 > Prevent the Xen and VirtIO balloon drivers from marking pages as > wired. This prevents them from increasing the system wired page count, > which can lead to mlock failing because of hitting the limit in > vm.max_wired. >=20 > In the Xen case make sure pages are zeroed before giving them back to > the hypervisor, or else we might be leaking data. Also remove the > balloon_{append/retrieve} and link pages directly into the > ballooned_pages queue using the plinks.q field in the page struct. >=20 > Sponsored by: Citrix Systems R&D > Reviewed by: xxx > Approved by: xxx >=20 > dev/virtio/balloon/virtio_balloon.c: > - Don't allocate pages with VM_ALLOC_WIRED. >=20 > dev/xen/balloon/balloon.c: > - Don't allocate pages with VM_ALLOC_WIRED. > - Make sure pages are zeroed before giving them back to the > hypervisor. > - Remove the balloon_entry struct and the balloon_{append/retrieve} > functions and use the page plinks.q entry to link the pages > directly into the ballooned_pages queue. > --- > sys/dev/virtio/balloon/virtio_balloon.c | 4 +- > sys/dev/xen/balloon/balloon.c | 87 ++++++++-----------------= ------ > 2 files changed, 23 insertions(+), 68 deletions(-) >=20 > diff --git a/sys/dev/virtio/balloon/virtio_balloon.c b/sys/dev/virtio/bal= loon/virtio_balloon.c > index d540099..6d00ef3 100644 > --- a/sys/dev/virtio/balloon/virtio_balloon.c > +++ b/sys/dev/virtio/balloon/virtio_balloon.c > @@ -438,8 +438,7 @@ vtballoon_alloc_page(struct vtballoon_softc *sc) > { > vm_page_t m; > =20 > - m =3D vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_WIRED | > - VM_ALLOC_NOOBJ); > + m =3D vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ); > if (m !=3D NULL) > sc->vtballoon_current_npages++; > =20 > @@ -450,7 +449,6 @@ static void > vtballoon_free_page(struct vtballoon_softc *sc, vm_page_t m) > { > =20 > - vm_page_unwire(m, PQ_INACTIVE); > vm_page_free(m); > sc->vtballoon_current_npages--; > } > diff --git a/sys/dev/xen/balloon/balloon.c b/sys/dev/xen/balloon/balloon.c > index fa56c86..0d2bba2 100644 > --- a/sys/dev/xen/balloon/balloon.c > +++ b/sys/dev/xen/balloon/balloon.c > @@ -94,13 +94,8 @@ SYSCTL_ULONG(_dev_xen_balloon, OID_AUTO, low_mem, CTLF= LAG_RD, > SYSCTL_ULONG(_dev_xen_balloon, OID_AUTO, high_mem, CTLFLAG_RD, > &bs.balloon_high, 0, "High-mem balloon"); > =20 > -struct balloon_entry { > - vm_page_t page; > - STAILQ_ENTRY(balloon_entry) list; > -}; > - > /* List of ballooned pages, threaded through the mem_map array. */ > -static STAILQ_HEAD(,balloon_entry) ballooned_pages; > +static TAILQ_HEAD(,vm_page) ballooned_pages; > =20 > /* Main work function, always executed in process context. */ > static void balloon_process(void *unused); > @@ -110,47 +105,6 @@ static void balloon_process(void *unused); > #define WPRINTK(fmt, args...) \ > printk(KERN_WARNING "xen_mem: " fmt, ##args) > =20 > -/* balloon_append: add the given page to the balloon. */ > -static int > -balloon_append(vm_page_t page) > -{ > - struct balloon_entry *entry; > - > - mtx_assert(&balloon_mutex, MA_OWNED); > - > - entry =3D malloc(sizeof(struct balloon_entry), M_BALLOON, M_NOWAIT); > - if (!entry) > - return (ENOMEM); > - entry->page =3D page; > - STAILQ_INSERT_HEAD(&ballooned_pages, entry, list); > - bs.balloon_low++; > - > - return (0); > -} > - > -/* balloon_retrieve: rescue a page from the balloon, if it is not empty.= */ > -static vm_page_t > -balloon_retrieve(void) > -{ > - vm_page_t page; > - struct balloon_entry *entry; > - > - mtx_assert(&balloon_mutex, MA_OWNED); > - > - if (STAILQ_EMPTY(&ballooned_pages)) > - return (NULL); > - > - entry =3D STAILQ_FIRST(&ballooned_pages); > - STAILQ_REMOVE_HEAD(&ballooned_pages, list); > - > - page =3D entry->page; > - free(entry, M_BALLOON); > -=09 > - bs.balloon_low--; > - > - return (page); > -} > - > static unsigned long=20 > current_target(void) > { > @@ -203,7 +157,6 @@ static int > increase_reservation(unsigned long nr_pages) > { > unsigned long pfn, i; > - struct balloon_entry *entry; > vm_page_t page; > long rc; > struct xen_memory_reservation reservation =3D { > @@ -217,10 +170,9 @@ increase_reservation(unsigned long nr_pages) > if (nr_pages > nitems(frame_list)) > nr_pages =3D nitems(frame_list); > =20 > - for (entry =3D STAILQ_FIRST(&ballooned_pages), i =3D 0; > - i < nr_pages; i++, entry =3D STAILQ_NEXT(entry, list)) { > - KASSERT(entry, ("ballooned_pages list corrupt")); > - page =3D entry->page; > + for (page =3D TAILQ_FIRST(&ballooned_pages), i =3D 0; > + i < nr_pages; i++, page =3D TAILQ_NEXT(page, plinks.q)) { > + KASSERT(page !=3D NULL, ("ballooned_pages list corrupt")); > frame_list[i] =3D (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); > } > =20 > @@ -245,8 +197,10 @@ increase_reservation(unsigned long nr_pages) > } > =20 > for (i =3D 0; i < nr_pages; i++) { > - page =3D balloon_retrieve(); > - KASSERT(page, ("balloon_retrieve failed")); > + page =3D TAILQ_FIRST(&ballooned_pages); > + KASSERT(page !=3D NULL, ("Unable to get ballooned page")); > + TAILQ_REMOVE(&ballooned_pages, page, plinks.q); > + bs.balloon_low--; > =20 > pfn =3D (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); > KASSERT((xen_feature(XENFEAT_auto_translated_physmap) || > @@ -255,7 +209,6 @@ increase_reservation(unsigned long nr_pages) > =20 > set_phys_to_machine(pfn, frame_list[i]); > =20 > - vm_page_unwire(page, PQ_INACTIVE); > vm_page_free(page); > } > =20 > @@ -286,24 +239,27 @@ decrease_reservation(unsigned long nr_pages) > for (i =3D 0; i < nr_pages; i++) { > if ((page =3D vm_page_alloc(NULL, 0,=20 > VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ |=20 > - VM_ALLOC_WIRED | VM_ALLOC_ZERO)) =3D=3D NULL) { > + VM_ALLOC_ZERO)) =3D=3D NULL) { > nr_pages =3D i; > need_sleep =3D 1; > break; > } > =20 > + if ((page->flags & PG_ZERO) =3D=3D 0) { > + /* > + * Zero the page, or else we might be leaking > + * important data to other domains on the same > + * host. > + */ > + pmap_zero_page(page); > + } > + > pfn =3D (VM_PAGE_TO_PHYS(page) >> PAGE_SHIFT); > frame_list[i] =3D PFNTOMFN(pfn); > =20 > set_phys_to_machine(pfn, INVALID_P2M_ENTRY); > - if (balloon_append(page) !=3D 0) { > - vm_page_unwire(page, PQ_INACTIVE); > - vm_page_free(page); > - > - nr_pages =3D i; > - need_sleep =3D 1; > - break; > - } > + TAILQ_INSERT_HEAD(&ballooned_pages, page, plinks.q); > + bs.balloon_low++; > } > =20 > set_xen_guest_handle(reservation.extent_start, frame_list); > @@ -438,7 +394,8 @@ balloon_init(void *arg) > /* Initialise the balloon with excess memory space. */ > for (pfn =3D xen_start_info->nr_pages; pfn < max_pfn; pfn++) { > page =3D PHYS_TO_VM_PAGE(pfn << PAGE_SHIFT); > - balloon_append(page); > + TAILQ_INSERT_HEAD(&ballooned_pages, page, plinks.q); > + bs.balloon_low++; > } > #undef max_pfn > #endif > --=20 > 1.7.7.5 (Apple Git-26) >=20 --N7OK10pvBj1bDp4V Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTpRioAAoJEJDCuSvBvK1BSxEP/1Oxdr7ActnPpw7M6pRHqXMe 63ZkiIdwovfBdABA/Qs/+EAgCuPxbZmaD3e3b/1ypi/qXihB4kbnWTTneXUei5NF 7Xot0YppkCmnKLiGiku3rV/biIFjmTJg5aCsrKdTpQfHCG1WInDwWZTW4UlMwkf2 j05qqBY6CeI8ckvmGDDZPpVAtnUnMI0AQOV2hve77kN3zWisxHG3qsBpCFIsDLAk 9eNBKasKHVWZlDTj1SHT9Fk3UgEqNOoSdOPgeMA+RMNAerLlg6uQXkdFZL8+nCA9 KzChvmglVepdC46qwai2UgkCr7ikEy6btijb4d+3pr0+XbzlfOCJ5nduwaYN72Uz 7qgxpX3AgagBIdLfv1zBtWX7K1fbQVkCLIe0pwPqKw8fAUP1lu2TmjsU+Ziq6wLa ncN0Lafh3sihUt6rsrW2GGFTtXiWqJqtb2QHdLYgxA7F27OupxLXl+ZXFgqTHSdc BXOFsE6NxtGkf8U9dnNHgHVDDyXKTMzy8/VblRxw2VIBSPV4ilCOXuE8E0+3/ASg mFZv2o8Yr7SO5OHouZAdKYvtW29Ch4YYl1HvJJt+ZHgZmvGQiv9HUqzPJvgOGrKK tNth+Vcbp14xEhgHrnuJeymtAa39hhBXJVvgOQJRJsWpyhhycLCAaIgr8h0qC8A/ Ax1qZuC61xLOmFWRHPMf =fFv2 -----END PGP SIGNATURE----- --N7OK10pvBj1bDp4V-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 21 13:54:37 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39F9BDBB; Sat, 21 Jun 2014 13:54:37 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8580424B7; Sat, 21 Jun 2014 13:54:36 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id c11so2975932lbj.41 for ; Sat, 21 Jun 2014 06:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7kxtJO3Okbi6E5l0FYUVUnuAyf5YEQMAl+BbOnIhYs8=; b=kaDDcTfiOgjKFl0p9wHqQRmgGvpBhISnxmN/9HWWu8wSpsqcGmWy7jsA6KUcGj+C1U yVmoXUs9pHIbzSWRNP1EuOaslTZK6hP1JWESog0elc5Fz4cnlk42rMv5vAP/KtCTuvKY FgcdvAbwEVu3tbkfA/lK9MWgTmxGfJnIewi8C/ji69NvOu0JjVSXT1eyg7Onepld1h/v xGvDnpBhHzx8qN7OlBurUG0n2wXmtUVxKiHVqpXyVcyd8Ku3+xzHnNjx/d67WfX7sTHp nkqMWa81NNpXHvfAsxRUzVQ7RxwYMcf+8BIjPB4GAY4nFridwmMuEg7q50gNkxtQ/46+ aNpw== X-Received: by 10.112.55.97 with SMTP id r1mr1667130lbp.74.1403358873441; Sat, 21 Jun 2014 06:54:33 -0700 (PDT) Received: from kloomba ([95.104.128.132]) by mx.google.com with ESMTPSA id ut18sm8977177lbb.38.2014.06.21.06.54.31 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jun 2014 06:54:32 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sat, 21 Jun 2014 17:52:01 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt, bhyve, and Jenkins configuration problems Message-ID: <20140621135159.GA47374@kloomba> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 13:54:37 -0000 --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > Hi, >=20 > I am trying to configure Jenkins to spawn bhyve VM's on demand > using libvirt. >=20 > I did the following: >=20 > (1) Looked at Roman's blog article: >=20 > http://empt1e.blogspot.com/2014/03/using-jenkins-libvirt-slave-plugin-wit= h.html >=20 > (2) Followed the libvirt configuration steps here: >=20 > http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html >=20 >=20 > Here are the screenshots of my Jenkins setp for libvirt: >=20 > http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html >=20 > However, after going through all that, > when I tried to configure a job to run on the libvirt node, > it did not start. >=20 > Any ideas what I am doing wrong? Hi Craig, Some questions: When you configure the connection in the 'Add a new Cloud' dialog, does 'test connection' work for you? I don't see a host and credentials specified, I'm not sure if jenkins libvirt plugin works with the local connection. In my setup, I have specified the host and the credentials like this: http://people.freebsd.org/~novel/misc/jenkins-libvirt-01.png Then, the new node screen doesn't look complete as well. There, one has to choose one of the pre-defined domains, specify its IP address and credentials so Jenkins could ssh to it and run its agent. Also, one has to install JDK as the agent is Java-based. As I've described in my post, I did it using: pkg install java/openjdk7 command in the guest. Here's a config of my sample node: http://people.freebsd.org/~novel/misc/jenkins-libvirt-02.png After that in the node page one could start the agent and if ssh credentials and IP address of the VM are correct, the agent will successfully start: http://people.freebsd.org/~novel/misc/jenkins-libvirt-03.png Then in the nodes list it should be up and ready for the builds: http://people.freebsd.org/~novel/misc/jenkins-libvirt-04.png Note: depending on the policy specified for node, it might be turned off when idle so it'll be reported down in this list until when new jobs for it appear. Roman Bogorodskiy --qMm9M+Fa2AknHoGS Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTpY3/AAoJEMltX/4IwiJqBXgH/3GOEQ1HI5aVrdnlABaFfTzC Oa0H5opoFQVmAqIfRmcxpVEf3kKezGft2W1w1oHwpxXu6GZP08hbw5eo2rMdwX6N azRKeinxIMyPKe9krS8wlHb8USkyBmDn7kr4B3zdzS7H0IQ1KKl/4oI+Z01Gyodr YzR30MuKQe9e46ASlK8/TYriHvBTfqq60ODfN2n+MCr3EGm1zvwNKwGBUb1kufdT 9qNAru8i6sL6oUKNPvr1nuJjLuARrQ7JKeNK9uIpCoOqpTlPW7Dqan3IiB2UvOBP KlyWLyOuZjyTFiM5GHb2pdNfssNdjS67tLMz2ZxXc9+qUmooMnjoGdjNpVx10Q8= =zs0W -----END PGP SIGNATURE----- --qMm9M+Fa2AknHoGS-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 21 15:44:46 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48EAA98D; Sat, 21 Jun 2014 15:44:46 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3811E2CD9; Sat, 21 Jun 2014 15:44:45 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ty20so3049867lab.25 for ; Sat, 21 Jun 2014 08:44:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=WoYwW6XgACJr912Xzt1mXCoURyLSg8BShJGlmAfmrIA=; b=PfpwR5KowhaLh0LpgUk0636PaHMUlIWU6k3AKSyYUX2GGqmAepnAJBCwgPBzE4gT2O HHO9dhpluwhC/ss35obYscdVpnckTZFbCRWlAIyFX6wOovBeUCoUOwBQ5auY2WncjRt1 gmQLAEcPZY8hAF6hLWySZ1BEEMqgg0ZbKI6MBtAkzpKiLFbNGaWB9Ig2E2lI8rtCa1hw 98GxDZc37339Mref32mv7JGX4ifQKg/akDi6SdVo1rl04DEM1WEZXgAyMw1EN6S687Vy YdZOolWa8RUwkQconbiU3T2cdjjIZkZYkeJUePf3Jcg3Qg6MPyl7d0k/4d1T9MkTmg4O m3Yg== X-Received: by 10.112.164.146 with SMTP id yq18mr7788242lbb.5.1403365483079; Sat, 21 Jun 2014 08:44:43 -0700 (PDT) Received: from kloomba ([95.104.128.132]) by mx.google.com with ESMTPSA id ob4sm9165650lbc.30.2014.06.21.08.44.41 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jun 2014 08:44:42 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sat, 21 Jun 2014 19:42:13 +0400 From: Roman Bogorodskiy To: Bartek Rutkowski Subject: Re: Request for testers: virt-manager, libvirt, and bhyve Message-ID: <20140621154211.GB47374@kloomba> References: <20140619054456.GA1851@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/NkBOFFp2J2Af1nK" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: jgh@FreeBSD.org, "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 15:44:46 -0000 --/NkBOFFp2J2Af1nK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Bartek Rutkowski wrote: > I am happy to look at updating the virt-manager - any chance you could > point me to what from the below list was the dependency causing > issues? >=20 > dbus>=3D0.83.2 > libxml2>=3D2.7.8 > virtinst>=3D0.600.4 > vte>=3D0.26.2 > gtk-vnc>=3D0.3.10 > librsvg2>=3D2.32.1 > libvirt>=3D0 Hi, I've looked at updating it to 1.0.1. Apparently, everything seemed to work fine, either something was updated since I've looked at it last time or I missed something. :-) Could you please review it: http://people.freebsd.org/~novel/misc/virt-manager-101.diff I did a basic testing and it seemed to work fine. I haven't yet backported my patch from the git master to add 'bhyve' into menu list, but one could still connect using: $ virt-manager -c "bhyve+ssh://root@localhost/system" Roman Bogorodskiy --/NkBOFFp2J2Af1nK Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTpafTAAoJEMltX/4IwiJqO1oH+gP31BxMG38dyuRjnXYS8S72 lf8gBTbJ3/XhJqmmrXdq1LfN4X48MqDgxmCPomQbBAzt/ToAx8qU47PWt4g9YZBZ AfxSHtrXp5TqCa427YM9ucW7E9dtXl5T1PbIOV29cl7oFuhl/2j+QfVkjrD/Y3cM 7DvbRcZtM1DY72IAtcImYxfjxvKdW5rkY9IdGJLacuMAZUDSrTBs6S4C+ATm5y/C LahrfA8SHfBH87gdzoUJNkmwt/Q3C4pFbkS+kHOia9HscQjGktLSnDsg244+M3ww dgg32O8wW6AM9vxoI6ZGMgECH5cP5VGIoH9/v2Oqyrq2tlo8zykPOg1QbdudpKo= =F6ra -----END PGP SIGNATURE----- --/NkBOFFp2J2Af1nK-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 21 17:51:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCC45C9C; Sat, 21 Jun 2014 17:51:17 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0759B25E5; Sat, 21 Jun 2014 17:51:16 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so3169749lan.16 for ; Sat, 21 Jun 2014 10:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=vOjWQ3SJvCplvSn1yL9D+isJJdwo6bEdYSlHtbnX15k=; b=hLJmfRSR0jRWPOP4PpCcNDGJegQKKzuDJHWC1n+IjKCPVmxzvkAE+AbgBVavD0ztZ/ QZExPHHSneYsYeWn/e4P643Btz4qTi0ez1mghOTO+nlTpfNlFwKehCWRI+NPUUH0s+Ou DLAAtSZkibQn1c6QHGoZ4wBYwsUYErHGJL3NDSNEH/yyNJuGBVK5GUXL3LTxF7Qo/g5k YZIvFRFDhX/XC+2wP83o/JqiumvVdL4A/DmbLx5tSAeCPJ4ydXwQKwrDRvzk4b+Vf6BE cwehtltH28Xc9Kup1Jq9SiUEkIdIDdxKYnZF0C5KqxyTjXY0VQHEet5Cpcgguq3V8+XJ MnuQ== X-Received: by 10.112.34.243 with SMTP id c19mr2612769lbj.57.1403373074684; Sat, 21 Jun 2014 10:51:14 -0700 (PDT) Received: from kloomba ([95.104.128.132]) by mx.google.com with ESMTPSA id ut18sm9375689lbb.38.2014.06.21.10.51.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jun 2014 10:51:13 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sat, 21 Jun 2014 21:48:44 +0400 From: Roman Bogorodskiy To: Craig Rodrigues , jgh@freebsd.org Subject: Re: Request for Testers: libvirt and bhyve Message-ID: <20140621174842.GC47374@kloomba> References: <20140619054935.GB1851@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3siQDZowHQqNOShm" Content-Disposition: inline In-Reply-To: <20140619054935.GB1851@kloomba> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 17:51:18 -0000 --3siQDZowHQqNOShm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Roman Bogorodskiy wrote: > Craig Rodrigues wrote: >=20 > > Hi, > >=20 > > I've had some success with getting libvirt and bhyve to work. > >=20 > > I wrote up a list of the steps that I followed here: > >=20 > > http://people.freebsd.org/~rodrigc/libvirt-bhyve/libvirt-bhyve.html > >=20 > > If anyone is interested, please try it out, and provide feedback to this > > list. It would be great to get more testers. > >=20 > > Roman, when do you think you will be able to commit all your latest > > fixes into the devel/libvirt port Makefile? It would greatly increase > > the usability of the port. Thanks for doing all the work to add bhyve > > support to libvirt!! >=20 > Hi! >=20 > I hope to get a clean and ready patch for the port this weekend and will > send it to jgh@. >=20 > Thanks a lot for your testing and the feedback! Hi, Here's the updated patch: http://people.freebsd.org/~novel/misc/libvirt_port_cleanup.diff Jason, could you please review this? Summary of changes: - Make sure not to build uml driver as it's useless on FreeBSD - Enable bhyve by default on 10.0 and newer on amd64 - Fix sample config handling to work with staging - Fix bhyve detection in the configure script - Add a patch to fix segfaults in case of problems with bridge creation - Enable networking port and install default networking definition; add dnsmasq dependency for that - Bump PORTREVISION This patch was created by Craig and modified by me. Roman Bogorodskiy --3siQDZowHQqNOShm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTpcV6AAoJEMltX/4IwiJqk80IAKSObI4Pb/vkYXmkrOZ13c0I Xds2yvkGpIt8FOZ28UOZ0/xeczQ92v0c36DnZbki2pivP+0PJj7EeV9AyMsrOibR nGgGfEH7xUJNX1mjTq0iSe2yjmTFOsoCAhp3rLyefZnmByeVa6lnLvBPPusOb+TP PJHIYYur3ruqRxjOP2PoA2/sBX8Dmwm1hYvFRsg8a03+U6HXwYb+xwXZtYFB8IlM 6Ba3adLpB/VxMbi2aniZtZJtZwke11VgIwO0FRACwQ0T/rQWdhc3BggW9ftT2yp2 Yj+bqExElpSfILCtns4zFGOgCd1chwk4htWA8bozKN7PX7S/D9F1d6WD9KlI6QM= =/Ilx -----END PGP SIGNATURE----- --3siQDZowHQqNOShm-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 21 20:50:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 183A2564; Sat, 21 Jun 2014 20:50:28 +0000 (UTC) Received: from unsane.co.uk (unsane-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:110::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "unsane.co.uk", Issuer "unsane.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A2FA72332; Sat, 21 Jun 2014 20:50:27 +0000 (UTC) Received: from vincemacbook.unsane.co.uk (vincemacbook.unsane.co.uk [10.10.10.20]) (authenticated bits=0) by unsane.co.uk (8.14.8/8.14.8) with ESMTP id s5LKoN7x036764 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 21 Jun 2014 21:50:23 +0100 (BST) (envelope-from vince@unsane.co.uk) Message-ID: <53A5F00F.2060004@unsane.co.uk> Date: Sat, 21 Jun 2014 21:50:23 +0100 From: Vincent Hoffman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: Request for Testers: libvirt and bhyve References: <53A42F70.7040707@unsane.co.uk> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 20:50:28 -0000 On 20/06/2014 18:48, Craig Rodrigues wrote: > On Fri, Jun 20, 2014 at 5:56 AM, Vincent Hoffman wrote: >> Hi Craig, >> I've given it a try and its looking good. > > Nice! > Do you have time to try virt-manager: > http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html > At the moment I dont have anything setup I could test it on. If I do I'll let you know. Vince >> Thanks for all the work on this. > Roman did all the hard work to get bhyve support into libvirt, > and upstream his changes to the libvirt project. All the thanks go to him! > > -- > Craig > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 23 03:19:56 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B27BAC6; Mon, 23 Jun 2014 03:19:56 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 54B5F2511; Mon, 23 Jun 2014 03:19:55 +0000 (UTC) Received: from Julian-MBP3.local (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5N3Jg1c090512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 22 Jun 2014 20:19:46 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53A79CC9.6080005@freebsd.org> Date: Mon, 23 Jun 2014 11:19:37 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Peter Grehan , Cory Smelosky Subject: Re: bhyve: vde2/openvswitch References: <53A474C7.9030909@freebsd.org> In-Reply-To: <53A474C7.9030909@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 03:19:56 -0000 On 6/21/14, 1:52 AM, Peter Grehan wrote: > Hi Cory, >> Is there any current support for vde2 > > None yet. > >> aside from using vde2_plug2tap? >> (Does that even work? I believe vde2 likes to open the tap interface >> exclusively. I need to bring up a bhyve test environment to check >> fully.) > ... >> How exactly does the bhyve network stack work through bridging? I'm >> imaging it's tying in to bridging APIs? > > bhyve uses a tap(4) device. Since that shows up as an ifnet to the > rest of the system, it can be used in any way an ethernet interface > can be used. For bridging, the bridge(4) device is used to join > tap/ethernet interfaces into an L2 bridge. > >> I'm assuming bhyve-to-bhyve network communication is also possible? > > Yes, though currently that is through tap/bridge devices. > >> I can't imagine it'd be too hard to implement vde2 support for >> networking...it'd give me more control over my VMs and give easier >> integration with SIMH VMs. > >> A virtual switch for bhyve would be a useful feature if there >> aren't any >> plans for one. Get up there and have it in base. ;) > > This has been talked about quite a bit, though nothing concrete. Allowing bhyve to talk to netgraph would instantly give this. I think you can already do it by hooking the tap device into the netgraph graph. (I haven't done this for ages bt it used to work) at one time virtual box could also use netgraph for its networking. Not sure if it still can. > >> Unrelated: any plans for a special bhyve-to-bhyve communication bus? >> Not too useful of a feature...I can't think of any use for it off-hand >> aside from dedicating a VM to administrating VMs without exposing much >> to the host beyond configurations. > > There should be code going in soon that allows the netmap VALE > switch to be used for high-speed VM-VM communication. This work also > allows different network back-ends to be written, so it should be a > simple task to drop in a vde2 driver. > > later, > > Peter. > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" > > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 23 19:21:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F11EC2D1; Mon, 23 Jun 2014 19:21:02 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id B37812A21; Mon, 23 Jun 2014 19:21:02 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 33131126DE; Tue, 24 Jun 2014 05:20:55 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.3.7-GA) with ESMTP id BVI00935 (AUTH peterg@ptree32.com.au); Tue, 24 Jun 2014 05:20:52 +1000 Message-ID: <53A87E12.3050201@freebsd.org> Date: Mon, 23 Jun 2014 12:20:50 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Julian Elischer Subject: Re: bhyve: vde2/openvswitch References: <53A474C7.9030909@freebsd.org> <53A79CC9.6080005@freebsd.org> In-Reply-To: <53A79CC9.6080005@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 19:21:03 -0000 Hi Julian, >>> A virtual switch for bhyve would be a useful feature if there aren't any >>> plans for one. Get up there and have it in base. ;) >> >> This has been talked about quite a bit, though nothing concrete. > Allowing bhyve to talk to netgraph would instantly give this. I think > you can already do it by > hooking the tap device into the netgraph graph. (I haven't done this for > ages bt it used to work) > at one time virtual box could also use netgraph for its networking. Not > sure if it still can. While I don't doubt the utility of being able to hook VMs to netgraph, what I was thinking (hoping) was something like a cloneable tap-like interface to bridge(4) that avoided having to create discrete tap interfaces and bind them to bridge. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 23 21:13:27 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11705B21; Mon, 23 Jun 2014 21:13:27 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E12C7250B; Mon, 23 Jun 2014 21:13:26 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id F1D76B91F; Mon, 23 Jun 2014 17:13:24 -0400 (EDT) From: John Baldwin To: freebsd-virtualization@freebsd.org Subject: Re: bhyve crash Date: Mon, 23 Jun 2014 12:58:54 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140620131323.GC83357@mouf.net> In-Reply-To: <20140620131323.GC83357@mouf.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201406231258.55006.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 23 Jun 2014 17:13:25 -0400 (EDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 21:13:27 -0000 On Friday, June 20, 2014 9:13:27 am Steve Wills wrote: > Hi, > > Just had a bhyve crash on 11-CURRENT. The host is r267362. The guest is > r266947. The crash was: > > pid 38256 (bhyve), uid 0: exited on signal 6 (core dumped) > > (The formatting of this may be off a bit) > > vmexit[12] > reason VMX > rip 0xffffffff80d9e11d > inst_length 3 > status 0 > exit_reason 2 > qualification 0x0000000000000000 > inst_type 0 > inst_error 0 > > I can provide the core dump file. Ideas? The backtrace from the core would be good as signal 6 is usually an abort() (such as from an assertion failure). -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 25 00:31:35 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99ADB59E; Wed, 25 Jun 2014 00:31:35 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 483942122; Wed, 25 Jun 2014 00:31:35 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id m20so1063375qcx.27 for ; Tue, 24 Jun 2014 17:31:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ot1A/1X0omLFIj1ZP4GS/JKPWK+DA+ZzlZE0WN1nE00=; b=d54dkFx2sjy0j07Tz4+g+7iyf2VshhN92tnP4Grb7amdbr881YFDzDZAmo+M3mYAz0 ab566AZKJZMWjWsKcnK6mIkOsyE6u376ksl4HckF85TIuvyBP8Ga3rgvKxgxzzmhSDmN QdbzgXyUkHxap4StjXpdWmWYZ/4d+F4ZyRmkTgIjhr2iMtqVT/V2CYTXPsz9/CyQyrf2 WYXgxTWn5y8tMp0HgSo5fPuxZ5y9Mjo5HlM1NIQH4vNmpDc9cjOjmfD5wn2iKU8cT/kx eEmg28nP+YXziizf8HZR/5ZD0HcpJsLzr5wxIlnV3LAQ6OOnz9UkQQE10v0/zYl/cyYm c6zw== MIME-Version: 1.0 X-Received: by 10.224.47.148 with SMTP id n20mr7136736qaf.90.1403656294472; Tue, 24 Jun 2014 17:31:34 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.140.30.165 with HTTP; Tue, 24 Jun 2014 17:31:34 -0700 (PDT) In-Reply-To: <20140621135159.GA47374@kloomba> References: <20140621135159.GA47374@kloomba> Date: Tue, 24 Jun 2014 17:31:34 -0700 X-Google-Sender-Auth: DyvbOY2SlyXjHMWtmZIwAuAFyA4 Message-ID: Subject: Re: libvirt, bhyve, and Jenkins configuration problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 00:31:35 -0000 On Sat, Jun 21, 2014 at 6:52 AM, Roman Bogorodskiy wrote: > > > When you configure the connection in the 'Add a new Cloud' dialog, does > 'test connection' work for you? I don't see a host and credentials > specified, I'm not sure if jenkins libvirt plugin works with the local > connection. > OK, following your example, I have gotten further along. I have updated my setup instructions here: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html In "Step 3", when I click on "Test Connection", I see OK: bhyve+ssh://root@localhost:22/system?no_tty=1 version=11000000 so that seems to work. For Step 5, when I enter the Host and Credentials information, does this information need to be the IP address and login information for the VM that has been spawned, and not the info for libvirt, as specified in Step 3? Does that mean that I need to configure my VM, so that it comes up with a known IP, so that it can be entered on this configuration page? Also, the VM, that is spawned, it must have openjdk installed? Does that mean that after libvirt starts the VM, Jenkins will try to SSH into the VM and start a Jenkins job by invoking java? Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 25 17:48:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 860A9AD2; Wed, 25 Jun 2014 17:48:42 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7FA02233; Wed, 25 Jun 2014 17:48:41 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id mc6so1010362lab.38 for ; Wed, 25 Jun 2014 10:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=ynObKZzrsOdM+0714hrRRiQw7Llh8CjKLoFBvqlYOn8=; b=fLm9vYVYQzX6deE5XXw63+7Uf9AKYgn3zABSsAC98f3kJVXHlvfQJC7Yxt6E3A4jIM ndi9tXfQIZ3Gc5yZ8IRqE3bAVQgIAnWlTE5cTbpYP9bEh3+JxHPgT++GJTm8kqyB6i0k KSCbURli7LtHGFRmlBB8DIFdWHU1lT/xBjGiC0h6Uzl7KjxoK5gD6Ab1rzh2EvbyuHr3 NkylPLxKUCsXqooiCl0HgHRbcBtYxVMAQJtVdMYz9WEODNRRllfzRphTpbjnSel6OxkG QdtEAL006AbSG3tOt5o15y2oBmH5pJXbCe7ml7YOvg+XEQzzn/Lvw36V5Re7OOqxq6t+ gDPw== MIME-Version: 1.0 X-Received: by 10.152.88.41 with SMTP id bd9mr1992067lab.91.1403718519618; Wed, 25 Jun 2014 10:48:39 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Wed, 25 Jun 2014 10:48:39 -0700 (PDT) Date: Wed, 25 Jun 2014 10:48:39 -0700 X-Google-Sender-Auth: Io_rmli2BZ_5eAa5ZBJOad3mehQ Message-ID: Subject: py-libvirt problems with bhyve From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" , freebsd-python@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 17:48:42 -0000 Hi, I did the following: (1) Set up libvirt, taking Roman's latest port, and configuring bhyve support: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html (2) Installed the py-libvirt port. I wrote this code: import libvirt import sys conn = libvirt.openReadOnly(None) if conn == None: print 'Failed to open connection to the hypervisor' sys.exit(1) try: print conn.listAllDomains() #print dir(conn) dom0 = conn.lookupByName("bhyve") except: print 'Failed to find the main domain' sys.exit(1) but when I ran the code, I got this error: [] libvirt: QEMU Driver error : Domain not found: no domain with matching name 'bhyve' Failed to find the main domain Can anyone tell me how to write a py-libvirt program to start/stop a bhyve VM? Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 25 18:04:02 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4ECC7110; Wed, 25 Jun 2014 18:04:02 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DF9E2400; Wed, 25 Jun 2014 18:04:01 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so2075811lbi.24 for ; Wed, 25 Jun 2014 11:03:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=HjlqYnDyIIoOIzqwNcesYbkHayS1GguJg1XuAKg1mP0=; b=VvyEzE9dCUoqEsxcMBa7aFkMHN51xB0nlY7qj5MlbVfI98aM/h88cCMZO3RfKgnlb/ pBJ+mXwUts45mEpaOelJhLOaFXMH3s30tyiCSqFfrpcHoLLANLT/PyFs8tercEaAZKv2 5ddv8KC20PpacXzrrE25215aRMQTOZTPlLFpBWjySgKsWNtpnKJ0QRNKdbwZ1EydYVTc tgP9ZrFSohy7WYS5pGPjhkV9q4DAwaBJu+OMqJgIp++5chtlV/exuCfVhfihevO4I4Pl ov9mP4RhWWzUgBTCBsv36xrdbahJS/WfimT70/TgNlemfL5n02uQPqtPH5EeUDJtyd3H u/yA== X-Received: by 10.112.53.161 with SMTP id c1mr2050733lbp.86.1403719439381; Wed, 25 Jun 2014 11:03:59 -0700 (PDT) Received: from kloomba ([95.104.140.69]) by mx.google.com with ESMTPSA id ba10sm4339535lbc.47.2014.06.25.11.03.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 11:03:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 25 Jun 2014 22:01:18 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: py-libvirt problems with bhyve Message-ID: <20140625180116.GA1304@kloomba> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" , freebsd-python@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 18:04:02 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > Hi, >=20 > I did the following: >=20 > (1) Set up libvirt, taking Roman's latest port, and configuring bhyve > support: >=20 > http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html >=20 > (2) Installed the py-libvirt port. >=20 >=20 > I wrote this code: >=20 > import libvirt > import sys >=20 > conn =3D libvirt.openReadOnly(None) ^^^ I think here the URI should be 'bhyve:///system'. Or you could tweak libvirt.conf and assign the bhyve url value to the 'uri_default' parameter. http://libvirt.org/uri.html#URI_default > if conn =3D=3D None: > print 'Failed to open connection to the hypervisor' > sys.exit(1) >=20 > try: > print conn.listAllDomains() > #print dir(conn) > dom0 =3D conn.lookupByName("bhyve") > except: > print 'Failed to find the main domain' > sys.exit(1) >=20 >=20 > but when I ran the code, I got this error: >=20 > [] > libvirt: QEMU Driver error : Domain not found: no domain with matching na= me > 'bhyve' > Failed to find the main domain >=20 >=20 > Can anyone tell me how to write a py-libvirt program to > start/stop a bhyve VM? >=20 > Thanks. > -- > Craig > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" Roman Bogorodskiy --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTqw5sAAoJEMltX/4IwiJqRuAH/jkzaoczuYaXU0pED1tfXyUN QRj33ooCdSYxqAYjQwc6YlGN2lVTnB6FVd4Dy1jkxJ+N6xruZ1OShRmEWtDkDmjx Yil0Fp9nm99f6iT/wY+yH1Tmla7MX+Bq/W4h3qAgqUOGUSDA3J35gYFFmBTA26H7 j4Caq8NeiQOxwcG8XC8Sb5h4ES3x9mW8B3f0gPQo8T+wsnsc1S/S1mCBSRh9W9nM /yf1vGLctwanSYAsg2pxw/Db0oFuB1qOdErqTVzNuqa8AVHIJNnLvG4E8DMnfetb /tz1RE0MLdA5CQB9E4HlakscH8s5wH644m9EQ+UQDTPjiM+CffbksXQcqmrZUFA= =QSSM -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 25 18:10:52 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C553290; Wed, 25 Jun 2014 18:10:52 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9643C247C; Wed, 25 Jun 2014 18:10:51 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id c11so2154219lbj.13 for ; Wed, 25 Jun 2014 11:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=qBat1SWOBgPFyPoSPYERFXg8RasuzWxnkw/vWdpOKv0=; b=TvkhRMl2c8g9x66i4s9vZr529iyHQw//BP78Smf7WYWAREsYjINANarmTn6GzC6NJr FqojNyDiAIj4ZeG6Mdt1g3GiZBaxXNpSxnRYdVzMGyFb3kPTgY9657zgZ8pNY0x6n7as rIrYFGnhAryYLLGYPfmJxtv3s2ofiBILu8F3X4jkVSpUnkdqomcEXVmn7nCJqt4Tr9iy 7OX4poTiWJBFkDdH3Dt+OPYICtNxsXYj8idFVbxTI3vf25pa/5f9j1vzejHs/Oa1vf0p cD+5bcnpPvyyDaeC5RaBS+m67rsH046nlFkOcooxXS8j1OxkPyMJb081o+17uQdpBFBi 7FhA== X-Received: by 10.152.27.66 with SMTP id r2mr6990539lag.53.1403719849386; Wed, 25 Jun 2014 11:10:49 -0700 (PDT) Received: from kloomba ([95.104.140.69]) by mx.google.com with ESMTPSA id h11sm4373022lbm.13.2014.06.25.11.10.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 11:10:48 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 25 Jun 2014 22:08:12 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt, bhyve, and Jenkins configuration problems Message-ID: <20140625180810.GB1304@kloomba> References: <20140621135159.GA47374@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="24zk1gE8NUlDmwG9" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 18:10:52 -0000 --24zk1gE8NUlDmwG9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Sat, Jun 21, 2014 at 6:52 AM, Roman Bogorodskiy > wrote: >=20 > > > > > > When you configure the connection in the 'Add a new Cloud' dialog, does > > 'test connection' work for you? I don't see a host and credentials > > specified, I'm not sure if jenkins libvirt plugin works with the local > > connection. > > >=20 > OK, following your example, I have gotten further along. > I have updated my setup instructions here: >=20 > http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html >=20 > In "Step 3", when I click on "Test Connection", I see >=20 > OK: bhyve+ssh://root@localhost:22/system?no_tty=3D1 version=3D11000000 >=20 > so that seems to work. >=20 > For Step 5, when I enter the Host and Credentials information, > does this information need to be the IP address and login information > for the VM that has been spawned, and not the info for libvirt, as > specified in Step 3? Correct. > Does that mean that I need to configure my VM, so that it comes up with a > known IP, so that > it can be entered on this configuration page? Correct. Though, normally you don't need to perform any special configuration for the IP address as long as the networking configuration is correct. The VM just needs to be configured to obtain an IP address via DHCP. Later on it should be persistent in dnsmasq. An IP address could be obtained either from the dnsmasq leases file or =66rom arp cache for example. > Also, the VM, that is spawned, it must have openjdk installed? Does that > mean that > after libvirt starts the VM, Jenkins will try to SSH into the VM and start > a Jenkins job > by invoking java? Yes, it's true, it starts its agent which does the job. PS Maybe there are other ways to push Jobs to Jenkins nodes, but I didn't look close into that. Roman Bogorodskiy --24zk1gE8NUlDmwG9 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTqxAKAAoJEMltX/4IwiJqRaoH/3Obhk23RlYWUbgPIxHXZZU+ n/U8bq1q29tdN8sK5tVWA74fqscPszgYSeZnSA76tVVRG5H1R7k3cZdT16uqN6Mt GPlKMttTKdO8n8aJZZbwm3/Xvu5MJTYgDC6ybIpU8T5dsbMFHzAK31DlVXZXUivi 4h+bVPhPDrnjAiHVzcerW2GxWJosp57MHu6N/DBxg+W+JZ5g4s261mgk2ABdpMjf o+pJ49bN0/LuHsKndro0VdR4Cgjfi1ZaXJ0sHrgWP7tY8V39g9a8rrLWVLfBy97j xdnQ2CxvPea+zI3Oo7s5X4ANJOqGfPZqdxYJN8qUrQuwdwh7TfeNc7HdFepqHhg= =6r/K -----END PGP SIGNATURE----- --24zk1gE8NUlDmwG9-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 25 20:28:58 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DF0C1BF; Wed, 25 Jun 2014 20:28:58 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC2F621F6; Wed, 25 Jun 2014 20:28:57 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so2202503lbi.24 for ; Wed, 25 Jun 2014 13:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=rS4wLdAEbX0sqKrBcA1PP94EvjOwzs83cJ4nw6KHtJk=; b=wa0LYJCoCKHKsbKYh4Q+QbBDMUZ4VhHGR0olvLl5wCMbSSrb6qgwWtUffmg+mu5woZ b3f/4DIs35mU5PhI59/87db9jC/ndlvPGf2qEOmW8IXOi1e/W8hqe036Qqeb9y/YhwQV lVDT7J2d30YBJuvgHXUyGVpRzlly57dMVX9QPSup/gakLCKl/uCZIcgBSDWKPCBE9CW5 ub4etA0aFywC8Q6diike2UbR6TSKE6GfRSdyQbd4nsVbkP8W5EWrKYpKK43rF1nsqoeO 1MTO6BnSbxhrmXQDAOrAWTZXUtCYT1B+eOe6hf9rIep0L5KlP+GU40QtDBs3s9unJ+KK 9dXA== MIME-Version: 1.0 X-Received: by 10.152.28.194 with SMTP id d2mr7492228lah.25.1403728135680; Wed, 25 Jun 2014 13:28:55 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Wed, 25 Jun 2014 13:28:55 -0700 (PDT) In-Reply-To: <20140625180810.GB1304@kloomba> References: <20140621135159.GA47374@kloomba> <20140625180810.GB1304@kloomba> Date: Wed, 25 Jun 2014 13:28:55 -0700 X-Google-Sender-Auth: 3Mzmy8xSw0-xL3ccKsEcp4wS2Q0 Message-ID: Subject: Re: libvirt, bhyve, and Jenkins configuration problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 20:28:58 -0000 On Wed, Jun 25, 2014 at 11:08 AM, Roman Bogorodskiy wrote: > > Does that mean that I need to configure my VM, so that it comes up with a > > known IP, so that > > it can be entered on this configuration page? > > Correct. > > Though, normally you don't need to perform any special configuration for > the IP address as long as the networking configuration is correct. > > The VM just needs to be configured to obtain an IP address via DHCP. > Later on it should be persistent in dnsmasq. > > An IP address could be obtained either from the dnsmasq leases file or > from arp cache for example. > But how does the Jenkins libvirt plugin know the IP address for the VM, unless you enter it in this field in the Jenkins UI? > > > Also, the VM, that is spawned, it must have openjdk installed? Does that > > mean that > > after libvirt starts the VM, Jenkins will try to SSH into the VM and > start > > a Jenkins job > > by invoking java? > > Yes, it's true, it starts its agent which does the job. > > PS Maybe there are other ways to push Jobs to Jenkins nodes, but I > didn't look close into that. > The easiest way is to have the openjdk installed in the VM, so that Jenkins can ssh in and run jobs. However, there is also the Publish Over SSH Plugin ( https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH+Plugin ) which allows Jenkins to SSH into a remote host and run different commands, instead of starting a full Jenkins job, which requires the openjdk to be available. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 10:43:19 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A876B67; Thu, 26 Jun 2014 10:43:19 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6468A29A7; Thu, 26 Jun 2014 10:43:18 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id cc10so775668wib.0 for ; Thu, 26 Jun 2014 03:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type; bh=Dy8QKDZ03Ikv8daEWWsW9jtdwrbpqHCdQcAOpbUtbxc=; b=pM9qLO4dk/fJy6cKKzo7Cc+f4Z6FF8lF+lWg6ZzwI9a0pnnovjjNvDSaiZMKsGz5MM yiRAxORouIOnr3Ank/NkDE/POvLW4wUasSIxeH58MZ2j/yKKQZ3mWnw2/Tnft8nEKJmd oTiI9cSyTT5bUPYp5pCuMILVAznZX3H95i0l83mvqHlSuJUS1Up8igFMfsZ+JjE+sTyM SwotdFAw2of+AJ71KFNbDMLAPSjZDRnOwZ4aXYHcKm8xt/T4+7nbBegJi1i4XMd5TUKH 32uMDsSvAEUPgPezmUyqxIUurguPi2GdxxHFv49EPh6M9/ttAfvcuK/5H6rysvEeDS8p jWHA== X-Received: by 10.194.236.132 with SMTP id uu4mr4722250wjc.54.1403779396200; Thu, 26 Jun 2014 03:43:16 -0700 (PDT) Received: from [172.16.4.106] (21.Red-80-59-189.staticIP.rima-tde.net. [80.59.189.21]) by mx.google.com with ESMTPSA id cz4sm64410454wib.23.2014.06.26.03.43.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 26 Jun 2014 03:43:15 -0700 (PDT) Message-ID: <53ABF85F.8070106@gmail.com> Date: Thu, 26 Jun 2014 10:39:27 +0000 From: Jacques Cale User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org, vince@unsane.co.uk, rodrigc@FreeBSD.org Subject: Re: Request for Testers: libvirt and bhyve Content-Type: multipart/mixed; boundary="------------050209070804050805060408" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 10:43:19 -0000 This is a multi-part message in MIME format. --------------050209070804050805060408 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I've updated the patch to use /var as localstatedir instead of /usr/local/var Without this patch, I can't connect to a Linux box with virt-manager because I need the ?socket=/var/run/libvirt/libvirt-sock parameter in libvirt uri --------------050209070804050805060408 Content-Type: text/plain; charset=UTF-8; name="libvirt_port_cleanup.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="libvirt_port_cleanup.diff" SW5kZXg6IE1ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIE1ha2VmaWxlCShyZXZpc2lvbiAz NTg3MjIpCisrKyBNYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMyw2ICszLDcgQEAKIAog UE9SVE5BTUU9CWxpYnZpcnQKIFBPUlRWRVJTSU9OPQkxLjIuNQorUE9SVFJFVklTSU9OPQkx CiBDQVRFR09SSUVTPQlkZXZlbAogTUFTVEVSX1NJVEVTPQlodHRwOi8vbGlidmlydC5vcmcv c291cmNlcy8gXAogCQlmdHA6Ly9saWJ2aXJ0Lm9yZy9saWJ2aXJ0LwpAQCAtMTcsNiArMTgs OCBAQAogCQlsaWJ4bWwyLnNvOiR7UE9SVFNESVJ9L3RleHRwcm9jL2xpYnhtbDIgXAogCQls aWJnbnV0bHMuc286JHtQT1JUU0RJUn0vc2VjdXJpdHkvZ251dGxzIFwKIAkJbGlic3NoMi5z bzoke1BPUlRTRElSfS9zZWN1cml0eS9saWJzc2gyCitCVUlMRF9ERVBFTkRTPQlkbnNtYXNx OiR7UE9SVFNESVJ9L2Rucy9kbnNtYXNxCitSVU5fREVQRU5EUz0JZG5zbWFzcToke1BPUlRT RElSfS9kbnMvZG5zbWFzcQogCiBPUFRJT05TX0RFRklORT0JRE9DUyBOTFMgUUVNVQogT1BU SU9OU19ERUZJTkVfYW1kNjQ9CUJIWVZFCkBAIC00MCwxNiArNDMsMTcgQEAKIFVTRV9SQ19T VUJSPQlsaWJ2aXJ0ZAogCiBHTlVfQ09ORklHVVJFPQl5ZXMKLUNPTkZJR1VSRV9BUkdTPQkt LXdpdGhvdXQtc2FzbCBcCitDT05GSUdVUkVfQVJHUz0JLS1sb2NhbHN0YXRlZGlyPS92YXIg XAorCQktLXdpdGhvdXQtc2FzbCBcCiAJCS0td2l0aG91dC1hdmFoaSBcCiAJCS0td2l0aG91 dC1wb2xraXQgXAogCQktLXdpdGhvdXQtaGFsIFwKIAkJLS13aXRob3V0LXVkZXYgXAogCQkt LXdpdGhvdXQtbmV0Y2YgXAotCQktLXdpdGhvdXQtbmV0d29yayBcCiAJCS0td2l0aG91dC1z eXNjdGwgXAogCQktLXdpdGhvdXQtbHhjIFwKLQkJLS13aXRoLXNzaDIKKwkJLS13aXRoLXNz aDIgXAorCQktLXdpdGhvdXQtdW1sCiAKICMgd29ya2Fyb3VuZCB0byBvdmVycmlkZSBwa2ct Y29uZmlnIGNhbGxzCiAjIHJhbmRvbV9kYXRhIGZpeCBjYW4gYmUgcmVtb3ZlZCB3aGVuIDgu eCBpcyBlb2wKQEAgLTc5LDYgKzgzLDEwIEBACiAKIC5pbmNsdWRlIDxic2QucG9ydC5vcHRp b25zLm1rPgogCisuaWYgJHtPU1ZFUlNJT059ID49IDEwMDAwMDAKK09QVElPTlNfREVGQVVM VF9hbWQ2ND0JQkhZVkUKKy5lbmRpZgorCiAjd29yayBhcm91bmQgYW4gb3B0aW9ucyBidWcg KHRoYW5rcyBtYXRAKQogLmlmICR7QVJDSH0gIT0gYW1kNjQKIFBMSVNUX1NVQis9CUJIWVZF PSJAY29tbWVudCAiCkBAIC0xMDMsMTUgKzExMSwxMCBAQAogCUAke1JFSU5QTEFDRV9DTUR9 IC1lICdzfHFlbXUuY29uZiQkfHFlbXUuY29uZi5zYW1wbGV8JyBcCiAJCSR7V1JLU1JDfS9z cmMvTWFrZWZpbGUuaW4KIAorCUAke1JFSU5QTEFDRV9DTUR9IC1lICdzfGNwICQkKERFU1RE SVIpfGNwIC1mICQkKERFU1RESVIpfCcgXAorCQkke1dSS1NSQ30vc3JjL01ha2VmaWxlLmlu CisKIHBvc3QtaW5zdGFsbDoKLQlAZm9yIGNvbmZpZ3VyYXRpb24gaW4gbGlidmlydCBsaWJ2 aXJ0ZCAke1FFTVVfQ09ORklHfTsgZG8gXAotCWlmIFsgISAtZSAiJHtTVEFHRURJUn0ke0VU Q0RJUn0vJCR7Y29uZmlndXJhdGlvbn0uY29uZiIgXTsgdGhlbiBcCi0JCQkke0VDSE9fTVNH fSAiCUluc3RhbGxpbmcgbG9jYWwgY29uZmlndXJhdGlvbiBmaWxlOiAke1NUQUdFRElSfSR7 RVRDRElSfS8kJHtjb25maWd1cmF0aW9ufS5jb25mIjsgXAotCQkJJHtDUH0gJHtTVEFHRURJ Un0ke0VUQ0RJUn0vJCR7Y29uZmlndXJhdGlvbn0uY29uZi5zYW1wbGUgJHtTVEFHRURJUn0k e0VUQ0RJUn0vJCR7Y29uZmlndXJhdGlvbn0uY29uZjsgXAotCQllbHNlIFwKLQkJCSR7RUNI T19NU0d9ICIJUHJlc2VydmluZyBsb2NhbCBjb25maWd1cmF0aW9uIGZpbGU6ICR7U1RBR0VE SVJ9JHtFVENESVJ9LyQke2NvbmZpZ3VyYXRpb259LmNvbmYiOyBcCi0JZmk7IFwKLQlkb25l CiAJQCR7TUtESVJ9ICR7U1RBR0VESVJ9JHtET0NTRElSfQogLmZvciBkb2MgaW4gQVVUSE9S UyBDaGFuZ2VMb2cgSU5TVEFMTCBORVdTIFJFQURNRSBUT0RPCiAJJHtJTlNUQUxMX0RBVEF9 ICR7V1JLU1JDfS8ke2RvY30gJHtTVEFHRURJUn0ke0RPQ1NESVJ9CkluZGV4OiBmaWxlcy9w YXRjaC1jb25maWd1cmUKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZmlsZXMvcGF0Y2gtY29uZmlndXJl CShyZXZpc2lvbiAwKQorKysgZmlsZXMvcGF0Y2gtY29uZmlndXJlCSh3b3JraW5nIGNvcHkp CkBAIC0wLDAgKzEsMTEgQEAKKy0tLSBjb25maWd1cmUub3JpZyAgICAgIDIwMTQtMDYtMDcg MTg6Mjc6NTcuODA5NjY5NTUzIC0wNzAwCisrKysgY29uZmlndXJlICAgMjAxNC0wNi0wNyAx ODo1OToxNi44OTE1NDE4MTIgLTA3MDAKK0BAIC02MjQ2Nyw3ICs2MjQ2Nyw3IEBACisgCisg CisgCistICAgICAgICBpZiB0ZXN0IC16ICIkQkhZVkUiIHx8IHRlc3QgLXogIiRCSFlWRUNU TCIgXAorKyAgICAgICAgaWYgdGVzdCAteiAiJEJIWVZFIiB8fCB0ZXN0IC16ICIkQkhZVkVD VEwiIHx8IFwKKyAgICAgICAgICAgICB0ZXN0IC16ICIkQkhZVkVMT0FEIiB8fCB0ZXN0ICIk d2l0aF9mcmVlYnNkIiA9ICJubyI7IHRoZW4KKyAgICAgICAgICAgICBpZiB0ZXN0ICIkd2l0 aF9iaHl2ZSIgPSAiY2hlY2siOyB0aGVuCisgICAgICAgICAgICAgICAgIHdpdGhfYmh5dmU9 Im5vIgoKUHJvcGVydHkgY2hhbmdlcyBvbjogZmlsZXMvcGF0Y2gtY29uZmlndXJlCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KQWRkZWQ6IGZic2Q6bm9rZXl3b3JkcwojIyAtMCwwICsxICMjCit5ZXMKXCBO byBuZXdsaW5lIGF0IGVuZCBvZiBwcm9wZXJ0eQpBZGRlZDogc3ZuOmVvbC1zdHlsZQojIyAt MCwwICsxICMjCituYXRpdmUKXCBObyBuZXdsaW5lIGF0IGVuZCBvZiBwcm9wZXJ0eQpBZGRl ZDogc3ZuOm1pbWUtdHlwZQojIyAtMCwwICsxICMjCit0ZXh0L3BsYWluClwgTm8gbmV3bGlu ZSBhdCBlbmQgb2YgcHJvcGVydHkKSW5kZXg6IGZpbGVzL3BhdGNoLWxpYnZpcnRfbm9fYnJp ZGdlLmRpZmYKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gZmlsZXMvcGF0Y2gtbGlidmlydF9ub19icmlk Z2UuZGlmZgkocmV2aXNpb24gMCkKKysrIGZpbGVzL3BhdGNoLWxpYnZpcnRfbm9fYnJpZGdl LmRpZmYJKHdvcmtpbmcgY29weSkKQEAgLTAsMCArMSwzOCBAQAorLS0tIHNyYy9iaHl2ZS9i aHl2ZV9jb21tYW5kLmMub3JpZwkyMDE0LTA1LTE5IDA1OjIwOjI0LjAwMDAwMDAwMCArMDQw MAorKysrIHNyYy9iaHl2ZS9iaHl2ZV9jb21tYW5kLmMJMjAxNC0wNi0xMiAxMTo1MDo0OC4w MDAwMDAwMDAgKzA0MDAKK0BAIC00NCw3ICs0NCw2IEBACisgICAgIHZpckRvbWFpbk5ldERl ZlB0ciBuZXQgPSBOVUxMOworICAgICBjaGFyICpicm5hbWUgPSBOVUxMOworICAgICBjaGFy ICpyZWFsaWZuYW1lID0gTlVMTDsKKy0gICAgaW50ICp0YXBmZCA9IE5VTEw7CisgICAgIGNo YXIgbWFjYWRkcltWSVJfTUFDX1NUUklOR19CVUZMRU5dOworIAorICAgICBpZiAoZGVmLT5u bmV0cyAhPSAxKSB7CitAQCAtODAsNyArNzksNyBAQAorIAorICAgICAgICAgaWYgKCFkcnlS dW4pCisgICAgICAgICAgICAgaWYgKHZpck5ldERldlRhcENyZWF0ZUluQnJpZGdlUG9ydChi cm5hbWUsICZuZXQtPmlmbmFtZSwgJm5ldC0+bWFjLAorLSAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgZGVmLT51dWlkLCB0YXBmZCwgMSwKKysgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRlZi0+dXVpZCwg TlVMTCwgMCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHZpckRvbWFpbk5ldEdldEFjdHVhbFZpcnRQb3J0UHJvZmlsZShuZXQpLAorICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmlyRG9tYWluTmV0 R2V0QWN0dWFsVmxhbihuZXQpLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgVklSX05FVERFVl9UQVBfQ1JFQVRFX0lGVVAgfCBWSVJfTkVUREVW X1RBUF9DUkVBVEVfUEVSU0lTVCkgPCAwKSB7CistLS0gc3JjL2JoeXZlL2JoeXZlX3Byb2Nl c3MuYy5vcmlnCTIwMTQtMDUtMTkgMDU6MjA6MjQuMDAwMDAwMDAwICswNDAwCisrKysgc3Jj L2JoeXZlL2JoeXZlX3Byb2Nlc3MuYwkyMDE0LTA2LTEyIDExOjUwOjEyLjAwMDAwMDAwMCAr MDQwMAorQEAgLTc3LDEwICs3NywxMiBAQAorICAgICAgICAgaW50IGFjdHVhbFR5cGUgPSB2 aXJEb21haW5OZXRHZXRBY3R1YWxUeXBlKG5ldCk7CisgCisgICAgICAgICBpZiAoYWN0dWFs VHlwZSA9PSBWSVJfRE9NQUlOX05FVF9UWVBFX0JSSURHRSkgeworLSAgICAgICAgICAgIGln bm9yZV92YWx1ZSh2aXJOZXREZXZCcmlkZ2VSZW1vdmVQb3J0KAorLSAgICAgICAgICAgICAg ICAgICAgICAgICAgICB2aXJEb21haW5OZXRHZXRBY3R1YWxCcmlkZ2VOYW1lKG5ldCksCist ICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5ldC0+aWZuYW1lKSk7CistICAgICAgICAg ICAgaWdub3JlX3ZhbHVlKHZpck5ldERldlRhcERlbGV0ZShuZXQtPmlmbmFtZSkpOworKyAg ICAgICAgICAgIGlmIChuZXQtPmlmbmFtZSkgeworKyAgICAgICAgICAgICAgICBpZ25vcmVf dmFsdWUodmlyTmV0RGV2QnJpZGdlUmVtb3ZlUG9ydCgKKysgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIHZpckRvbWFpbk5ldEdldEFjdHVhbEJyaWRnZU5hbWUobmV0KSwKKysg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5ldC0+aWZuYW1lKSk7CisrICAgICAg ICAgICAgICAgIGlnbm9yZV92YWx1ZSh2aXJOZXREZXZUYXBEZWxldGUobmV0LT5pZm5hbWUp KTsKKysgICAgICAgICAgICB9CisgICAgICAgICB9CisgICAgIH0KKyB9CgpQcm9wZXJ0eSBj aGFuZ2VzIG9uOiBmaWxlcy9wYXRjaC1saWJ2aXJ0X25vX2JyaWRnZS5kaWZmCl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KQWRkZWQ6IHN2bjptaW1lLXR5cGUKIyMgLTAsMCArMSAjIwordGV4dC9wbGFpbgpc IE5vIG5ld2xpbmUgYXQgZW5kIG9mIHByb3BlcnR5CkFkZGVkOiBmYnNkOm5va2V5d29yZHMK IyMgLTAsMCArMSAjIworeWVzClwgTm8gbmV3bGluZSBhdCBlbmQgb2YgcHJvcGVydHkKQWRk ZWQ6IHN2bjplb2wtc3R5bGUKIyMgLTAsMCArMSAjIworbmF0aXZlClwgTm8gbmV3bGluZSBh dCBlbmQgb2YgcHJvcGVydHkKSW5kZXg6IHBrZy1wbGlzdAo9PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBw a2ctcGxpc3QJKHJldmlzaW9uIDM1ODcyMikKKysrIHBrZy1wbGlzdAkod29ya2luZyBjb3B5 KQpAQCAtOCwxNiArOCwxMiBAQAogbWFuL21hbjEvdmlydC1ob3N0LXZhbGlkYXRlLjEuZ3oK IG1hbi9tYW44L2xpYnZpcnRkLjguZ3oKIG1hbi9tYW44L3ZpcnRsb2NrZC44Lmd6Ci1AdW5l eGVjIGlmIGNtcCAtcyAlRC8lJUVUQ0RJUiUlL2xpYnZpcnRkLmNvbmYuc2FtcGxlICVELyUl RVRDRElSJSUvbGlidmlydGQuY29uZjsgdGhlbiBybSAtZiAlRC8lJUVUQ0RJUiUlL2xpYnZp cnRkLmNvbmY7IGZpCi0lJUVUQ0RJUiUlL2xpYnZpcnRkLmNvbmYuc2FtcGxlCi1AZXhlYyBp ZiBbICEgLWYgJUQvJSVFVENESVIlJS9saWJ2aXJ0ZC5jb25mIF0gOyB0aGVuIGNwIC1wICVE LyUlRVRDRElSJSUvbGlidmlydGQuY29uZi5zYW1wbGUgJUQvJSVFVENESVIlJS9saWJ2aXJ0 ZC5jb25mOyBmaQotQHVuZXhlYyBpZiBjbXAgLXMgJUQvJSVFVENESVIlJS9saWJ2aXJ0LmNv bmYuc2FtcGxlICVELyUlRVRDRElSJSUvbGlidmlydC5jb25mOyB0aGVuIHJtIC1mICVELyUl RVRDRElSJSUvbGlidmlydC5jb25mOyBmaQotJSVFVENESVIlJS9saWJ2aXJ0LmNvbmYuc2Ft cGxlCi1AZXhlYyBpZiBbICEgLWYgJUQvJSVFVENESVIlJS9saWJ2aXJ0LmNvbmYgXSA7IHRo ZW4gY3AgLXAgJUQvJSVFVENESVIlJS9saWJ2aXJ0LmNvbmYuc2FtcGxlICVELyUlRVRDRElS JSUvbGlidmlydC5jb25mOyBmaQorQHNhbXBsZSAlJUVUQ0RJUiUlL2xpYnZpcnRkLmNvbmYu c2FtcGxlCitAc2FtcGxlICUlRVRDRElSJSUvbGlidmlydC5jb25mLnNhbXBsZQorJSVRRU1V JSVAc2FtcGxlICUlRVRDRElSJSUvcWVtdS5jb25mLnNhbXBsZQogJSVFVENESVIlJS92aXJ0 bG9ja2QuY29uZgotJSVRRU1VJSVAdW5leGVjIGlmIGNtcCAtcyAlRC8lJUVUQ0RJUiUlL3Fl bXUuY29uZi5zYW1wbGUgJUQvJSVFVENESVIlJS9xZW11LmNvbmY7IHRoZW4gcm0gLWYgJUQv JSVFVENESVIlJS9xZW11LmNvbmY7IGZpCi0lJVFFTVUlJSUlRVRDRElSJSUvcWVtdS5jb25m LnNhbXBsZQotJSVRRU1VJSVAZXhlYyBpZiBbICEgLWYgJUQvJSVFVENESVIlJS9xZW11LmNv bmYgXSA7IHRoZW4gY3AgLXAgJUQvJSVFVENESVIlJS9xZW11LmNvbmYuc2FtcGxlICVELyUl RVRDRElSJSUvcWVtdS5jb25mOyBmaQorJSVFVENESVIlJS9xZW11L25ldHdvcmtzL2RlZmF1 bHQueG1sCislJUVUQ0RJUiUlL3FlbXUvbmV0d29ya3MvYXV0b3N0YXJ0L2RlZmF1bHQueG1s CiAlJVFFTVUlJSUlRVRDRElSJSUvcWVtdS1sb2NrZC5jb25mCiBldGMvbG9ncm90YXRlLmQv bGlidmlydGQKIGV0Yy9sb2dyb3RhdGUuZC9saWJ2aXJ0ZC5seGMKQEAgLTQ3LDYgKzQzLDcg QEAKIHNoYXJlL2F1Z2Vhcy9sZW5zZXMvbGlidmlydGQuYXVnCiBzaGFyZS9hdWdlYXMvbGVu c2VzL3ZpcnRsb2NrZC5hdWcKICUlUUVNVSUlc2hhcmUvYXVnZWFzL2xlbnNlcy9saWJ2aXJ0 ZF9xZW11LmF1ZworJSVRRU1VJSVzaGFyZS9hdWdlYXMvbGVuc2VzL3Rlc3RzL3Rlc3RfbGli dmlydF9sb2NrZC5hdWcKIHNoYXJlL2F1Z2Vhcy9sZW5zZXMvdGVzdHMvdGVzdF9saWJ2aXJ0 ZC5hdWcKIHNoYXJlL2F1Z2Vhcy9sZW5zZXMvdGVzdHMvdGVzdF92aXJ0bG9ja2QuYXVnCiAl JVFFTVUlJXNoYXJlL2F1Z2Vhcy9sZW5zZXMvdGVzdHMvdGVzdF9saWJ2aXJ0ZF9xZW11LmF1 ZwpAQCAtMTg5LDYgKzE4Niw3IEBACiAlJURBVEFESVIlJS9zY2hlbWFzL25vZGVkZXYucm5n CiAlJURBVEFESVIlJS9zY2hlbWFzL253ZmlsdGVyLnJuZwogJSVEQVRBRElSJSUvc2NoZW1h cy9zZWNyZXQucm5nCislJURBVEFESVIlJS9zY2hlbWFzL3N0b3JhZ2Vjb21tb24ucm5nCiAl JURBVEFESVIlJS9zY2hlbWFzL3N0b3JhZ2Vwb29sLnJuZwogJSVEQVRBRElSJSUvc2NoZW1h cy9zdG9yYWdldm9sLnJuZwogJSVOTFMlJXNoYXJlL2xvY2FsZS9hZi9MQ19NRVNTQUdFUy9s aWJ2aXJ0Lm1vCkBAIC0yNzAsNiArMjY4LDggQEAKICUlUUVNVSUlbGliL2xpYnZpcnQvY29u bmVjdGlvbi1kcml2ZXIvbGlidmlydF9kcml2ZXJfcWVtdS5sYQogJSVCSFlWRSUlbGliL2xp YnZpcnQvY29ubmVjdGlvbi1kcml2ZXIvbGlidmlydF9kcml2ZXJfYmh5dmUubGEKICUlQkhZ VkUlJWxpYi9saWJ2aXJ0L2Nvbm5lY3Rpb24tZHJpdmVyL2xpYnZpcnRfZHJpdmVyX2JoeXZl LnNvCitsaWIvbGlidmlydC9jb25uZWN0aW9uLWRyaXZlci9saWJ2aXJ0X2RyaXZlcl9uZXR3 b3JrLnNvCitsaWIvbGlidmlydC9jb25uZWN0aW9uLWRyaXZlci9saWJ2aXJ0X2RyaXZlcl9u ZXR3b3JrLmxhCiBsaWIvbGlidmlydC9jb25uZWN0aW9uLWRyaXZlci9saWJ2aXJ0X2RyaXZl cl9zZWNyZXQuc28KIGxpYi9saWJ2aXJ0L2Nvbm5lY3Rpb24tZHJpdmVyL2xpYnZpcnRfZHJp dmVyX3NlY3JldC5sYQogbGliL2xpYnZpcnQvY29ubmVjdGlvbi1kcml2ZXIvbGlidmlydF9k cml2ZXJfc3RvcmFnZS5zbwpAQCAtMzI2LDIyICszMjYsMjkgQEAKICUlTkxTJSVAZGlycm10 cnkgc2hhcmUvbG9jYWxlL2JvCiAlJU5MUyUlQGRpcnJtdHJ5IHNoYXJlL2xvY2FsZS9hcy9M Q19NRVNTQUdFUwogJSVOTFMlJUBkaXJybXRyeSBzaGFyZS9sb2NhbGUvYXMKK0BkaXJybXRy eSAlJUVUQ0RJUiUlL3FlbXUvbmV0d29ya3MvYXV0b3N0YXJ0CitAZGlycm10cnkgJSVFVENE SVIlJS9xZW11L25ldHdvcmtzCitAZGlycm10cnkgJSVFVENESVIlJS9xZW11CiBAZGlycm0g JSVFVENESVIlJQotQGRpcnJtdHJ5IHZhci9ydW4vbGlidmlydC9sb2NrZAotQGRpcnJtdHJ5 IHZhci9ydW4vbGlidmlydAotQGRpcnJtdHJ5IHZhci9ydW4KLUBkaXJybXRyeSB2YXIvbG9n L2xpYnZpcnQvdW1sCi1AZGlycm10cnkgdmFyL2xvZy9saWJ2aXJ0L3FlbXUKLUBkaXJybXRy eSB2YXIvbG9nL2xpYnZpcnQvbHhjCi1AZGlycm10cnkgdmFyL2xvZy9saWJ2aXJ0Ci1AZGly cm10cnkgdmFyL2xvZwotQGRpcnJtdHJ5IHZhci9saWIvbGlidmlydC9sb2NrZC9maWxlcwot QGRpcnJtdHJ5IHZhci9saWIvbGlidmlydC9sb2NrZAotQGRpcnJtdHJ5IHZhci9saWIvbGli dmlydC9pbWFnZXMKLUBkaXJybXRyeSB2YXIvbGliL2xpYnZpcnQvYm9vdAotQGRpcnJtdHJ5 IHZhci9saWIvbGlidmlydC9maWxlc3lzdGVtcwotQGRpcnJtdHJ5IHZhci9saWIvbGlidmly dAotQGRpcnJtdHJ5IHZhci9saWIKLUBkaXJybXRyeSB2YXIvY2FjaGUvbGlidmlydAotQGRp cnJtdHJ5IHZhci9jYWNoZQotQGRpcnJtdHJ5IHZhcgorQGRpcnJtdHJ5IC92YXIvcnVuL2xp YnZpcnQvbmV0d29yaworJSVRRU1VJSVAZGlycm10cnkgL3Zhci9ydW4vbGlidmlydC9xZW11 CitAZGlycm10cnkgL3Zhci9ydW4vbGlidmlydC9sb2NrZAorQGRpcnJtdHJ5IC92YXIvcnVu L2xpYnZpcnQKK0BkaXJybXRyeSAvdmFyL2xvZy9saWJ2aXJ0L3VtbAorQGRpcnJtdHJ5IC92 YXIvbG9nL2xpYnZpcnQvcWVtdQorQGRpcnJtdHJ5IC92YXIvbG9nL2xpYnZpcnQvbHhjCitA ZGlycm10cnkgL3Zhci9sb2cvbGlidmlydAorQGRpcnJtdHJ5IC92YXIvbGliL2xpYnZpcnQv bG9ja2QvZmlsZXMKK0BkaXJybXRyeSAvdmFyL2xpYi9saWJ2aXJ0L2xvY2tkCitAZGlycm10 cnkgL3Zhci9saWIvbGlidmlydC9pbWFnZXMKK0BkaXJybXRyeSAvdmFyL2xpYi9saWJ2aXJ0 L2Jvb3QKK0BkaXJybXRyeSAvdmFyL2xpYi9saWJ2aXJ0L2Ruc21hc3EKK0BkaXJybXRyeSAv dmFyL2xpYi9saWJ2aXJ0L2ZpbGVzeXN0ZW1zCitAZGlycm10cnkgL3Zhci9saWIvbGlidmly dC9uZXR3b3JrCislJVFFTVUlJUBkaXJybXRyeSAvdmFyL2xpYi9saWJ2aXJ0L3FlbXUvY2hh bm5lbC90YXJnZXQKKyUlUUVNVSUlQGRpcnJtdHJ5IC92YXIvbGliL2xpYnZpcnQvcWVtdS9j aGFubmVsCislJVFFTVUlJUBkaXJybXRyeSAvdmFyL2xpYi9saWJ2aXJ0L3FlbXUKK0BkaXJy bXRyeSAvdmFyL2xpYi9saWJ2aXJ0CislJVFFTVUlJUBkaXJybXRyeSAvdmFyL2NhY2hlL2xp YnZpcnQvcWVtdQorQGRpcnJtdHJ5IC92YXIvY2FjaGUvbGlidmlydAorQGRpcnJtdHJ5IC92 YXIvY2FjaGUK --------------050209070804050805060408-- From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 19:05:08 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13273B75; Thu, 26 Jun 2014 19:05:08 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 131152CD0; Thu, 26 Jun 2014 19:05:06 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id w7so3293831lbi.35 for ; Thu, 26 Jun 2014 12:05:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=tXfKyRGZe4z/JIVa65QQEzSUkViPuwEG973qZ9HBuaI=; b=wgd+WZYr23n35+bzQl450Gndyr8MnkQ2a1sprrm90B/y2krWK0YNWBsOYkylw99hdD XSJ3mbIajrIltd3wrn2q86VWrKbZSb2YHZFzr53EsoQaEnkSV+xqSKk7z/+0h2H77wf9 RrT4lp7OpgYKDlq8gBCzQH0P2kfF1Z/1U+d6AhI0/fgyxTkcSEk51rOcZsGYz2RNz1qq v6zDuzs8g/vwc0Q2DpoHFIbi7TYMeQWoSHzHhTf/DTnX441RaXevS0ExBgeecUOSHNVv v2LNwPjWTJSx0MCQMqtSGoyQGiU7hjYA2FwjR8iehrb9cOrEeonei6X/t6b+tfVm4rrg N3vQ== MIME-Version: 1.0 X-Received: by 10.152.23.6 with SMTP id i6mr12889999laf.24.1403809504933; Thu, 26 Jun 2014 12:05:04 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Thu, 26 Jun 2014 12:05:04 -0700 (PDT) In-Reply-To: <20140625180116.GA1304@kloomba> References: <20140625180116.GA1304@kloomba> Date: Thu, 26 Jun 2014 12:05:04 -0700 X-Google-Sender-Auth: E17_fAuOIOB-VmbQng4lL9-yGRA Message-ID: Subject: Re: py-libvirt problems with bhyve From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-virtualization@freebsd.org" , freebsd-python@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 19:05:08 -0000 On Wed, Jun 25, 2014 at 11:01 AM, Roman Bogorodskiy wrote: > Craig Rodrigues wrote: > > > > > conn = libvirt.openReadOnly(None) > ^^^ > > I think here the URI should be 'bhyve:///system'. > Or you could tweak libvirt.conf and assign the bhyve url value to the > 'uri_default' parameter. > > http://libvirt.org/uri.html#URI_default > > Hi OK, I changed my steps a little bit. I did the following: (1) Set up libvirt, taking Roman's latest port, and configuring bhyve support: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html (2) Installed the py-libvirt port. I wrote this code: import libvirt import sys # Note we need to specify the URL here conn = libvirt.openReadOnly("bhyve:///system") if conn == None: print 'Failed to open connection to the hypervisor' sys.exit(1) try: print("All domains: ",conn.listAllDomains()) print("Defined domains: ", conn.listDefinedDomains()) #print dir(conn) dom0 = conn.lookupByName("bhyve") except: print 'Failed to find the main domain' sys.exit(1) print("Domain 0: id %d running" % (dom0.ID())) print(dom0.info()) I got this: ('All domains: ', []) ('Defined domains: ', ['bhyve']) Domain 0: id -1 running ('Domain info: ', [5, 2147483L, 0L, 1, 0L]) So that's an improvement. I started reading this tutorial on libvirt and KVM scripting: https://www.ibm.com/developerworks/linux/library/os-python-kvm-scripting1/ In that tutorial, they give an example where it is possible to configure the MAC and IP address of the VM in an XML config file. Is it possible to do that with libvirt and bhyve? Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 21:42:09 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7753EF8E for ; Thu, 26 Jun 2014 21:42:09 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 581492BCA for ; Thu, 26 Jun 2014 21:42:08 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 5C2521936DE for ; Thu, 26 Jun 2014 21:42:07 +0000 (UTC) Subject: jenkins bhyve vms crashing and burning after several days of use From: Sean Bruno Reply-To: sbruno@freebsd.org To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="us-ascii" Date: Thu, 26 Jun 2014 14:42:06 -0700 Message-ID: <1403818926.2417.6.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 21:42:09 -0000 so, we're seeing the bhyve vms running in the freebsd cluster for jenkins crashing and burning after a couple of days of use. vm exit[9] reason VMX rip 0x0000000029286336 inst_length 3 status 0 exit_reason 49 qualification 0x0000000000000000 inst_type 0 inst_error 0 It looks like we have an active core file on havoc.ysv if you have a moment to look at it: http://people.freebsd.org/~sbruno/bhyve.core FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 r267362: Wed Jun 11 14:56:34 UTC 2014 sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 21:46:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F33142D; Thu, 26 Jun 2014 21:46:36 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E1922BFC; Thu, 26 Jun 2014 21:46:36 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 8540C1936DE; Thu, 26 Jun 2014 21:46:35 +0000 (UTC) Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Sean Bruno Reply-To: sbruno@freebsd.org To: sbruno@freebsd.org In-Reply-To: <1403818926.2417.6.camel@bruno> References: <1403818926.2417.6.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Thu, 26 Jun 2014 14:46:34 -0700 Message-ID: <1403819194.2417.8.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 21:46:36 -0000 On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: > so, we're seeing the bhyve vms running in the freebsd cluster for > jenkins crashing and burning after a couple of days of use. > > vm exit[9] > reason VMX > rip 0x0000000029286336 > inst_length 3 > status 0 > exit_reason 49 > qualification 0x0000000000000000 > inst_type 0 > inst_error 0 > > > It looks like we have an active core file on havoc.ysv if you have a > moment to look at it: > > http://people.freebsd.org/~sbruno/bhyve.core > > FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 > r267362: Wed Jun 11 14:56:34 UTC 2014 > sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 > Also, from chaos.ysv http://people.freebsd.org/~sbruno/bhyve.core.chaos FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r267362: Wed Jun 11 15:50:24 UTC 2014 sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 22:00:12 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36E1FB06; Thu, 26 Jun 2014 22:00:12 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDED62D65; Thu, 26 Jun 2014 22:00:11 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id l6so3847956qcy.29 for ; Thu, 26 Jun 2014 15:00:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Tj0Kycr+Gb56CipOtWc4Te4dGW6qjdlXyoRlI8fj5EM=; b=yyDzQl8ZyiRxNFWBexfMX/jXkOxG+Le66UMuSNi64yh864nIRblyRFMDbw2RQH82kE 5/VTs30rkzza9HR6EpsPKJ38lzJg9NTQyD3kh94BBbvtv0EBSU03GKrQjLq4PCE94IDM /YVWllKLmFQg5WX+WDkgQxVfzBUOzVdVCSrfqbigDKgRsAxOdNLkPE4rkvgusb6s073e le7PFY9FB+TI7i9Wu+wFgPx5jOgBrG4AvS+dByuoX+CEV5XmExO8DfBzp7n/joFYG5ip HhW2vDm9cbwxpcevRt13OPkdTBONxmiDq89gyE1DV55wNzoMnzUJwaylRwWRYh7AkWYu swJQ== MIME-Version: 1.0 X-Received: by 10.140.19.21 with SMTP id 21mr25692728qgg.76.1403820011069; Thu, 26 Jun 2014 15:00:11 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Thu, 26 Jun 2014 15:00:11 -0700 (PDT) In-Reply-To: <1403819194.2417.8.camel@bruno> References: <1403818926.2417.6.camel@bruno> <1403819194.2417.8.camel@bruno> Date: Thu, 26 Jun 2014 15:00:11 -0700 Message-ID: Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Neel Natu To: Sean Bruno Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 22:00:12 -0000 Hi Sean, On Thu, Jun 26, 2014 at 2:46 PM, Sean Bruno wrote: > On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: >> so, we're seeing the bhyve vms running in the freebsd cluster for >> jenkins crashing and burning after a couple of days of use. >> >> vm exit[9] >> reason VMX >> rip 0x0000000029286336 >> inst_length 3 >> status 0 >> exit_reason 49 >> qualification 0x0000000000000000 >> inst_type 0 >> inst_error 0 >> >> >> It looks like we have an active core file on havoc.ysv if you have a >> moment to look at it: >> >> http://people.freebsd.org/~sbruno/bhyve.core >> >> FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 >> r267362: Wed Jun 11 14:56:34 UTC 2014 >> sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 >> > > Also, from chaos.ysv > > http://people.freebsd.org/~sbruno/bhyve.core.chaos > > FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 > r267362: Wed Jun 11 15:50:24 UTC 2014 > sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 > Can you tell us the processor and memory configuration on havoc and chaos? Also, could you execute the following commands on havoc: # bhyvectl --vm=vmname --cpu=9 --get-vmcs-guest-physical-address -- this will output the offending guest physical address that triggered the EPT misconfiguration # bhyvectl --vm=vmname --get-gpa-pmap= -- this will output the page table entries in the EPT that map to the offending GPA Hopefully that provides us with something to work with. best Neel > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 22:23:24 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8834DFDA; Thu, 26 Jun 2014 22:23:24 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51E012F69; Thu, 26 Jun 2014 22:23:24 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id F2A781936DE; Thu, 26 Jun 2014 22:23:22 +0000 (UTC) Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Sean Bruno Reply-To: sbruno@freebsd.org To: Neel Natu In-Reply-To: References: <1403818926.2417.6.camel@bruno> <1403819194.2417.8.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Thu, 26 Jun 2014 15:23:22 -0700 Message-ID: <1403821402.2417.12.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 22:23:24 -0000 On Thu, 2014-06-26 at 15:00 -0700, Neel Natu wrote: > Hi Sean, > > On Thu, Jun 26, 2014 at 2:46 PM, Sean Bruno wrote: > > On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: > >> so, we're seeing the bhyve vms running in the freebsd cluster for > >> jenkins crashing and burning after a couple of days of use. > >> > >> vm exit[9] > >> reason VMX > >> rip 0x0000000029286336 > >> inst_length 3 > >> status 0 > >> exit_reason 49 > >> qualification 0x0000000000000000 > >> inst_type 0 > >> inst_error 0 > >> > >> > >> It looks like we have an active core file on havoc.ysv if you have a > >> moment to look at it: > >> > >> http://people.freebsd.org/~sbruno/bhyve.core > >> > >> FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 > >> r267362: Wed Jun 11 14:56:34 UTC 2014 > >> sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 > >> > > > > Also, from chaos.ysv > > > > http://people.freebsd.org/~sbruno/bhyve.core.chaos > > > > FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 > > r267362: Wed Jun 11 15:50:24 UTC 2014 > > sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 > > > > Can you tell us the processor and memory configuration on havoc and chaos? > > Also, could you execute the following commands on havoc: > > # bhyvectl --vm=vmname --cpu=9 --get-vmcs-guest-physical-address > -- this will output the offending guest physical address that > triggered the EPT misconfiguration > > # bhyvectl --vm=vmname --get-gpa-pmap= > -- this will output the page table entries in the EPT that map to the > offending GPA > > Hopefully that provides us with something to work with. > > best > Neel > > > chaos: CPU: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz (2200.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d6 Family=0x6 Model=0x2d Stepping=6 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics avail memory = 66298322944 (63227 MB) havoc: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.14-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 Features=0xbfebfbff Features2=0x29ee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics avail memory = 16571621376 (15803 MB) There appear to be three vms running on havoc: root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 --get-vmcs-guest-physical-address gpa[9] 0x0000000000000000 root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 --get-vmcs-guest-physical-address gpa[9] 0x0000000000000000 root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 --get-vmcs-guest-physical-address gpa[9] 0x0000000000000000 root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 --get-gpa-pmap=0x0000000000000000 gpa 0: 0x300002c936e007 0x300002c9353007 0x300002c9352007 0 root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 --get-gpa-pmap=0x0000000000000000 gpa 0: 0x30000286cb0007 0x300003ad105007 0x3000019b1fd007 0 root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 --get-gpa-pmap=0x0000000000000000 gpa 0: 0x300002c9348007 0x300002c9339007 0 But there's no information available on chaos at the moment as there are no active vms running. sean From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 26 22:43:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B06CC456; Thu, 26 Jun 2014 22:43:15 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 626FA20FD; Thu, 26 Jun 2014 22:43:15 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id r5so3841667qcx.11 for ; Thu, 26 Jun 2014 15:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9geydb0V3Ca1W9MhVZ7detFE0Uo8dhHey8YmB7MAxAc=; b=HqQfudr8yv6+M0DcIEzkLS1mkaQg8oKUWL1RXJDbD13xi+j85gdqWnOWgu9drsPX+p BBGsXSRYMycOzc5d2cl7WFrOPCiJ+NBufdMKDvYn/jw1ZGfSTgm4h9Fw5isS+3sKAhlm fNWra6sURHwXb39JFxPeFsgmbXs9Mpd6M7OXst4UY0S1g426UxzEDg/Ax1EkiQFUj43u LgtAKri6WwRebbt6mi7TV3V11K3c+IbzWDRzZWNb8IdgaxcLeW0RqnY5dUlGZ95EWIg0 HxbcXW0OsMhAXn/8+J5U4HIyDh/XQkSnMzflSmmYIAxJ6Wc0xOU72x8/gTa1x8zVuNHy bP9g== MIME-Version: 1.0 X-Received: by 10.224.40.194 with SMTP id l2mr27800987qae.81.1403822594526; Thu, 26 Jun 2014 15:43:14 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Thu, 26 Jun 2014 15:43:14 -0700 (PDT) In-Reply-To: <1403821402.2417.12.camel@bruno> References: <1403818926.2417.6.camel@bruno> <1403819194.2417.8.camel@bruno> <1403821402.2417.12.camel@bruno> Date: Thu, 26 Jun 2014 15:43:14 -0700 Message-ID: Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Neel Natu To: Sean Bruno Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 22:43:15 -0000 Hi Sean, On Thu, Jun 26, 2014 at 3:23 PM, Sean Bruno wrote: > On Thu, 2014-06-26 at 15:00 -0700, Neel Natu wrote: >> Hi Sean, >> >> On Thu, Jun 26, 2014 at 2:46 PM, Sean Bruno wrote: >> > On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: >> >> so, we're seeing the bhyve vms running in the freebsd cluster for >> >> jenkins crashing and burning after a couple of days of use. >> >> >> >> vm exit[9] >> >> reason VMX >> >> rip 0x0000000029286336 >> >> inst_length 3 >> >> status 0 >> >> exit_reason 49 >> >> qualification 0x0000000000000000 >> >> inst_type 0 >> >> inst_error 0 >> >> >> >> >> >> It looks like we have an active core file on havoc.ysv if you have a >> >> moment to look at it: >> >> >> >> http://people.freebsd.org/~sbruno/bhyve.core >> >> >> >> FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 >> >> r267362: Wed Jun 11 14:56:34 UTC 2014 >> >> sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 >> >> >> > >> > Also, from chaos.ysv >> > >> > http://people.freebsd.org/~sbruno/bhyve.core.chaos >> > >> > FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 >> > r267362: Wed Jun 11 15:50:24 UTC 2014 >> > sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 >> > >> >> Can you tell us the processor and memory configuration on havoc and chaos? >> >> Also, could you execute the following commands on havoc: >> >> # bhyvectl --vm=vmname --cpu=9 --get-vmcs-guest-physical-address >> -- this will output the offending guest physical address that >> triggered the EPT misconfiguration >> >> # bhyvectl --vm=vmname --get-gpa-pmap= >> -- this will output the page table entries in the EPT that map to the >> offending GPA >> >> Hopefully that provides us with something to work with. >> >> best >> Neel >> >> > > > chaos: > CPU: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz (2200.05-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x206d6 Family=0x6 Model=0x2d Stepping=6 > Features=0xbfebfbff > Features2=0x1fbee3ff > AMD Features=0x2c100800 > AMD Features2=0x1 > TSC: P-state invariant, performance statistics > avail memory = 66298322944 (63227 MB) > > havoc: > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.14-MHz > K8-class CPU) > Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 > Features=0xbfebfbff > Features2=0x29ee3ff > AMD Features=0x2c100800 > AMD Features2=0x1 > TSC: P-state invariant, performance statistics > avail memory = 16571621376 (15803 MB) > Thanks, we'll see if there are relevant errata for these processors. > > There appear to be three vms running on havoc: > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 > --get-vmcs-guest-physical-address > gpa[9] 0x0000000000000000 > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 > --get-vmcs-guest-physical-address > gpa[9] 0x0000000000000000 > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 > --get-vmcs-guest-physical-address > gpa[9] 0x0000000000000000 > > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 > --get-gpa-pmap=0x0000000000000000 > gpa 0: 0x300002c936e007 0x300002c9353007 0x300002c9352007 0 > > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 > --get-gpa-pmap=0x0000000000000000 > gpa 0: 0x30000286cb0007 0x300003ad105007 0x3000019b1fd007 0 > > root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 > --get-gpa-pmap=0x0000000000000000 > gpa 0: 0x300002c9348007 0x300002c9339007 0 > > > But there's no information available on chaos at the moment as there are > no active vms running. > Sorry, I should explained a bit more. After a bhyve(8) exits because of the EPT misconfiguration error there are breadcrumbs left over in the VMCS as well as the nested page tables. We can use them to diagnose what happened. The bhyvectl commands above should be executed after the VM exits but before it is restarted again. Once it restarts, the breadcrumbs get written over and are of no use. The "--vm=" passed to the bhyvectl command should be of the virtual machine that crashed. The "--cpu=" passed to the bhyvectl command should be the vcpuid that detected the EPT misconfiguration. The reason I used '9' as an example above was because you saw this on the console: vm exit[9] reason VMX rip 0x0000000029286336 Hope that helps. best Neel > sean > From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 27 05:49:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02749B92; Fri, 27 Jun 2014 05:49:42 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D5122345; Fri, 27 Jun 2014 05:49:41 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id s18so2553051lam.6 for ; Thu, 26 Jun 2014 22:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=0ZjGhGC7Fh3HQz9Okgnp5yMScop/MqPoQHW6eDgCxF4=; b=acrKnAp96DoxIxqFgsyEfhI+DMYQE4gRqGtugEkTZtZEZ3IWaLAdYX+TLIYNbIVeoJ XbSTjZs1GHLj2hQN4toBirIIihaDfs5mZiS38LbnXv4ttdAUKVEMhpuWU+QHXBYjlj7A YaHKWL09c4+rDw5Vhvgnw8JnFDLJjJeUP1GKqfDxuyiIiNDypJLfQY12W0oJb9nsvPVo vqGm/wcsv5/sKpJubXKvSadI3ekAz5D/42TQniFKujPgQxLzNtQjLp9y0/vZFPLquHe9 lc0drXGttXniKP03eeQwEO3QGq8njMAbSnHsC1itYj7MYt7Vnoa6tGumfmEFg3cB3Okn Z4Aw== X-Received: by 10.152.44.135 with SMTP id e7mr14949506lam.42.1403848178543; Thu, 26 Jun 2014 22:49:38 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id n3sm3954662lan.3.2014.06.26.22.49.37 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Jun 2014 22:49:37 -0700 (PDT) Sender: Roman Bogorodskiy Date: Fri, 27 Jun 2014 09:49:32 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt, bhyve, and Jenkins configuration problems Message-ID: <20140627054931.GA15290@dev.san.ru> References: <20140621135159.GA47374@kloomba> <20140625180810.GB1304@kloomba> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 05:49:42 -0000 Craig Rodrigues wrote: > On Wed, Jun 25, 2014 at 11:08 AM, Roman Bogorodskiy > wrote: >=20 > > > Does that mean that I need to configure my VM, so that it comes up wi= th a > > > known IP, so that > > > it can be entered on this configuration page? > > > > Correct. > > > > Though, normally you don't need to perform any special configuration for > > the IP address as long as the networking configuration is correct. > > > > The VM just needs to be configured to obtain an IP address via DHCP. > > Later on it should be persistent in dnsmasq. > > >=20 > > An IP address could be obtained either from the dnsmasq leases file or > > from arp cache for example. > > >=20 > But how does the Jenkins libvirt plugin know the IP address for the VM, > unless you enter it in this field in the Jenkins UI? Sorry for confusion, yes, it's required to enter the IP address into Jenkins configuration dialog, I was just talking about libvirt / guest configuration side of things. Roman Bogorodskiy From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 27 21:23:13 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D77B0E8C; Fri, 27 Jun 2014 21:23:13 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8710A2EAE; Fri, 27 Jun 2014 21:23:13 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id f51so13494qge.36 for ; Fri, 27 Jun 2014 14:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P5iQ+Vz8bMkgYh0MypBPX4Ypt7kOQ8hgr9bDjtE8n4o=; b=ysXrdssZG3zChbRovd+rrf/06jyqmx6geXaQBrCG/ZP2OBil0hTRrsbbtlvSGytclx 0vi/hyWrFsA5+Oet8cLLAljYGPeCofbUtsM/EqvKHUtPU4goNwnRFanq/E9oBnAESZbI nODz8f9TK9BmeKSHeepZQVz+uIjbUq4QCC4xh5Bntk2+Qgz2kOHFY+vBO2cDt/FcDZXj 21yr+QRFSyvvWbgY+QElk83AOKIPqBQtONCU0xw1dVfq7bMpJfSH/wp2LeKWdR4vXAqv 5AZH7dcIFPgO1u5T+5SG1In8fv4/J2FCEuOR5uVEBddmMR0txMxUETHQZ5JSM4syRN/f x7Xg== MIME-Version: 1.0 X-Received: by 10.224.47.77 with SMTP id m13mr38657908qaf.69.1403904192675; Fri, 27 Jun 2014 14:23:12 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Fri, 27 Jun 2014 14:23:12 -0700 (PDT) In-Reply-To: References: <1403818926.2417.6.camel@bruno> <1403819194.2417.8.camel@bruno> <1403821402.2417.12.camel@bruno> Date: Fri, 27 Jun 2014 14:23:12 -0700 Message-ID: Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Neel Natu To: Sean Bruno Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 21:23:13 -0000 Hi, On Thu, Jun 26, 2014 at 3:43 PM, Neel Natu wrote: > Hi Sean, > > On Thu, Jun 26, 2014 at 3:23 PM, Sean Bruno wrote: >> On Thu, 2014-06-26 at 15:00 -0700, Neel Natu wrote: >>> Hi Sean, >>> >>> On Thu, Jun 26, 2014 at 2:46 PM, Sean Bruno wrote: >>> > On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: >>> >> so, we're seeing the bhyve vms running in the freebsd cluster for >>> >> jenkins crashing and burning after a couple of days of use. >>> >> >>> >> vm exit[9] >>> >> reason VMX >>> >> rip 0x0000000029286336 >>> >> inst_length 3 >>> >> status 0 >>> >> exit_reason 49 >>> >> qualification 0x0000000000000000 >>> >> inst_type 0 >>> >> inst_error 0 >>> >> >>> >> >>> >> It looks like we have an active core file on havoc.ysv if you have a >>> >> moment to look at it: >>> >> >>> >> http://people.freebsd.org/~sbruno/bhyve.core >>> >> >>> >> FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 >>> >> r267362: Wed Jun 11 14:56:34 UTC 2014 >>> >> sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 >>> >> >>> > >>> > Also, from chaos.ysv >>> > >>> > http://people.freebsd.org/~sbruno/bhyve.core.chaos >>> > >>> > FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 >>> > r267362: Wed Jun 11 15:50:24 UTC 2014 >>> > sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 >>> > >>> >>> Can you tell us the processor and memory configuration on havoc and chaos? >>> >>> Also, could you execute the following commands on havoc: >>> >>> # bhyvectl --vm=vmname --cpu=9 --get-vmcs-guest-physical-address >>> -- this will output the offending guest physical address that >>> triggered the EPT misconfiguration >>> >>> # bhyvectl --vm=vmname --get-gpa-pmap= >>> -- this will output the page table entries in the EPT that map to the >>> offending GPA >>> >>> Hopefully that provides us with something to work with. >>> >>> best >>> Neel >>> >>> > >> >> chaos: >> CPU: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz (2200.05-MHz K8-class CPU) >> Origin="GenuineIntel" Id=0x206d6 Family=0x6 Model=0x2d Stepping=6 >> Features=0xbfebfbff >> Features2=0x1fbee3ff >> AMD Features=0x2c100800 >> AMD Features2=0x1 >> TSC: P-state invariant, performance statistics >> avail memory = 66298322944 (63227 MB) >> >> havoc: >> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 >> CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.14-MHz >> K8-class CPU) >> Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 >> Features=0xbfebfbff >> Features2=0x29ee3ff >> AMD Features=0x2c100800 >> AMD Features2=0x1 >> TSC: P-state invariant, performance statistics >> avail memory = 16571621376 (15803 MB) >> > > Thanks, we'll see if there are relevant errata for these processors. > Actually these processors have entirely different microarchitectures (Nehalem and Sandybridge) so its unlikely that this is due to processor errata. >> >> There appear to be three vms running on havoc: >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 >> --get-vmcs-guest-physical-address >> gpa[9] 0x0000000000000000 >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 >> --get-vmcs-guest-physical-address >> gpa[9] 0x0000000000000000 >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 >> --get-vmcs-guest-physical-address >> gpa[9] 0x0000000000000000 >> >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 >> --get-gpa-pmap=0x0000000000000000 >> gpa 0: 0x300002c936e007 0x300002c9353007 0x300002c9352007 0 >> >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 >> --get-gpa-pmap=0x0000000000000000 >> gpa 0: 0x30000286cb0007 0x300003ad105007 0x3000019b1fd007 0 >> >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 >> --get-gpa-pmap=0x0000000000000000 >> gpa 0: 0x300002c9348007 0x300002c9339007 0 >> >> >> But there's no information available on chaos at the moment as there are >> no active vms running. >> > > Sorry, I should explained a bit more. > > After a bhyve(8) exits because of the EPT misconfiguration error there > are breadcrumbs left over in the VMCS as well as the nested page > tables. We can use them to diagnose what happened. > > The bhyvectl commands above should be executed after the VM exits but > before it is restarted again. Once it restarts, the breadcrumbs get > written over and are of no use. > > The "--vm=" passed to the bhyvectl command should be of the > virtual machine that crashed. > The "--cpu=" passed to the bhyvectl command should be the > vcpuid that detected the EPT misconfiguration. The reason I used '9' > as an example above was because you saw this on the console: > > vm exit[9] > reason VMX > rip 0x0000000029286336 > > Hope that helps. > I submitted a change in r267966 to dump this information to the console. It is also stashed in the process memory so we can inspect it in a coredump. Would it be possible to upgrade chaos and/or havoc to r267966 so we can make progress on debugging this issue? best Neel > best > Neel > >> sean >> From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 27 23:46:56 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F92D812; Fri, 27 Jun 2014 23:46:56 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE65A2A83; Fri, 27 Jun 2014 23:46:55 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 7B2BA1936DE; Fri, 27 Jun 2014 23:46:53 +0000 (UTC) Subject: Re: jenkins bhyve vms crashing and burning after several days of use From: Sean Bruno Reply-To: sbruno@freebsd.org To: Neel Natu In-Reply-To: References: <1403818926.2417.6.camel@bruno> <1403819194.2417.8.camel@bruno> <1403821402.2417.12.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Fri, 27 Jun 2014 16:46:51 -0700 Message-ID: <1403912811.5727.7.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 23:46:56 -0000 On Fri, 2014-06-27 at 14:23 -0700, Neel Natu wrote: > Hi, > > On Thu, Jun 26, 2014 at 3:43 PM, Neel Natu wrote: > > Hi Sean, > > > > On Thu, Jun 26, 2014 at 3:23 PM, Sean Bruno wrote: > >> On Thu, 2014-06-26 at 15:00 -0700, Neel Natu wrote: > >>> Hi Sean, > >>> > >>> On Thu, Jun 26, 2014 at 2:46 PM, Sean Bruno wrote: > >>> > On Thu, 2014-06-26 at 14:42 -0700, Sean Bruno wrote: > >>> >> so, we're seeing the bhyve vms running in the freebsd cluster for > >>> >> jenkins crashing and burning after a couple of days of use. > >>> >> > >>> >> vm exit[9] > >>> >> reason VMX > >>> >> rip 0x0000000029286336 > >>> >> inst_length 3 > >>> >> status 0 > >>> >> exit_reason 49 > >>> >> qualification 0x0000000000000000 > >>> >> inst_type 0 > >>> >> inst_error 0 > >>> >> > >>> >> > >>> >> It looks like we have an active core file on havoc.ysv if you have a > >>> >> moment to look at it: > >>> >> > >>> >> http://people.freebsd.org/~sbruno/bhyve.core > >>> >> > >>> >> FreeBSD havoc.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2 > >>> >> r267362: Wed Jun 11 14:56:34 UTC 2014 > >>> >> sbruno@havoc.freebsd.org:/usr/obj/usr/src/sys/HAVOC amd64 > >>> >> > >>> > > >>> > Also, from chaos.ysv > >>> > > >>> > http://people.freebsd.org/~sbruno/bhyve.core.chaos > >>> > > >>> > FreeBSD chaos.ysv.freebsd.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 > >>> > r267362: Wed Jun 11 15:50:24 UTC 2014 > >>> > sbruno@chaos.ysv.freebsd.org:/usr/obj/usr/src/sys/CHAOS amd64 > >>> > > >>> > >>> Can you tell us the processor and memory configuration on havoc and chaos? > >>> > >>> Also, could you execute the following commands on havoc: > >>> > >>> # bhyvectl --vm=vmname --cpu=9 --get-vmcs-guest-physical-address > >>> -- this will output the offending guest physical address that > >>> triggered the EPT misconfiguration > >>> > >>> # bhyvectl --vm=vmname --get-gpa-pmap= > >>> -- this will output the page table entries in the EPT that map to the > >>> offending GPA > >>> > >>> Hopefully that provides us with something to work with. > >>> > >>> best > >>> Neel > >>> > >>> > > >> > >> chaos: > >> CPU: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz (2200.05-MHz K8-class CPU) > >> Origin="GenuineIntel" Id=0x206d6 Family=0x6 Model=0x2d Stepping=6 > >> Features=0xbfebfbff > >> Features2=0x1fbee3ff > >> AMD Features=0x2c100800 > >> AMD Features2=0x1 > >> TSC: P-state invariant, performance statistics > >> avail memory = 66298322944 (63227 MB) > >> > >> havoc: > >> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > >> CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.14-MHz > >> K8-class CPU) > >> Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 > >> Features=0xbfebfbff > >> Features2=0x29ee3ff > >> AMD Features=0x2c100800 > >> AMD Features2=0x1 > >> TSC: P-state invariant, performance statistics > >> avail memory = 16571621376 (15803 MB) > >> > > > > Thanks, we'll see if there are relevant errata for these processors. > > > > Actually these processors have entirely different microarchitectures > (Nehalem and Sandybridge) so its unlikely that this is due to > processor errata. > > >> > >> There appear to be three vms running on havoc: > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 > >> --get-vmcs-guest-physical-address > >> gpa[9] 0x0000000000000000 > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 > >> --get-vmcs-guest-physical-address > >> gpa[9] 0x0000000000000000 > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 > >> --get-vmcs-guest-physical-address > >> gpa[9] 0x0000000000000000 > >> > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm1 --cpu=9 > >> --get-gpa-pmap=0x0000000000000000 > >> gpa 0: 0x300002c936e007 0x300002c9353007 0x300002c9352007 0 > >> > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm2 --cpu=9 > >> --get-gpa-pmap=0x0000000000000000 > >> gpa 0: 0x30000286cb0007 0x300003ad105007 0x3000019b1fd007 0 > >> > >> root@havoc.ysv:/home/sbruno # bhyvectl --vm=vm3 --cpu=9 > >> --get-gpa-pmap=0x0000000000000000 > >> gpa 0: 0x300002c9348007 0x300002c9339007 0 > >> > >> > >> But there's no information available on chaos at the moment as there are > >> no active vms running. > >> > > > > Sorry, I should explained a bit more. > > > > After a bhyve(8) exits because of the EPT misconfiguration error there > > are breadcrumbs left over in the VMCS as well as the nested page > > tables. We can use them to diagnose what happened. > > > > The bhyvectl commands above should be executed after the VM exits but > > before it is restarted again. Once it restarts, the breadcrumbs get > > written over and are of no use. > > > > The "--vm=" passed to the bhyvectl command should be of the > > virtual machine that crashed. > > The "--cpu=" passed to the bhyvectl command should be the > > vcpuid that detected the EPT misconfiguration. The reason I used '9' > > as an example above was because you saw this on the console: > > > > vm exit[9] > > reason VMX > > rip 0x0000000029286336 > > > > Hope that helps. > > > > I submitted a change in r267966 to dump this information to the > console. It is also stashed in the process memory so we can inspect it > in a coredump. > > Would it be possible to upgrade chaos and/or havoc to r267966 so we > can make progress on debugging this issue? > > best > Neel > > > best > > Neel > > > >> sean > >> Yeah, I'll see if I can get that done this weekend. Waiting for build breakages to be resolved. :-) sean From owner-freebsd-virtualization@FreeBSD.ORG Sun Jun 29 16:44:51 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 844131F5; Sun, 29 Jun 2014 16:44:51 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A02FE2757; Sun, 29 Jun 2014 16:44:50 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id mc6so4274606lab.27 for ; Sun, 29 Jun 2014 09:44:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=nfEjwxXbx97LaEpMZucxMGzcwxTcfZ93Ga6xAgJMfVo=; b=Br6fCKPneLo04k/gnnyuns+n9UIB4Mls/uigzK9Si9Lclp5d6tl4/QEOHtNNNdL4fJ D1IUC7f5CxtIRqQULPXNp+Bw58l4myAYrYjg9pGbHVrfh5Hr0WTkdSgQhrh8ifM9qOt8 XSdWVlpgsBXAN2SxjLAueCa545KsrqFwWuoZi6pXnOaSDKFCZr8+CxJ82YUgFRlwQonv /rH6yjS6aUFjY9N2GW/7NGHC/DeHgbwkuYLeQu44I6QjBgjYmMUvuL8HPAuv0RP9Mxa0 2kL6VWnZYYyW1A/ho94JGS2Qp332z/AL+ke9wqGkjDdXWaQYERr+e1Fy+R4IG+aOaLSS v2GA== X-Received: by 10.152.20.101 with SMTP id m5mr1650118lae.68.1404060288131; Sun, 29 Jun 2014 09:44:48 -0700 (PDT) Received: from kloomba ([217.65.223.117]) by mx.google.com with ESMTPSA id yr11sm20082607lbb.5.2014.06.29.09.44.46 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Jun 2014 09:44:47 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 29 Jun 2014 20:41:58 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: py-libvirt problems with bhyve Message-ID: <20140629164157.GA17814@kloomba> References: <20140625180116.GA1304@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+QahgC5+KEYLbs62" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-virtualization@freebsd.org" , freebsd-python@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jun 2014 16:44:51 -0000 --+QahgC5+KEYLbs62 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Wed, Jun 25, 2014 at 11:01 AM, Roman Bogorodskiy > wrote: >=20 > > Craig Rodrigues wrote: > > > > > > > > conn =3D libvirt.openReadOnly(None) > > ^^^ > > > > I think here the URI should be 'bhyve:///system'. >=20 >=20 > > Or you could tweak libvirt.conf and assign the bhyve url value to the > > 'uri_default' parameter. > > > > http://libvirt.org/uri.html#URI_default > > > > >=20 > Hi >=20 > OK, I changed my steps a little bit. > I did the following: >=20 > (1) Set up libvirt, taking Roman's latest port, and configuring bhyve > support: >=20 > http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html >=20 > (2) Installed the py-libvirt port. >=20 >=20 > I wrote this code: >=20 > import libvirt > import sys >=20 > # Note we need to specify the URL here > conn =3D libvirt.openReadOnly("bhyve:///system") > if conn =3D=3D None: > print 'Failed to open connection to the hypervisor' > sys.exit(1) >=20 > try: > print("All domains: ",conn.listAllDomains()) > print("Defined domains: ", conn.listDefinedDomains()) > #print dir(conn) > dom0 =3D conn.lookupByName("bhyve") > except: > print 'Failed to find the main domain' > sys.exit(1) >=20 > print("Domain 0: id %d running" % (dom0.ID())) > print(dom0.info()) >=20 >=20 >=20 > I got this: >=20 > ('All domains: ', []) > ('Defined domains: ', ['bhyve']) > Domain 0: id -1 running > ('Domain info: ', [5, 2147483L, 0L, 1, 0L]) >=20 >=20 > So that's an improvement. >=20 >=20 > I started reading this tutorial on libvirt and KVM scripting: >=20 > https://www.ibm.com/developerworks/linux/library/os-python-kvm-scripting1/ >=20 > In that tutorial, they give an example where it is possible > to configure the MAC and IP address of the VM in an XML config file. >=20 > Is it possible to do that with libvirt and bhyve? Generally, the dnsmasq related code is the same, so it should work. The script in this example seem to operate only on the network definition file, it misses the actual applying of the changes (or maybe I missed that in the article). I didn't try the script itself, but tried the suggested changes for the network definition, i.e. adding stuff like: do the section. I did 'net-destroy default', then 'net-edit default', made this change, stopped the dnsmasq and did 'net-start default' (I'm not sure if it's a bug or a feature that dnsmasq didn't stop, need to look closer into that) and rebooted a domain. It seemed to work, it got the configured IP address and the hostname. Roman Bogorodskiy --+QahgC5+KEYLbs62 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTsEHVAAoJEMltX/4IwiJqatkIAKncQVJGMDsaDsftyLaD5SzY S6Osw/egmVt45BxX9xnscLdbjSK7UxIV301O5j2eNL3FPmnLTscxBNWxND4LcAre NT4ocj1WXslfKful4MQu8x1d+ou6uIZkbZ7wUt6SmuSCh6WmGq/qf8kTpn5gbFJZ 1wb+7ZCXP4/AophbfYwS7A5dNwOWm7XQ9VDEC8VB2QBMDTpqOUjdYdboTRh3vdc6 BjNBHQaerYXxqh7v6QtOhDE50EB/Q8OEGz+D43SjtHioVhfruVGvcNEHSbm3YM+v 647ljViV3QARLhqrYLCd66Z77zPLKACFPjMEcYFYFsqBr7Mq0u4fCjn26YOJvwA= =Vamj -----END PGP SIGNATURE----- --+QahgC5+KEYLbs62-- From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 30 07:44:07 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C891C54 for ; Mon, 30 Jun 2014 07:44:07 +0000 (UTC) Received: from maild-eb.linkedin.com (maild-eb.linkedin.com [199.101.162.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.linkedin.com", Issuer "DigiCert Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5248027F2 for ; Mon, 30 Jun 2014 07:44:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linkedin.com; s=proddkim1024; t=1404114239; bh=nKOyQo3ZCXsdeNGX4G7QY8VEMkb+A+dujNXsrQz36vI=; h=From:Subject:MIME-Version:Content-Type:To:Date:X-LinkedIn-Class: X-LinkedIn-Template:X-LinkedIn-fbl; b=bi14rJ6Ws8bpbUdMPgnlPD7sv+hYp1isUpjTKMdr2dCTgQhlADjx2Rw6R0hDVCTF7 s4QsBBGPta7RfJbdJ2sums1annhSFdH7zbJL4dcT6nJ0g9mvEJEr5skqORAeiQn9XO VjSJn4aOo9Bzc1DfzmYMlU0Rji/diTESpgH71fVU= From: Luiz Gustavo Santos Costa via LinkedIn Message-ID: <1161101113.1129178.1404114239189.JavaMail.app@ela4-app2369.prod> Subject: =?UTF-8?Q?O_convite_de_Luiz_Gustavo_Santos_C?= =?UTF-8?Q?osta_est=C3=A1_aguardando_sua_resposta.?= MIME-Version: 1.0 To: Date: Mon, 30 Jun 2014 07:43:59 +0000 (UTC) X-LinkedIn-Class: INVITE-REMIND-GUEST X-LinkedIn-Template: inv_exp_19 X-LinkedIn-fbl: s-2gvghrg6hpxna4jpt32hy0akzh9mh0ypy3evw1yefowzp2r0hi26fepe X-LinkedIn-Id: -afxa6c-hx1hcwv1-3f Content-Type: text/plain;charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 07:44:07 -0000 Luiz Gustavo Santos Costa quer fazer parte da sua rede no LinkedIn. Como de= seja responder? Aceitar: https://www.linkedin.com/e/v2?e=3D-afxa6c-hx1hcwv1-3f&a=3DpreRegIn= vite&tracking=3Deml-comm_invg-b-accept-reminder&ek=3Dinv_exp_19&invitationI= D=3D5884646811586551808&sharedKey=3DoAVJOF1R Visualizar o perfil de Luiz Gustavo Santos Costa: http://www.linkedin.com/e= /v2?e=3D-afxa6c-hx1hcwv1-3f&t=3Dnpv&tracking=3Deml-comm_invg-b-accept-remin= der&ek=3Dinv_exp_19 Voc=C3=AA est=C3=A1 recebendo e-mails de lembretes sobre convites pendentes= . Cancele sua inscri=C3=A7=C3=A3o aqui: http://www.linkedin.com/e/v2?e=3D-a= fxa6c-hx1hcwv1-3f&t=3Dgoo&tracking=3Deml-comm_mebc-f-unsub&ek=3Dinv_exp_19&= gid=3D5884646811586551808&id=3D20060&email=3Dfreebsd-virtualization%40freeb= sd%2Eorg Saiba por que inclu=C3=ADmos isso neste link: http://www.linkedin.com/e/v2= ?e=3D-afxa6c-hx1hcwv1-3f&a=3DcustomerServiceUrl&ek=3Dinv_exp_19&articleId= =3D4788 ©2014 LinkedIn Ireland Limited, registrada na Irlanda como uma empresa= privada e ltda. Registro n=C3=BAmero 477441. Registrada em: 70 Sir John Ro= berson's Quay, Dublin 2. From owner-freebsd-virtualization@FreeBSD.ORG Tue Jul 1 00:30:24 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DAE1A31C; Tue, 1 Jul 2014 00:30:24 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39B912659; Tue, 1 Jul 2014 00:30:24 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id u10so6430786lbd.36 for ; Mon, 30 Jun 2014 17:30:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=tDBAOwkcR0dHi8JFhyzdJnlhViYkBm7UoeGcb8kGI7Q=; b=HqXsBaitTf7+ybPFBGAq1RsjO6lyKItjiLcI0cfP7QsMmTWpKo544cZtz5xZHu4We0 kLTjrIt8aXlhFpfi7SYCHprwB3b57VBgfa4Ic7ymH65huHmgKepRjME2URF8DZNXGVbB z9QZGFeZ6pniMWz1XXoVHdBa6cTbKnFva8PEqAHqmj/+io+vMp4/jbJcFgMIOu3tfqoA uB3IE7vWppG96Yz2CDuUFA8Zav1lqTl/EYwFDv6UxOwBMcdbYUzEcT2IThrvyS3n4eKO S56hYgsyXwLpTUGyyLFFp0qY7u4XLGghFmDMaPhW3ZFAAve0+h4LZqxWH1RIv6J125ZX mwGA== MIME-Version: 1.0 X-Received: by 10.112.30.99 with SMTP id r3mr17447795lbh.14.1404174622192; Mon, 30 Jun 2014 17:30:22 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Mon, 30 Jun 2014 17:30:22 -0700 (PDT) Date: Mon, 30 Jun 2014 17:30:22 -0700 X-Google-Sender-Auth: Q8rZUtFfrRGCZYBaZzUmKlyLUUM Message-ID: Subject: BAFUG July 10: libvirt virtualization API for controlling bhyve virtual machines From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-current Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 00:30:24 -0000 Hi, On July 10, 2014, in Mountain View, California, I will be giving a brief tech talk on: "libvirt virtualization API for controlling bhyve virtual machines" I have been testing the extensions that Roman Bogorodskiy has made to the lbivirt library for supporting bhyve. I will talk about my experiences with libvirt and bhyve. Here are more details about the talk plus directions to the location: http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/191061222/ Feel free to sign up on the meetup site and attend if you are in the area! -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Sat Jul 5 20:34:45 2014 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D350984A for ; Sat, 5 Jul 2014 20:34:45 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B89D02B6D for ; Sat, 5 Jul 2014 20:34:45 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s65KYjmA060477 for ; Sat, 5 Jul 2014 21:34:45 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 186146] vtscsi detects two disks when only one s configured Date: Sat, 05 Jul 2014 20:34:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: gavin@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jul 2014 20:34:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186146 Gavin Atkinson changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org Summary|Google Compute Engine, |vtscsi detects two disks | |when only one s configured -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 15:46:33 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15443958; Sun, 6 Jul 2014 15:46:33 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DB2482C38; Sun, 6 Jul 2014 15:46:32 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s66FkMMH083091 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Jul 2014 15:46:27 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s66FkMBN083090; Sun, 6 Jul 2014 15:46:22 GMT (envelope-from swills) Date: Sun, 6 Jul 2014 15:46:22 +0000 From: Steve Wills To: current@freebsd.org, virtualization@freebsd.org Subject: Re: tmpfs panic Message-ID: <20140706154621.GA81830@mouf.net> References: <20140706135333.GA80856@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline In-Reply-To: <20140706135333.GA80856@mouf.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sun, 06 Jul 2014 15:46:28 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 15:46:33 -0000 --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I should have noted this system is running in bhyve. Also I'm told this pan= ic may be related to the fact that the system is running in bhyve. Looking at it a little more closely: (kgdb) list *__mtx_lock_sleep+0xb1 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex.c:4= 31). 426 * owner stops running or the state of the lock cha= nges. 427 */ 428 v =3D m->mtx_lock; 429 if (v !=3D MTX_UNOWNED) { 430 owner =3D (struct thread *)(v & ~MTX_FLAGMA= SK); 431 if (TD_IS_RUNNING(owner)) { 432 if (LOCK_LOG_TEST(&m->lock_object, = 0)) 433 CTR3(KTR_LOCK, 434 "%s: spinning on %p hel= d by %p", 435 __func__, m, owner); (kgdb)=20 I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CONTEN= DED is spuriously left behind, and to ask how lock prefix is handled in bhyve. = Any of that make sense to anyone? Thanks, Steve On Sun, Jul 06, 2014 at 01:53:37PM +0000, Steve Wills wrote: > Hi, >=20 > Just experienced this tmpfs panic on r268160: >=20 > Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of mem= ory. >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 12; apic id =3D 0c > fault virtual address =3D 0x378 > fault code =3D supervisor read data, page not present > instruction pointer =3D 0x20:0xffffffff809638d1 > stack pointer =3D 0x28:0xfffffe07243800a0 > frame pointer =3D 0x28:0xfffffe0724380120 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 65339 (pkg-static) > [ thread pid 65339 tid 101641 ] > Stopped at __mtx_lock_sleep+0xb1: movl 0x378(%rax),%ecx > db> bt > Tracing pid 65339 tid 101641 td 0xfffff80286b2e490 > __mtx_lock_sleep() at __mtx_lock_sleep+0xb1/frame 0xfffffe0724380120 > free_unr() at free_unr+0x9d/frame 0xfffffe0724380160 > tmpfs_free_node() at tmpfs_free_node+0xf2/frame 0xfffffe07243801a0 > tmpfs_reclaim() at tmpfs_reclaim+0xdc/frame 0xfffffe07243801d0 > VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xa7/frame 0xfffffe0724380200 > vgonel() at vgonel+0x24c/frame 0xfffffe0724380280 > vrecycle() at vrecycle+0x84/frame 0xfffffe07243802c0 > tmpfs_inactive() at tmpfs_inactive+0x18/frame 0xfffffe07243802d0 > VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xa7/frame 0xfffffe0724380300 > vinactive() at vinactive+0x181/frame 0xfffffe0724380360 > vputx() at vputx+0x30d/frame 0xfffffe07243803d0 > vn_close() at vn_close+0x13e/frame 0xfffffe0724380450 > vn_closefile() at vn_closefile+0x48/frame 0xfffffe07243804d0 > _fdrop() at _fdrop+0x29/frame 0xfffffe07243804f0 > closef() at closef+0x2ae/frame 0xfffffe0724380580 > fdescfree() at fdescfree+0x64c/frame 0xfffffe0724380630 > exit1() at exit1+0x682/frame 0xfffffe07243806c0 > sigexit() at sigexit+0x929/frame 0xfffffe0724380980 > postsig() at postsig+0x3c4/frame 0xfffffe0724380a70 > ast() at ast+0x487/frame 0xfffffe0724380ab0 > doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffc6e0 > db>=20 >=20 > Any further debugging I can do? >=20 > Thanks, > Steve --YiEDa0DAkWCtVeE4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTuW9NAAoJEPXPYrMgexuhjY4IAJQGh3fo3FBJKkK0Kfh21W2W Qlr4NCa04RB7PY7ra9dqvIvIztMhzgrD+qCcPQYdj0Jt+BqdAJo9yjTDkJAPh++d dbJXDYD2wr8TV/bPo+iU3tMy5qRF2SnJjiqJzf6MBzQ/IwLzSigCF+5yVL85Uest U0sX8sEOOu3mYvp7lOe9PeyvzpB0BBvckVzLQyvynsFwFgvzcfC9FNQYjo8i3KFy +/5PkANY199FyVxp/+F2/yA5G9c+OiF1F+LUeaCgr82ye9dMwrDV9gU/ySLkDixd +SWbscnBnNBLD26ne4EbaVPxw7TdMaQCR0uxOkloMIEXFJIkpFmTxeKi/z0w5zM= =8lEj -----END PGP SIGNATURE----- --YiEDa0DAkWCtVeE4-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 16:28:08 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA031C02; Sun, 6 Jul 2014 16:28:08 +0000 (UTC) Received: from mail-ob0-x230.google.com (mail-ob0-x230.google.com [IPv6:2607:f8b0:4003:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E5362F43; Sun, 6 Jul 2014 16:28:08 +0000 (UTC) Received: by mail-ob0-f176.google.com with SMTP id wm4so3507674obc.7 for ; Sun, 06 Jul 2014 09:28:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K+p0NqvRof+s7RlnKnG60FYptpbpmCPqYXAd+kWYKpI=; b=UOUbY1ZJx1hTVCRll2RwLbl0u/n5Ll/uiOpD6553kXO6LwwjNPgWmgqRaFJVs1KZAK LkDGMDkzfkmMfaREBYwCqhdvwjXLxY/RBqMZ1ss2sI4pY5StelS+8Ehl/iJedjpWoPKY wZKiKxUfPMQLYQWAOamItTYenSYLT1azIDG/AixiPWRxDXEy62wGR/yNmBDhO/bwkIAc poT0P7rEaHXUZxACMSt5WxjqCDpBpRV6mnsbABkn48+MQoVQWarJVkgIfFMI1O4iryMj inVPyEQfOehgZ02isNvulc4dOD90Qa49G/92+HPaC8fKbkyzqb2hxdDQk2BsXuumw6Rh fK5A== MIME-Version: 1.0 X-Received: by 10.60.155.231 with SMTP id vz7mr26097732oeb.56.1404664087919; Sun, 06 Jul 2014 09:28:07 -0700 (PDT) Received: by 10.76.151.227 with HTTP; Sun, 6 Jul 2014 09:28:07 -0700 (PDT) In-Reply-To: <20140706154621.GA81830@mouf.net> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> Date: Sun, 6 Jul 2014 12:28:07 -0400 Message-ID: Subject: Re: tmpfs panic From: Ryan Stone To: Steve Wills Content-Type: text/plain; charset=UTF-8 Cc: virtualization@freebsd.org, FreeBSD Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 16:28:09 -0000 On Sun, Jul 6, 2014 at 11:46 AM, Steve Wills wrote: > I should have noted this system is running in bhyve. Also I'm told this panic > may be related to the fact that the system is running in bhyve. > > Looking at it a little more closely: > > (kgdb) list *__mtx_lock_sleep+0xb1 > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex.c:431). > 426 * owner stops running or the state of the lock changes. > 427 */ > 428 v = m->mtx_lock; > 429 if (v != MTX_UNOWNED) { > 430 owner = (struct thread *)(v & ~MTX_FLAGMASK); > 431 if (TD_IS_RUNNING(owner)) { > 432 if (LOCK_LOG_TEST(&m->lock_object, 0)) > 433 CTR3(KTR_LOCK, > 434 "%s: spinning on %p held by %p", > 435 __func__, m, owner); > (kgdb) > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CONTENDED > is spuriously left behind, and to ask how lock prefix is handled in bhyve. Any > of that make sense to anyone? The mutex has both MTX_CONTESTED and MTX_UNOWNED set on it? That is a special sentinel value that is set on a mutex when it is destroyed (see MTX_DESTROYED in sys/mutex.h). If that is the case it looks like you've stumbled upon some kind of use-after-free in tmpfs. I doubt that bhyve is responsible (other than perhaps changing the timing around making the panic more likely to happen). From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 17:25:24 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8E6EDF3; Sun, 6 Jul 2014 17:25:23 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A4FF423AD; Sun, 6 Jul 2014 17:25:23 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s66HPCoU084581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Jul 2014 17:25:17 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s66HPCu8084580; Sun, 6 Jul 2014 17:25:12 GMT (envelope-from swills) Date: Sun, 6 Jul 2014 17:25:12 +0000 From: Steve Wills To: Ryan Stone Subject: Re: tmpfs panic Message-ID: <20140706172511.GA84461@mouf.net> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OXfL5xGRrasGEqWY" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sun, 06 Jul 2014 17:25:18 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: virtualization@freebsd.org, FreeBSD Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 17:25:24 -0000 --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 06, 2014 at 12:28:07PM -0400, Ryan Stone wrote: > On Sun, Jul 6, 2014 at 11:46 AM, Steve Wills wrote: > > I should have noted this system is running in bhyve. Also I'm told this= panic > > may be related to the fact that the system is running in bhyve. > > > > Looking at it a little more closely: > > > > (kgdb) list *__mtx_lock_sleep+0xb1 > > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex= =2Ec:431). > > 426 * owner stops running or the state of the lock= changes. > > 427 */ > > 428 v =3D m->mtx_lock; > > 429 if (v !=3D MTX_UNOWNED) { > > 430 owner =3D (struct thread *)(v & ~MTX_FL= AGMASK); > > 431 if (TD_IS_RUNNING(owner)) { > > 432 if (LOCK_LOG_TEST(&m->lock_obje= ct, 0)) > > 433 CTR3(KTR_LOCK, > > 434 "%s: spinning on %p= held by %p", > > 435 __func__, m, owner); > > (kgdb) > > > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CO= NTENDED > > is spuriously left behind, and to ask how lock prefix is handled in bhy= ve. Any > > of that make sense to anyone? >=20 > The mutex has both MTX_CONTESTED and MTX_UNOWNED set on it? That is a > special sentinel value that is set on a mutex when it is destroyed > (see MTX_DESTROYED in sys/mutex.h). If that is the case it looks like > you've stumbled upon some kind of use-after-free in tmpfs. I doubt > that bhyve is responsible (other than perhaps changing the timing > around making the panic more likely to happen). Given the first thing seen was: Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of memor= y. this sounds reasonable to me. What can I do to help find and elliminate the source of the error? Steve --OXfL5xGRrasGEqWY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTuYZ3AAoJEPXPYrMgexuh06IIAJXfbe7rm1wQYoEz0ZpJU3jL QJvtyCllWlP/0w2zuGl7HSr0iGIdLqURKUmY7deA0tt+F5gP6IoSsXm5esAOEnFU 5ZTyOoHlkbSC4rMnjyRN/2isfdjmRruOG/NR37wvIXVPtH/8eJPK02utaBHJg3/6 8/PcGDxi3GOUaf0Kf3TURwSmLnG1ddhODBcCjMaC1GW5+OYSxsFN0fj5gAvkqzXY 6yFBZhRrvD9slOVkV1s3SaMDpjhLA6kd6Cipb2DT0HtgGTE5UF0RRc9OJUiTiKkb GSQEYfOymUdPUETJol1L7aZEk+lczflJAbC7Sn/HsirZgnL3Il72DQprZL6Jqsc= =+qu/ -----END PGP SIGNATURE----- --OXfL5xGRrasGEqWY-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 18:12:33 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEA8523B; Sun, 6 Jul 2014 18:12:33 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EE3E278E; Sun, 6 Jul 2014 18:12:33 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s66ICQ2p044478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 6 Jul 2014 21:12:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s66ICQ2p044478 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s66ICQXm044477; Sun, 6 Jul 2014 21:12:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 6 Jul 2014 21:12:26 +0300 From: Konstantin Belousov To: Steve Wills Subject: Re: tmpfs panic Message-ID: <20140706181226.GE93733@kib.kiev.ua> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> <20140706172511.GA84461@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="l118U0+vX1D/6gtA" Content-Disposition: inline In-Reply-To: <20140706172511.GA84461@mouf.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: virtualization@freebsd.org, Ryan Stone , FreeBSD Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 18:12:33 -0000 --l118U0+vX1D/6gtA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 06, 2014 at 05:25:12PM +0000, Steve Wills wrote: > On Sun, Jul 06, 2014 at 12:28:07PM -0400, Ryan Stone wrote: > > On Sun, Jul 6, 2014 at 11:46 AM, Steve Wills wrote: > > > I should have noted this system is running in bhyve. Also I'm told th= is panic > > > may be related to the fact that the system is running in bhyve. > > > > > > Looking at it a little more closely: > > > > > > (kgdb) list *__mtx_lock_sleep+0xb1 > > > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mut= ex.c:431). > > > 426 * owner stops running or the state of the lo= ck changes. > > > 427 */ > > > 428 v =3D m->mtx_lock; > > > 429 if (v !=3D MTX_UNOWNED) { > > > 430 owner =3D (struct thread *)(v & ~MTX_= FLAGMASK); > > > 431 if (TD_IS_RUNNING(owner)) { > > > 432 if (LOCK_LOG_TEST(&m->lock_ob= ject, 0)) > > > 433 CTR3(KTR_LOCK, > > > 434 "%s: spinning on = %p held by %p", > > > 435 __func__, m, owne= r); > > > (kgdb) > > > > > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_= CONTENDED > > > is spuriously left behind, and to ask how lock prefix is handled in b= hyve. Any > > > of that make sense to anyone? > >=20 > > The mutex has both MTX_CONTESTED and MTX_UNOWNED set on it? That is a > > special sentinel value that is set on a mutex when it is destroyed > > (see MTX_DESTROYED in sys/mutex.h). If that is the case it looks like > > you've stumbled upon some kind of use-after-free in tmpfs. I doubt > > that bhyve is responsible (other than perhaps changing the timing > > around making the panic more likely to happen). >=20 > Given the first thing seen was: >=20 > Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of mem= ory. >=20 > this sounds reasonable to me. >=20 > What can I do to help find and elliminate the source of the error? The most worrying fact there is that the mutex which is creating trouble cannot be anything other but allnode_lock, from the backtrace. For this mutex to be destroyed, the unmount of the corresponding mount point must run to completion. In particular, it must get past the vflush(9) call in tmpfs_unmount(). This call reclaims all vnodes belonging to the unmounted mount point. New vnodes cannot be instantiated meantime, since insmntque(9) is blocked by the MNTK_UNMOUNT flag. That said, the backtrace indicates that we have live vnode, which is reclaimed, and also we have the mutex which is in the destroyed (?) state. My basic claim is that the two events cannot co-exist, at least, this code path was heavily exercised and most issues were fixed during several years. I cannot exclude the possibility of tmpfs/VFS screwing things up, but given the above reasoning, and the fact that this is the first appearance of the MTX_DESTROED problem for the tmpfs unmounting code, which was not changed for long time, I would at least ask some things about bhyve. I.e., I would rather first look at the locked prefix emulation then at the tmpfs. --l118U0+vX1D/6gtA Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTuZGJAAoJEJDCuSvBvK1B1QUP/AlpvZasnrIqIIq2SK8EuVFs 9G4jIYeaYI8bDSFrzQe/RbZETI89YLRVTSOBvO2W4lazu8em4oLBFYw7U8Kdw3bn ITX14XRvCvRAb27Dg2PKo2tr2vN7sdlIZUdl4prBcs8ik9zQi1+kgiT1BF/snTAD CqRsLoVmKlocfEfZSINEA7Emm+9OM5rsF7ktCiPIMVl/NeoznvfVz7j+ilz5RAer Qeb3TWfBorbzZXO5nXFYxBaw6lgP/ZHqfdj0bGyaUQhhrNz/V02cmi1gITtZEKmR QBdbdATcV/Y6hdAZkEUwRrHAw7/Zl/JF+0Hk4mjru7Ri8cV1Ki4K4bZ4iEWgCIqn aDLIFjX1iTixmA03wDbX30Vo9gsX8m5qq4wYHHDLbttsO2LtclWiwpj1rqS7bjKS Un6I2KUS+kT5vJ/YPNgg1a0P340CbLQy4rz6akl+T+VS390VrDwOH+IiTMD5hoWd Dy+BRZn0gVxToHls89xu+Wc6u1KPK+0Iwngr+JktiCYlJggsEYjs+RsC0fRX3eC1 9mVYvVBo4Mjg8hFWP/U1xE+stJvYfzplj/JlUACtoV7gK90gHBllTL7Gqu1DoZjY g2ozS3yvosdeiVB8q9hXCtY03yZt+ddNFjd67oQ2k8/3O41zhoHPeqFWTMepL/id GlPcsrk94NTHNiVAxNqt =9jD2 -----END PGP SIGNATURE----- --l118U0+vX1D/6gtA-- From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 20:49:06 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0AF4B2B2; Sun, 6 Jul 2014 20:49:06 +0000 (UTC) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1AD92414; Sun, 6 Jul 2014 20:49:05 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id c9so3045081qcz.9 for ; Sun, 06 Jul 2014 13:49:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PDrP+622UD29NYEpFznnuKNY/1ELnRV9F65xLlxBoE0=; b=uHAg8S9xXcvIblgQiFN0/FeCI4L+uI7picazBsN8KEc4JGXJp70fEaLMga5jf1j9pu AaBzUDtsjgCAhP43NUxNs8YPxTQXL0fkKvok+pLDwNuNNVmsM7JyUg9Fl717QJIscK1B 7A+iNkq/+1vD96byLvNExjLk5QbI0GK6h1mOuX3OVHbvsOrxkskqSO1qyUHgw5NJ3BBu NJPwu3G3XVHDNVlvZnWLWgG3iR9NN2ya7eS72WAi/QDfPcinn3+y3+qMCw/OoK4IyKiu tOpAANnddflWYNKP2pJrrfmgO36hnBooZonXb7aO0yKbAOk5wBCAp4aY5USFWifZq4dn 9zCg== MIME-Version: 1.0 X-Received: by 10.224.123.71 with SMTP id o7mr40761859qar.38.1404679744739; Sun, 06 Jul 2014 13:49:04 -0700 (PDT) Received: by 10.140.48.37 with HTTP; Sun, 6 Jul 2014 13:49:04 -0700 (PDT) In-Reply-To: <20140706154621.GA81830@mouf.net> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> Date: Sun, 6 Jul 2014 13:49:04 -0700 Message-ID: Subject: Re: tmpfs panic From: Neel Natu To: Steve Wills Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" , "current@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 20:49:06 -0000 Hi Steve, On Sun, Jul 6, 2014 at 8:46 AM, Steve Wills wrote: > I should have noted this system is running in bhyve. Also I'm told this panic > may be related to the fact that the system is running in bhyve. > > Looking at it a little more closely: > > (kgdb) list *__mtx_lock_sleep+0xb1 > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex.c:431). > 426 * owner stops running or the state of the lock changes. > 427 */ > 428 v = m->mtx_lock; > 429 if (v != MTX_UNOWNED) { > 430 owner = (struct thread *)(v & ~MTX_FLAGMASK); > 431 if (TD_IS_RUNNING(owner)) { > 432 if (LOCK_LOG_TEST(&m->lock_object, 0)) > 433 CTR3(KTR_LOCK, > 434 "%s: spinning on %p held by %p", > 435 __func__, m, owner); > (kgdb) > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CONTENDED > is spuriously left behind, and to ask how lock prefix is handled in bhyve. Any > of that make sense to anyone? > Regarding the lock prefix: since bhyve only supports hardware that has nested paging, the hypervisor doesn't get in the way of instructions that access memory. This includes instructions with lock prefixes or any other prefixes for that matter. If there is a VM exit due to a nested page fault then the faulting instruction is restarted after resolving the fault. Having said that, there are more plausible explanations that might implicate bhyve: incorrect translations in the nested page tables, stale translations in the TLB etc. Do you have a core file for the panic? It would be very useful to debug this further. > Thanks, > Steve > > On Sun, Jul 06, 2014 at 01:53:37PM +0000, Steve Wills wrote: >> Hi, >> >> Just experienced this tmpfs panic on r268160: >> >> Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of memory. >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 12; apic id = 0c >> fault virtual address = 0x378 >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff809638d1 >> stack pointer = 0x28:0xfffffe07243800a0 >> frame pointer = 0x28:0xfffffe0724380120 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 65339 (pkg-static) >> [ thread pid 65339 tid 101641 ] >> Stopped at __mtx_lock_sleep+0xb1: movl 0x378(%rax),%ecx >> db> bt >> Tracing pid 65339 tid 101641 td 0xfffff80286b2e490 >> __mtx_lock_sleep() at __mtx_lock_sleep+0xb1/frame 0xfffffe0724380120 >> free_unr() at free_unr+0x9d/frame 0xfffffe0724380160 >> tmpfs_free_node() at tmpfs_free_node+0xf2/frame 0xfffffe07243801a0 >> tmpfs_reclaim() at tmpfs_reclaim+0xdc/frame 0xfffffe07243801d0 >> VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xa7/frame 0xfffffe0724380200 >> vgonel() at vgonel+0x24c/frame 0xfffffe0724380280 >> vrecycle() at vrecycle+0x84/frame 0xfffffe07243802c0 >> tmpfs_inactive() at tmpfs_inactive+0x18/frame 0xfffffe07243802d0 >> VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xa7/frame 0xfffffe0724380300 >> vinactive() at vinactive+0x181/frame 0xfffffe0724380360 >> vputx() at vputx+0x30d/frame 0xfffffe07243803d0 >> vn_close() at vn_close+0x13e/frame 0xfffffe0724380450 >> vn_closefile() at vn_closefile+0x48/frame 0xfffffe07243804d0 >> _fdrop() at _fdrop+0x29/frame 0xfffffe07243804f0 >> closef() at closef+0x2ae/frame 0xfffffe0724380580 >> fdescfree() at fdescfree+0x64c/frame 0xfffffe0724380630 >> exit1() at exit1+0x682/frame 0xfffffe07243806c0 >> sigexit() at sigexit+0x929/frame 0xfffffe0724380980 >> postsig() at postsig+0x3c4/frame 0xfffffe0724380a70 >> ast() at ast+0x487/frame 0xfffffe0724380ab0 >> doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffc6e0 >> db> >> >> Any further debugging I can do? >> >> Thanks, >> Steve > > From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 21:07:57 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A2CD8E5; Sun, 6 Jul 2014 21:07:57 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A0952587; Sun, 6 Jul 2014 21:07:57 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s66L7lI5087928 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Jul 2014 21:07:52 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s66L7lQn087927; Sun, 6 Jul 2014 21:07:47 GMT (envelope-from swills) Date: Sun, 6 Jul 2014 21:07:47 +0000 From: Steve Wills To: Neel Natu Subject: Re: tmpfs panic Message-ID: <20140706210746.GA85257@mouf.net> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2fHTh5uZTiUOsy+g" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sun, 06 Jul 2014 21:07:52 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-virtualization@freebsd.org" , "current@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 21:07:57 -0000 --2fHTh5uZTiUOsy+g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 06, 2014 at 01:49:04PM -0700, Neel Natu wrote: > Hi Steve, >=20 > On Sun, Jul 6, 2014 at 8:46 AM, Steve Wills wrote: > > I should have noted this system is running in bhyve. Also I'm told this= panic > > may be related to the fact that the system is running in bhyve. > > > > Looking at it a little more closely: > > > > (kgdb) list *__mtx_lock_sleep+0xb1 > > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex= =2Ec:431). > > 426 * owner stops running or the state of the lock= changes. > > 427 */ > > 428 v =3D m->mtx_lock; > > 429 if (v !=3D MTX_UNOWNED) { > > 430 owner =3D (struct thread *)(v & ~MTX_FL= AGMASK); > > 431 if (TD_IS_RUNNING(owner)) { > > 432 if (LOCK_LOG_TEST(&m->lock_obje= ct, 0)) > > 433 CTR3(KTR_LOCK, > > 434 "%s: spinning on %p= held by %p", > > 435 __func__, m, owner); > > (kgdb) > > > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CO= NTENDED > > is spuriously left behind, and to ask how lock prefix is handled in bhy= ve. Any > > of that make sense to anyone? > > >=20 > Regarding the lock prefix: since bhyve only supports hardware that has > nested paging, the hypervisor doesn't get in the way of instructions > that access memory. This includes instructions with lock prefixes or > any other prefixes for that matter. If there is a VM exit due to a > nested page fault then the faulting instruction is restarted after > resolving the fault. >=20 > Having said that, there are more plausible explanations that might > implicate bhyve: incorrect translations in the nested page tables, > stale translations in the TLB etc. >=20 > Do you have a core file for the panic? It would be very useful to > debug this further. No, unfortunately I did not have swap or dumpdev setup at the time so I was unable to get a core dump from the crashed kernel. (Bhyve did not crash.) I= 've setup swap in the VM and set the dumpdev as well, so if it happens again I should get a core. Steve --2fHTh5uZTiUOsy+g Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTubqhAAoJEPXPYrMgexuhGaEIAIPXiqXQoTRWwFWOfiYAuk3Q lTXzSniPbGlogInzSP7F1ZqmYEqdABLnPUYN5oThqG2xbhVky1dJiGsS6gDeNY1e wrWRM77WWPuhPrZduXHUGb0ppBvuXaknoGGep21rTsKMspRCbBk75bEdKlfvskpF 9cl0DdfCPdkpJg3/sv6hIMDNv62vkK3lYHLdBMrWoGaO+aBJ7+SDPb3J5k8yGBMf UhJKE5eOfVKYJnGMdNKZ1vjfEC85BtRbQnLLJOM5M7EYxHRmddo2UFddPcqtr5JO PN3sHwQl64MEXswnxHyyPhjUhKXia1Xqd+nOIwts2rcfioZsiLyV6KIkqrou05s= =ljLU -----END PGP SIGNATURE----- --2fHTh5uZTiUOsy+g-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Jul 11 19:39:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B15D8FD7; Fri, 11 Jul 2014 19:39:26 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0DBD62EF6; Fri, 11 Jul 2014 19:39:25 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id gl10so1257233lab.31 for ; Fri, 11 Jul 2014 12:39:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/h0Q5W47YxZ9sekq5UbePh6QzrthfgDrwUPwLB/ovr0=; b=zu6ey8/2b0RwcRoas28hNWxqOqNTfhQIc55DTLLmP0XVQv8aw4JKbdBZsQQjRPo4T8 U+vwDKAzqOvMsLVVxB68fjwdUTAHIsEvLhi9S7DGWwlFS1v7SdVEk4Net+DpPzK2K+od dnuNxNaRBMQDZm7dYvEKDPtAa/uXO5Z08eTL/w5KcAR111PsdFZuLel1JxrWaIiOfsNF DliBjx20oAxfLCf2FX9hHT/Th7qZMy8o7OyYBKPfkvtawcxWsDUY0IJyI/qHqAH0IWj9 jv+aIg40miAbcU+coSSLPo68vtDDOywaFjX3YNsTW+1r0hxrNFhkMqoxn0+qyV6wl/WA lNiw== MIME-Version: 1.0 X-Received: by 10.112.131.202 with SMTP id oo10mr663530lbb.65.1405107563966; Fri, 11 Jul 2014 12:39:23 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Fri, 11 Jul 2014 12:39:23 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Jul 2014 12:39:23 -0700 X-Google-Sender-Auth: vmNamqUE8J-l8_HRHItKNlksfnU Message-ID: Subject: Re: BAFUG July 10: libvirt virtualization API for controlling bhyve virtual machines From: Craig Rodrigues To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-current Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 19:39:26 -0000 Hi, Slides for this talk are available here: http://people.freebsd.org/~rodrigc/libvirt/presentations/ iXsystems recorded video of the presentation. When they put it online, I'll send the link. -- Craig On Mon, Jun 30, 2014 at 5:30 PM, Craig Rodrigues wrote: > Hi, > > On July 10, 2014, in Mountain View, California, I will be giving a brief > tech talk on: > > "libvirt virtualization API for controlling bhyve virtual machines" > > I have been testing the extensions that Roman Bogorodskiy < > novel@freebsd.org> > has made to the lbivirt library for supporting bhyve. I will talk about > my experiences with libvirt and bhyve. > > Here are more details about the talk plus directions to the location: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/191061222/ > > Feel free to sign up on the meetup site and attend if you are in the area! > -- > Craig > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jul 14 15:04:51 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66C5D20B; Mon, 14 Jul 2014 15:04:51 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03FA12122; Mon, 14 Jul 2014 15:04:50 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6EF4j64080571 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 14 Jul 2014 18:04:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s6EF4j64080571 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6EF4jdC080570; Mon, 14 Jul 2014 18:04:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 14 Jul 2014 18:04:45 +0300 From: Konstantin Belousov To: Steve Wills Subject: Re: tmpfs panic Message-ID: <20140714150445.GJ93733@kib.kiev.ua> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tby8mjhWFMo012j0" Content-Disposition: inline In-Reply-To: <20140706154621.GA81830@mouf.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: virtualization@freebsd.org, current@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 15:04:51 -0000 --tby8mjhWFMo012j0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 06, 2014 at 03:46:22PM +0000, Steve Wills wrote: > I should have noted this system is running in bhyve. Also I'm told this p= anic > may be related to the fact that the system is running in bhyve. >=20 > Looking at it a little more closely: >=20 > (kgdb) list *__mtx_lock_sleep+0xb1 > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex.c= :431). > 426 * owner stops running or the state of the lock c= hanges. > 427 */ > 428 v =3D m->mtx_lock; > 429 if (v !=3D MTX_UNOWNED) { > 430 owner =3D (struct thread *)(v & ~MTX_FLAG= MASK); > 431 if (TD_IS_RUNNING(owner)) { > 432 if (LOCK_LOG_TEST(&m->lock_object= , 0)) > 433 CTR3(KTR_LOCK, > 434 "%s: spinning on %p h= eld by %p", > 435 __func__, m, owner); > (kgdb)=20 >=20 > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CONT= ENDED > is spuriously left behind, and to ask how lock prefix is handled in bhyve= =2E Any > of that make sense to anyone? >=20 > Thanks, > Steve >=20 > On Sun, Jul 06, 2014 at 01:53:37PM +0000, Steve Wills wrote: > > Hi, > >=20 > > Just experienced this tmpfs panic on r268160: > >=20 > > Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of m= emory. > >=20 > >=20 > > Fatal trap 12: page fault while in kernel mode > > cpuid =3D 12; apic id =3D 0c > > fault virtual address =3D 0x378 > > fault code =3D supervisor read data, page not present > > instruction pointer =3D 0x20:0xffffffff809638d1 > > stack pointer =3D 0x28:0xfffffe07243800a0 > > frame pointer =3D 0x28:0xfffffe0724380120 > > code segment =3D base 0x0, limit 0xfffff, type 0x1b > > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > > current process =3D 65339 (pkg-static) > > [ thread pid 65339 tid 101641 ] > > Stopped at __mtx_lock_sleep+0xb1: movl 0x378(%rax),%ecx > > db> bt > > Tracing pid 65339 tid 101641 td 0xfffff80286b2e490 > > __mtx_lock_sleep() at __mtx_lock_sleep+0xb1/frame 0xfffffe0724380120 > > free_unr() at free_unr+0x9d/frame 0xfffffe0724380160 > > tmpfs_free_node() at tmpfs_free_node+0xf2/frame 0xfffffe07243801a0 > > tmpfs_reclaim() at tmpfs_reclaim+0xdc/frame 0xfffffe07243801d0 > > VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xa7/frame 0xfffffe0724380200 > > vgonel() at vgonel+0x24c/frame 0xfffffe0724380280 > > vrecycle() at vrecycle+0x84/frame 0xfffffe07243802c0 > > tmpfs_inactive() at tmpfs_inactive+0x18/frame 0xfffffe07243802d0 > > VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xa7/frame 0xfffffe0724380300 > > vinactive() at vinactive+0x181/frame 0xfffffe0724380360 > > vputx() at vputx+0x30d/frame 0xfffffe07243803d0 > > vn_close() at vn_close+0x13e/frame 0xfffffe0724380450 > > vn_closefile() at vn_closefile+0x48/frame 0xfffffe07243804d0 > > _fdrop() at _fdrop+0x29/frame 0xfffffe07243804f0 > > closef() at closef+0x2ae/frame 0xfffffe0724380580 > > fdescfree() at fdescfree+0x64c/frame 0xfffffe0724380630 > > exit1() at exit1+0x682/frame 0xfffffe07243806c0 > > sigexit() at sigexit+0x929/frame 0xfffffe0724380980 > > postsig() at postsig+0x3c4/frame 0xfffffe0724380a70 > > ast() at ast+0x487/frame 0xfffffe0724380ab0 > > doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffc6e0 > > db>=20 > >=20 > > Any further debugging I can do? > >=20 > > Thanks, > > Steve >=20 >=20 This is supposedly fixed with r268617. --tby8mjhWFMo012j0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTw/GMAAoJEJDCuSvBvK1BOVoP+wT6cFd+bEPrId4k2CXtd8Gm RP34XpfopJaGiTl56W8ETIop+bVO6t9Z6uVmky2N1rVUaJqTfiAXuNBTCLG4U8Rd uNO/kK1D6G/Y1ZzW8P5QxVomdaabF/k2HWYa08UV+0Yl5QXrhIa0eaxccP6TBISz V8/HTXj2VvkXDBuQsquc6k10OGxZjmaj113TCoRBUXCKGgMk0kGb15PN7zefbr5Y bJXgf49WU5AJp2M2Dmpk2JFjXJ8GSapo/FdQVTxHfAdGa2sV3Vl0i8lWiFqm7oaf a1irBQwdV6HTvoIg1OXviOk30ZFRQ/UfVRxsXVU0QmDCe4aLJc1uGwndr8XeJmaP J/CJEidDQJA8oVGaTL7UvOkExgQpDm6RXvh8/pVkbo1Lw1EmVhnaSBWAXR1zSIap ORZ7UNGD6p76GEn4Ic2HfgFsnTeDuw8sFdPaIJYzWJrucZciWFS17EpOHeVLxxJ7 FRCKeRoPwXqyaxZJHpSdx6KGrKeVNNS+D24bPqORSLyO4lfbXmiR4PLl3lDyNY/d K9jXoZVOxtdHUU/ulU7jEOtAshvAyv2nxPUmfiiVlJIs1QybSY+8bEOgL6/OYzUD r8wHuq92gSg7OXOYzgI1ECA2znHjP5zydSgkgiz8d53Ok09S3JfviG/4b8pgsVUf 1yXg13mE3Kop9kjYWUY8 =74qe -----END PGP SIGNATURE----- --tby8mjhWFMo012j0--