From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 10 01:33:50 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07A437CE; Mon, 10 Nov 2014 01:33:50 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 773EBF2B; Mon, 10 Nov 2014 01:33:49 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id pn19so6829428lab.4 for ; Sun, 09 Nov 2014 17:33:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=NU2X8LRXWxqU+NH7LLOpMPR++tS3nvPhyI0SvzKZvMo=; b=v3Zf5bIJC8J/j1YjD8wKzQntmXvVQuiTimGfa5Sg6LaL/qb/1PMEV8PzIXgorTXbWn g2OqVzJYGoDKhf7IhW0CfUOfBbhC6RkcuqHZkktZsWEfvdLstFnPpljVJs1Y/rSmmCh+ as08q5S8pkd5AfwYampA6/a37ElIbQ3x0xo3jxy8H2kbmAwL4IrybtT73OgHW2uRNxJf r11n0C6kElU1ONpndN6qFMIkjrauIaCuZBm7Zaj7Ur2p3BTAKSkEJ9cBg6EKlWHbsLQF Xa1RUdr42HNxNecvfhDJFCUrgvFrGUASwQm0IGDgEZyy0hu2HY2quQy0ewAFyqr0vP2W kR9Q== MIME-Version: 1.0 X-Received: by 10.152.37.104 with SMTP id x8mr6484542laj.74.1415583227074; Sun, 09 Nov 2014 17:33:47 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.130.168 with HTTP; Sun, 9 Nov 2014 17:33:46 -0800 (PST) In-Reply-To: <7EAA2A23-06F9-44C9-A3E1-62AA37EE5CDA@lists.zabbadoz.net> References: <20141012182551.002b3cc0a45a56d3f34e6174@yamagi.org> <3B4471A7-CDF4-440D-BDD8-3D5B2256B8DD@lists.zabbadoz.net> <7EAA2A23-06F9-44C9-A3E1-62AA37EE5CDA@lists.zabbadoz.net> Date: Sun, 9 Nov 2014 17:33:46 -0800 X-Google-Sender-Auth: hIDLToTb1ol9m0GH421ObSESjlQ Message-ID: Subject: Re: Enabling VIMAGE by default for FreeBSD 11? From: Craig Rodrigues To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Nov 2014 01:33:50 -0000 On Sun, Oct 12, 2014 at 6:07 PM, Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > > > > Can you provide a pointer to your Perforce branch? > > //depot/user/bz/vimage/src/... > > Hi, Since I am more familiar with git than Perforce, I converted your Perforce branch to git and put it on github: https://github.com/rodrigc/bz-vimage I took a look at the history of that branch, and it looks like you merged quite a lot of changes in this branch back to FreeBSD. There were a few places where it looks like the code in your branch diverged from FreeBSD (in carp area, for example). Offhand, can you remember any VIMAGE related memory leaks you might have fixed in this branch which you did not merge back? This one looks pretty simple by removing UMA_ZONE_NOFREE in a few places: https://github.com/rodrigc/bz-vimage/commit/ebe7e4c5e7e5b3dbfc442a25f10ca8681c605c89 In this one, you added dom_pr_register() and dom_pr_unregister() hooks: https://github.com/rodrigc/bz-vimage/commit/a1d5c8bc2f4484e58594ca8fad793aa339a5ef29 but I'm not sure if you wanted to merge this back to FreeBSD or not. Can you think of anything else in this branch that we need for VIMAGE? Thanks. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 10 09:30:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AE9B184; Mon, 10 Nov 2014 09:30:30 +0000 (UTC) Received: from mail-vc0-x236.google.com (mail-vc0-x236.google.com [IPv6:2607:f8b0:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F9F8750; Mon, 10 Nov 2014 09:30:30 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id im17so465395vcb.41 for ; Mon, 10 Nov 2014 01:30:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RTfCTeIcpIkN67rd52RVDCU/QbfGdmyV22ZFj+9h4jQ=; b=E8V6GfwXbQElZGTLKhpsedNtn6LxAZtT7hUuMqSsXlLitRYtibq9JB/r8YQYoBaSPN fxTDP1x/so5HrQaIrKuTuul+fOeYarJpGyzfYgAWOULsF70k5TLTjfksbsitMfAoD/03 ch+y0rUg5b4Z/DPfRwZ3neG62f7XNGuYhprjq/YFjt5NTeFnUgz0cMwxVNmYybB0gdtS K0tfsQjRnV3q72E6oOP/lC79Wn55dhl8B1yZGSi3K7EB4+w4W6t/ACr6K5Ja0nukK/To bilHIG87Y/7IgfLmh77nrktEV7DZqF37os4UBbV+bDCW2mDtKHkZCbPPkCrAkSqrumM2 LnVg== MIME-Version: 1.0 X-Received: by 10.52.9.37 with SMTP id w5mr1487067vda.38.1415611829334; Mon, 10 Nov 2014 01:30:29 -0800 (PST) Received: by 10.220.168.199 with HTTP; Mon, 10 Nov 2014 01:30:29 -0800 (PST) In-Reply-To: References: <20141012182551.002b3cc0a45a56d3f34e6174@yamagi.org> <3B4471A7-CDF4-440D-BDD8-3D5B2256B8DD@lists.zabbadoz.net> <7EAA2A23-06F9-44C9-A3E1-62AA37EE5CDA@lists.zabbadoz.net> Date: Mon, 10 Nov 2014 10:30:29 +0100 Message-ID: Subject: Re: Enabling VIMAGE by default for FreeBSD 11? From: Nikolay Denev To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "Bjoern A. Zeeb" , freebsd-arch , "freebsd-virtualization@freebsd.org" , FreeBSD Net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Nov 2014 09:30:30 -0000 On Mon, Nov 10, 2014 at 2:33 AM, Craig Rodrigues wrote: > On Sun, Oct 12, 2014 at 6:07 PM, Bjoern A. Zeeb < > bzeeb-lists@lists.zabbadoz.net> wrote: > >> >> >> > Can you provide a pointer to your Perforce branch? >> >> //depot/user/bz/vimage/src/... >> >> > Hi, > > Since I am more familiar with git than Perforce, I converted > your Perforce branch to git and put it on github: > > https://github.com/rodrigc/bz-vimage > > I took a look at the history of that branch, and it looks like you > merged quite a lot of changes in this branch back to FreeBSD. > There were a few places where it looks like the code in your branch > diverged from FreeBSD (in carp area, for example). > > Offhand, can you remember any VIMAGE related memory leaks > you might have fixed in this branch which you did not merge back? > > This one looks pretty simple by removing UMA_ZONE_NOFREE in a few > places: > > https://github.com/rodrigc/bz-vimage/commit/ebe7e4c5e7e5b3dbfc442a25f10ca8681c605c89 > > > In this one, you added dom_pr_register() and dom_pr_unregister() hooks: > > https://github.com/rodrigc/bz-vimage/commit/a1d5c8bc2f4484e58594ca8fad793aa339a5ef29 > > but I'm not sure if you wanted to merge this back to FreeBSD or not. > > Can you think of anything else in this branch that we need for VIMAGE? > > Thanks. > > -- > Craig > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" I haven't checked if this is fixed in CURRENT, but at least on 10.0-STABLE r270295M, gif(4) does not seem to play well with VIMAGE. I've just noticed that gif(4) interface unit numbers seem to be unique per machine, regardless of the vnet (I guess unit numbering not properly virtualized), so that if I create gif0 in one vnet jail and try the same in another vnet jail I get "ifconfig: SIOCIFCREATE2: File exists" What's even worse is that once the jail is destroyed, the gif(4) tunnel interface disappears from the system (no longer shows in ifconfig), but you can't reuse the unit number, so I continue to get SIOCIFCREATE2: File exists for gif0 on the host or other vnet jails. --Nikolay From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 10 14:14:34 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4E422F2; Mon, 10 Nov 2014 14:14:33 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AB13098F; Mon, 10 Nov 2014 14:14:33 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sAAEEP7R053048 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 10 Nov 2014 06:14:28 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <5460C839.7060404@freebsd.org> Date: Mon, 10 Nov 2014 22:14:17 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Nikolay Denev , Craig Rodrigues Subject: Re: Enabling VIMAGE by default for FreeBSD 11? References: <20141012182551.002b3cc0a45a56d3f34e6174@yamagi.org> <3B4471A7-CDF4-440D-BDD8-3D5B2256B8DD@lists.zabbadoz.net> <7EAA2A23-06F9-44C9-A3E1-62AA37EE5CDA@lists.zabbadoz.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Bjoern A. Zeeb" , FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Nov 2014 14:14:34 -0000 On 11/10/14, 5:30 PM, Nikolay Denev wrote: > On Mon, Nov 10, 2014 at 2:33 AM, Craig Rodrigues wrote: >> On Sun, Oct 12, 2014 at 6:07 PM, Bjoern A. Zeeb < >> bzeeb-lists@lists.zabbadoz.net> wrote: >> >>> >>>> Can you provide a pointer to your Perforce branch? >>> //depot/user/bz/vimage/src/... >>> >>> >> Hi, >> >> Since I am more familiar with git than Perforce, I converted >> your Perforce branch to git and put it on github: >> >> https://github.com/rodrigc/bz-vimage >> >> I took a look at the history of that branch, and it looks like you >> merged quite a lot of changes in this branch back to FreeBSD. >> There were a few places where it looks like the code in your branch >> diverged from FreeBSD (in carp area, for example). >> >> Offhand, can you remember any VIMAGE related memory leaks >> you might have fixed in this branch which you did not merge back? >> >> This one looks pretty simple by removing UMA_ZONE_NOFREE in a few >> places: >> >> https://github.com/rodrigc/bz-vimage/commit/ebe7e4c5e7e5b3dbfc442a25f10ca8681c605c89 >> >> >> In this one, you added dom_pr_register() and dom_pr_unregister() hooks: >> >> https://github.com/rodrigc/bz-vimage/commit/a1d5c8bc2f4484e58594ca8fad793aa339a5ef29 >> >> but I'm not sure if you wanted to merge this back to FreeBSD or not. >> >> Can you think of anything else in this branch that we need for VIMAGE? >> >> Thanks. >> >> -- >> Craig >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > I haven't checked if this is fixed in CURRENT, but at least on > 10.0-STABLE r270295M, > gif(4) does not seem to play well with VIMAGE. > > I've just noticed that gif(4) interface unit numbers seem to be unique > per machine, regardless of the vnet (I guess unit numbering not > properly virtualized), > so that if I create gif0 in one vnet jail and try the same in another > vnet jail I get "ifconfig: SIOCIFCREATE2: File exists" > > What's even worse is that once the jail is destroyed, the gif(4) > tunnel interface disappears from the system (no longer shows in > ifconfig), but you can't reuse the unit number, so > I continue to get SIOCIFCREATE2: File exists for gif0 on the host or > other vnet jails. yes there are some parts of the system where the design is not compatible with virtualization. for example.. if a single /dev entry controls stuff.. which kind of complicates things.. > > --Nikolay > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 10 23:49:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24C4218A for ; Mon, 10 Nov 2014 23:49:48 +0000 (UTC) Received: from mail1.bur200.uecomm.net.au (mail1.bur200.uecomm.net.au [218.185.0.70]) by mx1.freebsd.org (Postfix) with ESMTP id D5C49E4E for ; Mon, 10 Nov 2014 23:49:47 +0000 (UTC) Received: from mail.flexibledrive.com.au (unknown [115.186.196.106]) by mail1.bur200.uecomm.net.au (Postfix) with ESMTP id 700BBD3D3 for ; Tue, 11 Nov 2014 10:49:44 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.flexibledrive.com.au (Postfix) with ESMTP id D54A1E67C0 for ; Tue, 11 Nov 2014 10:49:44 +1100 (EST) X-Virus-Scanned: amavisd-new at fdrive.com.au Received: from mail.flexibledrive.com.au ([127.0.0.1]) by localhost (mail.flexibledrive.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KuZinhbvmExJ for ; Tue, 11 Nov 2014 10:49:36 +1100 (EST) Received: from ws-pross.vv.fda (ws-pross.vv.fda [192.168.50.199]) by mail.flexibledrive.com.au (Postfix) with ESMTPS id 6BAECE679A for ; Tue, 11 Nov 2014 10:49:36 +1100 (EST) Date: Tue, 11 Nov 2014 10:49:36 +1100 (AEDT) From: Peter Ross X-X-Sender: petros@linux-vic-05.vv.fda To: freebsd-virtualization@freebsd.org Subject: VirtualBox & X11: Guest console does not display remotey anynmore Message-ID: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Nov 2014 23:49:48 -0000 Hi, since yesterday I have a weird problem. I cannot display VirtualBox _guest_ consoles remotely anymore (the GUI for configuration works). It worked two weeks ago and I am not aware of any changes. I am running Virtualbox headless on a server but use the GUI for installs when needed. I forward ssh (-X) and use "su -m" to keep the DISPLAY environment variable(DISPLAY=localhost:10.0) and xauth. Two weeks ago I did just that, 1. started "VirtualBox", 2. created a VM and a disk, 3. attached an ISO disk 4. pressed start and the "BIOS" appeared Yesterday it all goes up to step 3- just 4 does not work. I get an error instead of the BIOS screen, and on the server I get: X11 connection rejected because of wrong authentication. I checked all variables and parts - nothing has changed since the last install that worked a fortnight ago. Same kernel, same VirtualBox, same xauth, same desktop. # uname -a FreeBSD DellT620one.vv.fda 10.1-PRERELEASE FreeBSD 10.1-PRERELEASE #0 r273569: Fri Oct 24 14:43:34 AEDT 2014 root@DellT410two.vv.fda:/usr/obj/usr/src/sys/IPSEC amd64 # pkg info | grep virtual virtualbox-ose-4.3.18 General-purpose full virtualizer for x86 hardware virtualbox-ose-kmod-4.3.18 VirtualBox kernel module for FreeBSD The desktop Oracle Linux 6.5. I used this on the 28th of October successfully. No fresh installs/upgrades since, I believe. I can open X11 clients remotely (e.g. the VirtualBox control GUI works - just not the guest console) Do you have any ideas what could go wrong? Thanks for ideas Peter From owner-freebsd-virtualization@FreeBSD.ORG Tue Nov 11 00:01:31 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB2583DF for ; Tue, 11 Nov 2014 00:01:31 +0000 (UTC) Received: from mail1.bur200.uecomm.net.au (mail1.bur200.uecomm.net.au [218.185.0.70]) by mx1.freebsd.org (Postfix) with ESMTP id A7937FE0 for ; Tue, 11 Nov 2014 00:01:31 +0000 (UTC) Received: from mail.flexibledrive.com.au (unknown [115.186.196.106]) by mail1.bur200.uecomm.net.au (Postfix) with ESMTP id 9D4C0D46B for ; Tue, 11 Nov 2014 10:39:43 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.flexibledrive.com.au (Postfix) with ESMTP id 75C38E6A2E for ; Tue, 11 Nov 2014 10:39:43 +1100 (EST) X-Virus-Scanned: amavisd-new at fdrive.com.au Received: from mail.flexibledrive.com.au ([127.0.0.1]) by localhost (mail.flexibledrive.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9STn2IlGI0OL for ; Tue, 11 Nov 2014 10:39:35 +1100 (EST) Received: from ws-pross.vv.fda (ws-pross.vv.fda [192.168.50.199]) by mail.flexibledrive.com.au (Postfix) with ESMTPS id 1659CE6104 for ; Tue, 11 Nov 2014 10:39:35 +1100 (EST) Date: Tue, 11 Nov 2014 10:39:34 +1100 (AEDT) From: Peter Ross X-X-Sender: petros@linux-vic-05.vv.fda To: freebsd-virtualization@freebsd.org Subject: Re: VIMAGE and VirtualBox networking question Message-ID: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 00:01:32 -0000 Nikos Vassiliadis wrote: > On 11/05/14 00:29, Manas Bhatnagar wrote: >> Install ports tree in a temporary location (another computer or >> something), grab the required folder and move it to the computer that >> you want to install the port on and do 'make install clean' ? probably >> won't work like that, you would need to make sure the directories for >> dependencies are there too :o) > Tried to do this with a full ports tree and I stopped it after gcc > started to build. BTW: You need /usr/src available for the creation of the package. Preferably matching the kernel you want to install it.. I do this regularly (but without VIMAGE) in a jail where I build all my packages. ("make install" and "pkg create" in a script) I do all updates from source. I only update kernel, user land and ports when security is at risk. Usually independent from each other. But VirtualBox is an exception. I update kernel and VirtualBox at the same time. I was not brave enough to enable VIMAGE and VirtualBox on the same server. But I may do that soon. I plan a major reconfiguration ca. end of the year. I hope it is stable enough. Regards Peter From owner-freebsd-virtualization@FreeBSD.ORG Tue Nov 11 00:14:53 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E87EE95A for ; Tue, 11 Nov 2014 00:14:53 +0000 (UTC) Received: from mail1.bur200.uecomm.net.au (mail1.bur200.uecomm.net.au [218.185.0.70]) by mx1.freebsd.org (Postfix) with ESMTP id 99CD1154 for ; Tue, 11 Nov 2014 00:14:53 +0000 (UTC) Received: from mail.flexibledrive.com.au (unknown [115.186.196.106]) by mail1.bur200.uecomm.net.au (Postfix) with ESMTP id 4DEB7D55D for ; Tue, 11 Nov 2014 11:14:51 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.flexibledrive.com.au (Postfix) with ESMTP id B4CE8E682F for ; Tue, 11 Nov 2014 11:14:51 +1100 (EST) X-Virus-Scanned: amavisd-new at fdrive.com.au Received: from mail.flexibledrive.com.au ([127.0.0.1]) by localhost (mail.flexibledrive.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ryN2JO4OEfLc for ; Tue, 11 Nov 2014 11:14:43 +1100 (EST) Received: from ws-pross.vv.fda (ws-pross.vv.fda [192.168.50.199]) by mail.flexibledrive.com.au (Postfix) with ESMTPS id 4FC66E6948 for ; Tue, 11 Nov 2014 11:14:43 +1100 (EST) Date: Tue, 11 Nov 2014 11:14:42 +1100 (AEDT) From: Peter Ross X-X-Sender: petros@linux-vic-05.vv.fda To: freebsd-virtualization@freebsd.org Subject: Re: VirtualBox & X11: Guest console does not display remotey anynmore In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 00:14:54 -0000 On Tue, 11 Nov 2014, Peter Ross wrote: > since yesterday I have a weird problem. I cannot display VirtualBox > _guest_ consoles remotely anymore (the GUI for configuration works). > > It worked two weeks ago and I am not aware of any changes. > > I am running Virtualbox headless on a server but use the GUI for installs > when needed. I forward ssh (-X) and use "su -m" to keep the DISPLAY > environment variable(DISPLAY=localhost:10.0) and xauth. > > Two weeks ago I did just that, > > 1. started "VirtualBox", > 2. created a VM and a disk, > 3. attached an ISO disk > 4. pressed start and the "BIOS" appeared > > Yesterday it all goes up to step 3- just 4 does not work. I get an error > instead of the BIOS screen, and on the server I get: > > X11 connection rejected because of wrong authentication. I found a workaround: VBoxSDL --startvm SageConfig This works and I can install my beloved Windows 7;-) Weird.. as said, the other way worked two weeks ago and nothing changed (I believe) Regards Peter From owner-freebsd-virtualization@FreeBSD.ORG Tue Nov 11 22:12:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8089C992 for ; Tue, 11 Nov 2014 22:12:42 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 42EF06BF for ; Tue, 11 Nov 2014 22:12:42 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 8EC6D12699; Wed, 12 Nov 2014 08:12:34 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id BZR26236 (AUTH peterg@ptree32.com.au); Wed, 12 Nov 2014 08:12:33 +1000 Message-ID: <546289CE.9090903@freebsd.org> Date: Tue, 11 Nov 2014 14:12:30 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: =?UTF-8?B?7Jik64+Z7ISx?= Subject: Re: bhyveload Will not support i386 root on zfs vm ? please help.. References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 22:12:42 -0000 Hi, > I've tried install i386 vm and that didn't work Thanks for the report. The issue is now fixed in 11-CURRENT as of r274407, and the change will be MFC'd to 10-stable in a week or so. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 21:46:43 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DEA17799 for ; Wed, 12 Nov 2014 21:46:43 +0000 (UTC) Received: from mail2.openmailbox.org (mail2.openmailbox.org [62.4.1.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9DC92D0F for ; Wed, 12 Nov 2014 21:46:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id D6927201ABA for ; Wed, 12 Nov 2014 22:39:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=openmailbox.org; h=user-agent:message-id:subject:subject:from:from:date:date :content-transfer-encoding:content-type:content-type :mime-version:received:received; s=openmailbox; t=1415828365; bh=sW9/AfvTj0FVd1rwZFT0N7iyGA9R3bz4xB2LmPu3Rj0=; b=liyl7Kfi66us v3JGpMldf6r1d3hbkCW89DEKqGYmWkXpucbDk2qPq2obHdsUQadYJkoCCUvKpNmZ bX04YOUQ6zilCWQgyCNU1fCD382W16V9ufu3XGY5EHEKnUb66JF7AoTdiY5nSkOb 1tr5FbCrlc4/fYI9DV9sL83JGzIkdtw= X-Virus-Scanned: at openmailbox.org Received: from mail2.openmailbox.org ([62.4.1.33]) by localhost (mail.openmailbox.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EUKaUL627NNt for ; Wed, 12 Nov 2014 22:39:25 +0100 (CET) Received: from www.openmailbox.org (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id 712742000C5 for ; Wed, 12 Nov 2014 22:39:25 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 12 Nov 2014 23:39:25 +0200 From: tinkr@openmailbox.org To: freebsd-virtualization@freebsd.org Subject: How hard is BHyVe's 16 vCPU limit, is it configurable under any =?UTF-8?Q?circumstance=3F?= Message-ID: <8e39523570b667d223091adc3791c9e2@openmailbox.org> X-Sender: tinkr@openmailbox.org User-Agent: Roundcube Webmail/1.0.2 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 21:46:44 -0000 Hi! In order justify giving energy to BHyVe, I need to know if it's "future-proof" in that the 16 vCPU limit can be increased? Please let the world know if BHyVe's 16 vCPU limit can be lifted in some way, by configuration, patch, etc. (and if you want to, why this limit is in place by default today). Thanks!! Tinker From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 21:55:20 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 603A9A40 for ; Wed, 12 Nov 2014 21:55:20 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 39C1BDFE for ; Wed, 12 Nov 2014 21:55:19 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id B64D26EA44 for ; Wed, 12 Nov 2014 21:55:10 +0000 (UTC) Message-ID: <5463D746.4030803@freebsd.org> Date: Wed, 12 Nov 2014 16:55:18 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: How hard is BHyVe's 16 vCPU limit, is it configurable under any circumstance? References: <8e39523570b667d223091adc3791c9e2@openmailbox.org> In-Reply-To: <8e39523570b667d223091adc3791c9e2@openmailbox.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uxxN9oA2n98iBCkgMAgjejrhQE3nMXXEh" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 21:55:20 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --uxxN9oA2n98iBCkgMAgjejrhQE3nMXXEh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2014-11-12 16:39, tinkr@openmailbox.org wrote: > Hi! >=20 > In order justify giving energy to BHyVe, I need to know if it's > "future-proof" in that the 16 vCPU limit can be increased? >=20 > Please let the world know if BHyVe's 16 vCPU limit can be lifted in som= e > way, by configuration, patch, etc. (and if you want to, why this limit > is in place by default today). >=20 > Thanks!! > Tinker >=20 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" You can increase the limit by editing sys/amd64/include/vmm.h #define VM_MAXCPU 16 =46rom what I've been told, things scale badly above 24 CPUs. They plan t= o solve this issue, but have not yet. If you system has enough cores to support using more than 16 per VM, you can modify the file and recompile the kernel and use as many CPUs as you want, but not much testing has happened with bigger numbers. --=20 Allan Jude --uxxN9oA2n98iBCkgMAgjejrhQE3nMXXEh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUY9dIAAoJEJrBFpNRJZKfs/EP/2nHN5D6amqE5CtfI/PAdK8s mJmBHII4socC5F7eyHfjue1grb8kW9MaJR0/9v1hAcAFvtGo/8qK3junc0/BGc8V +3uMG2wQ4k1E4XDCvjk/J82DpGKDZVdd3A/lLKYixhuN+QDRnGyrmACbgnXCMDt/ lsd2VaPScZ8Hql5WrTAmNJceyGyb53I0M3AboAEPMFb3kQR8iW3dym8qDYKmijzJ sBbBrNwNNTwDLqxdeBj01AtTDpdhtQOf4QReDtZ2u2fXnpRsBDSlL4si+6/c+Gl3 CvPYy2/dP8GBoOvNGT6a3nlGyWFluwtnKBzGz4EyRsdclXvkEETvASi8cgQ2kvwU h+pGIKhT5vV9IXPYyIxuocaTZh3y3HhK3Bs1NDQoBGafjQDc6cP8VJOdZEXUxTLU D/k2ofQv9r+brP8laabFuiqHCVrUtH4r5YQct1qh8aYR+ZUA6+tHCQCdwL2iYF3R ZgGCowZD3kd5LE3mZREtRLfLlrzjoB/ytyrlGQkRXCcUj18KtYw+6ZEVR0xccos0 TPDws9nIOf5DfcZYOiOm128qNeOIDPlFoHckW1VB1yRRpi1ENBUM/eEvhsHJSLFR AkV3/agE913Nb/z5z9o5X489WADY1OJBnHlSZNzEs7rZjF91yTIIP0kaRrPI9fXm d+rRYq0YKMqiexQ/p2Ac =p3qT -----END PGP SIGNATURE----- --uxxN9oA2n98iBCkgMAgjejrhQE3nMXXEh-- From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 22:09:01 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BC07D78; Wed, 12 Nov 2014 22:09:01 +0000 (UTC) Received: from mail2.openmailbox.org (mail2.openmailbox.org [62.4.1.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D7ABF33; Wed, 12 Nov 2014 22:08:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id CAA102026F8; Wed, 12 Nov 2014 23:08:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=openmailbox.org; h=user-agent:message-id:references:in-reply-to:subject:subject :from:from:date:date:content-transfer-encoding:content-type :content-type:mime-version:received:received; s=openmailbox; t= 1415830121; bh=Xl2SE0LTxE1MbZ5EScwcYsX20LtfbpVcp6rl/X1zw/E=; b=j 4kFOxiZNY2IS6O0wmeBrk8HZN7XH+SdroOOfX5lv1ibPHuyB7UUwodZFfRKoBfRk 8MjrW/vynkzb6PwiLzjHQbqryn8SokJjWU4YZLEdR5c5r/6l6za+c+ZpE3RE2znr H4f9GUbkEpS/0N1pn4wfAQWJ5d18QLFzK7nJRYtx50= X-Virus-Scanned: at openmailbox.org Received: from mail2.openmailbox.org ([62.4.1.33]) by localhost (mail.openmailbox.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p-ceXJpBeVpW; Wed, 12 Nov 2014 23:08:41 +0100 (CET) Received: from www.openmailbox.org (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id 1F1D02020BC; Wed, 12 Nov 2014 23:08:40 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 13 Nov 2014 00:08:40 +0200 From: Tinker To: Allan Jude Subject: Re: How hard is BHyVe's 16 vCPU limit, is it configurable under any =?UTF-8?Q?circumstance=3F?= In-Reply-To: <5463D746.4030803@freebsd.org> References: <8e39523570b667d223091adc3791c9e2@openmailbox.org> <5463D746.4030803@freebsd.org> Message-ID: <49cc2695179830c899aef87b7a8b72d8@openmailbox.org> X-Sender: tinkr@openmailbox.org User-Agent: Roundcube Webmail/1.0.2 Cc: owner-freebsd-virtualization@freebsd.org, freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 22:09:01 -0000 On 2014-11-12 23:55, Allan Jude wrote: > On 2014-11-12 16:39, tinkr@openmailbox.org wrote: >> Hi! >> >> In order justify giving energy to BHyVe, I need to know if it's >> "future-proof" in that the 16 vCPU limit can be increased? >> >> Please let the world know if BHyVe's 16 vCPU limit can be lifted in >> some >> way, by configuration, patch, etc. (and if you want to, why this limit >> is in place by default today). >> >> Thanks!! >> Tinker >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > You can increase the limit by editing sys/amd64/include/vmm.h > > #define VM_MAXCPU 16 > > From what I've been told, things scale badly above 24 CPUs. They plan > to > solve this issue, but have not yet. If you system has enough cores to > support using more than 16 per VM, you can modify the file and > recompile > the kernel and use as many CPUs as you want, but not much testing has > happened with bigger numbers. Dear Allan, Thank you very much for responding. Aha, great! Only for completeness, do you have any particular idea about * When the scaling above 24 vCPU:s will be optimized, like approx how many years away is it (like 1 or more than 1)?, and * What the technological reason for the scaling is, is it that somehow the BHyVe instances on the different cores need to inter-communicate, for instance that all disk and network IO is done via one single core currently? In all cases, your response is great news, as your baseline answer that it's doable and only a question of optimization and tweaking of present functionality - Thanks again! Tinker From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 22:31:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D80C4A62; Wed, 12 Nov 2014 22:31:17 +0000 (UTC) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67D742ED; Wed, 12 Nov 2014 22:31:17 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id b13so15427726wgh.26 for ; Wed, 12 Nov 2014 14:31:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LebA+T87HannqAvVtzNUYjGtPmqgGQLNYMf4WJnxokA=; b=cAnGvvKpO+w/m3NvVk5XW2/e1qnaqseWslEeZg4hU3l59B9LwnoSLyCNs9h33jh1vh hA9ECIw2d0EJlXPmOuMPFCbPcNB/EH5LJhM1ohT3bOwuq417fY7b30JI/BGzc8sQDHvo P+suQfkJLUSDBOB3DnqJ040ZJJb1O1pmWyvDCU0G1YdLlFBfsZOIBQqaWQ4YKQJSFOAu aCngDoy3xheshCIbekIxw9rxzjsuOqkOrIA7IOovznrhqcMbDnPClKiHdIs/KMZfwcpy uJ1Mws7Md45eVV7SM/CeVt3vlU6A0soa2KGrxbmIgeihkpShj4Ko34iMyHMHSi6kVYgg SFXQ== MIME-Version: 1.0 X-Received: by 10.180.9.106 with SMTP id y10mr52823522wia.30.1415831475826; Wed, 12 Nov 2014 14:31:15 -0800 (PST) Received: by 10.27.11.42 with HTTP; Wed, 12 Nov 2014 14:31:15 -0800 (PST) In-Reply-To: <49cc2695179830c899aef87b7a8b72d8@openmailbox.org> References: <8e39523570b667d223091adc3791c9e2@openmailbox.org> <5463D746.4030803@freebsd.org> <49cc2695179830c899aef87b7a8b72d8@openmailbox.org> Date: Wed, 12 Nov 2014 14:31:15 -0800 Message-ID: Subject: Re: How hard is BHyVe's 16 vCPU limit, is it configurable under any circumstance? From: Neel Natu To: Tinker Content-Type: text/plain; charset=UTF-8 Cc: owner-freebsd-virtualization@freebsd.org, "freebsd-virtualization@freebsd.org" , Allan Jude X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 22:31:18 -0000 Hi Tinker, On Wed, Nov 12, 2014 at 2:08 PM, Tinker wrote: > On 2014-11-12 23:55, Allan Jude wrote: >> >> On 2014-11-12 16:39, tinkr@openmailbox.org wrote: >>> >>> Hi! >>> >>> In order justify giving energy to BHyVe, I need to know if it's >>> "future-proof" in that the 16 vCPU limit can be increased? >>> >>> Please let the world know if BHyVe's 16 vCPU limit can be lifted in some >>> way, by configuration, patch, etc. (and if you want to, why this limit >>> is in place by default today). >>> >>> Thanks!! >>> Tinker >>> >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to >>> "freebsd-virtualization-unsubscribe@freebsd.org" >> >> >> You can increase the limit by editing sys/amd64/include/vmm.h >> >> #define VM_MAXCPU 16 >> >> From what I've been told, things scale badly above 24 CPUs. They plan to >> solve this issue, but have not yet. If you system has enough cores to >> support using more than 16 per VM, you can modify the file and recompile >> the kernel and use as many CPUs as you want, but not much testing has >> happened with bigger numbers. > > > Dear Allan, > > Thank you very much for responding. > > Aha, great! > > > Only for completeness, do you have any particular idea about > * When the scaling above 24 vCPU:s will be optimized, like approx how many > years away is it (like 1 or more than 1)?, and bhyve allocates memory for the maximum number of vcpus when the virtual machine is created. This is simple to implement and also fits in nicely with bhyve's loader model where the guest loader (bhyveload/grub-bhyve) is separate from bhyve. The downside is that if VM_MAXCPU is a large number then there is a large memory penalty for virtual machines that only use 1 or 2 vcpus. Hence the desire to cap it at a smallish number. There are a few ways to deal with this: - patch the code to change VM_MAXCPU (this is what you need to do today) - allow the maximum vcpus to be changed via a tunable (this can be done in short order) - the limit can go away when bhyve moves to a single process model because we'll know the number of vcpus at VM creation time. > * What the technological reason for the scaling is, is it that somehow the > BHyVe instances on the different cores need to inter-communicate, for > instance that all disk and network IO is done via one single core currently? > Actually, the performance depends more on the workload and the guest OS so you should definitely try how it goes for your application. We'll be happy to help fix any issues that come up. best Neel > > In all cases, your response is great news, as your baseline answer that it's > doable and only a question of optimization and tweaking of present > functionality - > > Thanks again! > > Tinker > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 23:46:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73484E14 for ; Wed, 12 Nov 2014 23:46:36 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 4A3E1C2F for ; Wed, 12 Nov 2014 23:46:35 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id DD4AD6EB4C for ; Wed, 12 Nov 2014 23:46:34 +0000 (UTC) Message-ID: <5463F15C.6070103@freebsd.org> Date: Wed, 12 Nov 2014 18:46:36 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: How hard is BHyVe's 16 vCPU limit, is it configurable under any circumstance? References: <8e39523570b667d223091adc3791c9e2@openmailbox.org> <5463D746.4030803@freebsd.org> <49cc2695179830c899aef87b7a8b72d8@openmailbox.org> In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PRKRNQXt4xFtxCCkBG9Dd8TkXAFHG1FXR" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 23:46:36 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --PRKRNQXt4xFtxCCkBG9Dd8TkXAFHG1FXR Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2014-11-12 17:31, Neel Natu wrote: > Hi Tinker, >=20 > On Wed, Nov 12, 2014 at 2:08 PM, Tinker wrote: >> On 2014-11-12 23:55, Allan Jude wrote: >>> >>> On 2014-11-12 16:39, tinkr@openmailbox.org wrote: >>>> >>>> Hi! >>>> >>>> In order justify giving energy to BHyVe, I need to know if it's >>>> "future-proof" in that the 16 vCPU limit can be increased? >>>> >>>> Please let the world know if BHyVe's 16 vCPU limit can be lifted in = some >>>> way, by configuration, patch, etc. (and if you want to, why this lim= it >>>> is in place by default today). >>>> >>>> Thanks!! >>>> Tinker >>>> >>>> _______________________________________________ >>>> freebsd-virtualization@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>>> To unsubscribe, send any mail to >>>> "freebsd-virtualization-unsubscribe@freebsd.org" >>> >>> >>> You can increase the limit by editing sys/amd64/include/vmm.h >>> >>> #define VM_MAXCPU 16 >>> >>> From what I've been told, things scale badly above 24 CPUs. They plan= to >>> solve this issue, but have not yet. If you system has enough cores to= >>> support using more than 16 per VM, you can modify the file and recomp= ile >>> the kernel and use as many CPUs as you want, but not much testing has= >>> happened with bigger numbers. >> >> >> Dear Allan, >> >> Thank you very much for responding. >> >> Aha, great! >> >> >> Only for completeness, do you have any particular idea about >> * When the scaling above 24 vCPU:s will be optimized, like approx how= many >> years away is it (like 1 or more than 1)?, and >=20 > bhyve allocates memory for the maximum number of vcpus when the > virtual machine is created. This is simple to implement and also fits > in nicely with bhyve's loader model where the guest loader > (bhyveload/grub-bhyve) is separate from bhyve. >=20 > The downside is that if VM_MAXCPU is a large number then there is a > large memory penalty for virtual machines that only use 1 or 2 vcpus. > Hence the desire to cap it at a smallish number. >=20 > There are a few ways to deal with this: > - patch the code to change VM_MAXCPU (this is what you need to do today= ) > - allow the maximum vcpus to be changed via a tunable (this can be > done in short order) > - the limit can go away when bhyve moves to a single process model > because we'll know the number of vcpus at VM creation time. >=20 >> * What the technological reason for the scaling is, is it that someho= w the >> BHyVe instances on the different cores need to inter-communicate, for >> instance that all disk and network IO is done via one single core curr= ently? >> >=20 > Actually, the performance depends more on the workload and the guest > OS so you should definitely try how it goes for your application. > We'll be happy to help fix any issues that come up. >=20 > best > Neel >=20 >> >> In all cases, your response is great news, as your baseline answer tha= t it's >> doable and only a question of optimization and tweaking of present >> functionality - >> >> Thanks again! >> >> Tinker >> Right, I forgot that at MeetBSD it was decided that making the max cpus a sysctl was a good stop-gap measure to hold us over until the redesign of bhyveload. --=20 Allan Jude --PRKRNQXt4xFtxCCkBG9Dd8TkXAFHG1FXR Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUY/FlAAoJEJrBFpNRJZKfhAoP/i1Ed8s13RRjI6AcLmsZYDy1 nCk7Sz6dk1OwmZoBJqO5jEGUZbRpmBSv6Df0MGsdKvPfog5t0D6MTo91T2z0e1ma xadZw4EwiIbm7ZeLXRNjOyhvYOrqga6T4lKENxJ7QvPKVy90nMi9j8xIb8OAVHO+ dGKQrFrhsYBnhUQp63y7pwTc2DM5RwRysEiXVtDFx9Ofdj7Eochg2PTCdiwRzUe9 dERdzMf89xdqVd7+4fonTvVSOOZ2PG8JzoJri2f05x7CPkl8KerCx8l//ARxDOHk Pu9dBQIoOlgLvF4tRDzBL1bwqV073Lx8ZsPgWpolT/9BjqAVXWiof4AiNXQ+vAMd M5+YvncLEQYgP4h4pvppx0akZoW1nIxE7M0vkqaVOyx1AntTFvJFmitEOlO1ZILX 2qwATN1JlZwm0DK2zX9XV7t4xEdYo09ONU4T6MSVmP3P73zaXOcAliQBk8NfSVBw qOScZcPLGnpaAMHbDzk+yg23HX43ZmKKqJJkqAE/gNJyiC9tMRmlFk59CxAhaM/2 FzrRCoj0sOMGgWe+JvkOTituv3WECdDiwkfGRZyDjQYBuiDFfwoGB1icysiUo9+O xyQv1vmM9Xfe+RO68dopi5P8b8zgVT/b29HQ3OlUmO+jQ0gxQzGusRlY4ZVgOLSp TqsNefle7dcJ9rCJ8ySS =jYFN -----END PGP SIGNATURE----- --PRKRNQXt4xFtxCCkBG9Dd8TkXAFHG1FXR-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Nov 14 01:33:18 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25ACC293 for ; Fri, 14 Nov 2014 01:33:18 +0000 (UTC) Received: from mail1.bur200.uecomm.net.au (mail1.bur200.uecomm.net.au [218.185.0.70]) by mx1.freebsd.org (Postfix) with ESMTP id D6F6675B for ; Fri, 14 Nov 2014 01:33:16 +0000 (UTC) Received: from mail.flexibledrive.com.au (unknown [115.186.196.106]) by mail1.bur200.uecomm.net.au (Postfix) with ESMTP id E658CD528 for ; Fri, 14 Nov 2014 12:33:06 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.flexibledrive.com.au (Postfix) with ESMTP id 81AD3E6B7B for ; Fri, 14 Nov 2014 12:33:06 +1100 (EST) X-Virus-Scanned: amavisd-new at fdrive.com.au Received: from mail.flexibledrive.com.au ([127.0.0.1]) by localhost (mail.flexibledrive.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c+mZkDu5dZMD for ; Fri, 14 Nov 2014 12:32:58 +1100 (EST) Received: from ws-pross.vv.fda (ws-pross.vv.fda [192.168.50.199]) by mail.flexibledrive.com.au (Postfix) with ESMTPS id 1DAEBE6B78 for ; Fri, 14 Nov 2014 12:32:58 +1100 (EST) Date: Fri, 14 Nov 2014 12:32:57 +1100 (AEDT) From: Peter Ross X-X-Sender: petros@linux-vic-05.vv.fda To: freebsd-virtualization@freebsd.org Subject: VirtualBox performance Message-ID: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2014 01:33:18 -0000 Hi all, I am running VirtualBox on a new server (a Dell T620). Inside is Oracle Linux. I run a Java app and it is very slow to start. Memory should not be an issue. I have 32 GB and 8 GB for the VM. The server has 2 Xeon CPUs with 4 core each (hypertthreated so VBox sees 16 CPUs). I do not see any difference between using 1 CPU or 16 for the VM.. Interestingly all the host CPUs are 60-70% in system calls when I run it on 16 CPUs, according to top. I am tempted to experiment with CPU sets, btw. I wonder whether VirtualBox and multiple CPUs are a good mix at all.. Do you have any recommendations how to speed up my VM and Java? Thank you Peter From owner-freebsd-virtualization@FreeBSD.ORG Fri Nov 14 19:06:23 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D25F1D03 for ; Fri, 14 Nov 2014 19:06:23 +0000 (UTC) Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 926C01E0 for ; Fri, 14 Nov 2014 19:06:23 +0000 (UTC) Received: by mail-oi0-f52.google.com with SMTP id u20so12214575oif.25 for ; Fri, 14 Nov 2014 11:06:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K2t9cgOF+oD/fv67o2d/rth2H010mWSxF/op5wvf66I=; b=Ew+xFQ7V/hzoCQlbA/f3MGjlEi4ZKEoyxdR3CO4xaxriOlDWO8ef7A16M6MkgsokX/ 57HxBowzSbjsYFqb3C7Oq6Dydhbc8aktxD6WRpzj07OG/t6nDY1hWUvtDV+Ud/OjS8wW sSLH9misLLNnD//kBRLgXJWc5PEF/Kx0Dq33bgVIUoOuHvd+bD5J2mj/RefbD1PBTBHP IKvy0IFK17HhoFtmcH/qP5n+9y/ybcCYV5B09opriZUDnQbt13wkdQKkiM+cwDeuHFha vTg9D4InGUBtSpNcju3eLV3EJVdRJFRXJ9kGFc4PFTwdm6Ln8Lpi2bMV5W5n8zevraOZ Y+dw== MIME-Version: 1.0 X-Received: by 10.202.76.210 with SMTP id z201mr121708oia.123.1415991982763; Fri, 14 Nov 2014 11:06:22 -0800 (PST) Received: by 10.182.74.5 with HTTP; Fri, 14 Nov 2014 11:06:22 -0800 (PST) Received: by 10.182.74.5 with HTTP; Fri, 14 Nov 2014 11:06:22 -0800 (PST) In-Reply-To: References: Date: Fri, 14 Nov 2014 23:06:22 +0400 Message-ID: Subject: Re: VirtualBox performance From: Mikhail Tsatsenko To: Peter Ross Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2014 19:06:24 -0000 14.11.2014 4:33 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82=D0= =B5=D0=BB=D1=8C "Peter Ross" =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB: > > Hi all, > > I am running VirtualBox on a new server (a Dell T620). > > Inside is Oracle Linux. I run a Java app and it is very slow to start. Hi, In order to utilize multiple CPUs inside virtual machine you have to enable io-apic setting. > Memory should not be an issue. I have 32 GB and 8 GB for the VM. > > The server has 2 Xeon CPUs with 4 core each (hypertthreated so VBox sees 16 CPUs). > > I do not see any difference between using 1 CPU or 16 for the VM.. > > Interestingly all the host CPUs are 60-70% in system calls when I run it on 16 CPUs, according to top. > > I am tempted to experiment with CPU sets, btw. > > I wonder whether VirtualBox and multiple CPUs are a good mix at all.. > > Do you have any recommendations how to speed up my VM and Java? > > Thank you > Peter > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 09:37:15 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0D39A02 for ; Sat, 15 Nov 2014 09:37:15 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE2FFF2 for ; Sat, 15 Nov 2014 09:37:15 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sAF9bDgE073584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 15 Nov 2014 01:37:14 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sAF9bDob073583 for freebsd-virtualization@freebsd.org; Sat, 15 Nov 2014 01:37:13 -0800 (PST) (envelope-from jmg) Date: Sat, 15 Nov 2014 01:37:13 -0800 From: John-Mark Gurney To: freebsd-virtualization@freebsd.org Subject: bhyve remote kgdb does not support kernel modules... Message-ID: <20141115093713.GY24601@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 15 Nov 2014 01:37:14 -0800 (PST) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 09:37:15 -0000 So, I got remote kgdb working w/ bhyve, but kernel modules aren't loaded... I'm getting: (kgdb) bt #0 kdb_enter (why=0xffffffff8102ee88 "panic", msg=) at ../../../kern/subr_kdb.c:444 #1 0xffffffff8094aa79 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:739 #2 0xffffffff8094a8c9 in kassert_panic (fmt=) at ../../../kern/kern_shutdown.c:634 #3 0xffffffff8093583c in __mtx_lock_flags (c=0xfffff80002919720, opts=0, file=0xffffffff81c3f0fa "/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c", line=442) at ../../../kern/kern_mutex.c:217 #4 0xffffffff81c3e3a7 in ?? () #5 0xfffff80002a474a0 in ?? () #6 0xfffff80002919720 in ?? () Where those should be in the aesni.ko module... If I run kgdb on a local machine, it properly finds the kernel modules... Any hits on how to fix this? Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 09:43:42 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6AE7C02 for ; Sat, 15 Nov 2014 09:43:42 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 78B95100 for ; Sat, 15 Nov 2014 09:43:41 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sAF9hfNx073648 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 15 Nov 2014 01:43:41 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sAF9heoB073647 for freebsd-virtualization@freebsd.org; Sat, 15 Nov 2014 01:43:40 -0800 (PST) (envelope-from jmg) Date: Sat, 15 Nov 2014 01:43:40 -0800 From: John-Mark Gurney To: freebsd-virtualization@freebsd.org Subject: Re: bhyve remote kgdb does not support kernel modules... Message-ID: <20141115094340.GZ24601@funkthat.com> References: <20141115093713.GY24601@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141115093713.GY24601@funkthat.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 15 Nov 2014 01:43:41 -0800 (PST) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 09:43:42 -0000 John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800: > So, I got remote kgdb working w/ bhyve, but kernel modules aren't > loaded... > > I'm getting: > (kgdb) bt > #0 kdb_enter (why=0xffffffff8102ee88 "panic", msg=) > at ../../../kern/subr_kdb.c:444 > #1 0xffffffff8094aa79 in vpanic (fmt=, > ap=) at ../../../kern/kern_shutdown.c:739 > #2 0xffffffff8094a8c9 in kassert_panic (fmt=) > at ../../../kern/kern_shutdown.c:634 > #3 0xffffffff8093583c in __mtx_lock_flags (c=0xfffff80002919720, opts=0, > file=0xffffffff81c3f0fa "/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c", line=442) at ../../../kern/kern_mutex.c:217 > #4 0xffffffff81c3e3a7 in ?? () > #5 0xfffff80002a474a0 in ?? () > #6 0xfffff80002919720 in ?? () > > Where those should be in the aesni.ko module... If I run kgdb on a > local machine, it properly finds the kernel modules... Any hits on > how to fix this? Also, any plans to support the Z2 write-watchpoint packet? (kgdb) watch *(char *)0xfffff80002919708 Hardware watchpoint 1: *(char *) 18446735277659625224 (kgdb) c Continuing. Can't clear hardware watchpoints without the 'Z2' (write-watchpoint) packet -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 16:13:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 753FEDDB for ; Sat, 15 Nov 2014 16:13:00 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D2809C2 for ; Sat, 15 Nov 2014 16:13:00 +0000 (UTC) Received: from portis.head.local ([109.193.238.174]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MQiVh-1XPzig2g7Z-00U0AA; Sat, 15 Nov 2014 17:12:52 +0100 Message-ID: <54677B7B.4080202@gmx.com> Date: Sat, 15 Nov 2014 17:12:43 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Peter Ross , freebsd-virtualization@freebsd.org Subject: Re: VIMAGE and VirtualBox networking question References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:CsL2nKhB5NwA+cWGJ4Krm7o6DHzj2xWUhd2GgflF3AL/88xTHaJ AWGwhUdxx9rfVh2cmuzhlFyfxHFPv24CH0MXq3wQJz+ws8jMTzUfPgWaC1O3kELOS2U3bah YiPBJLrFVj2lbW4/mWzLXdTZskGD5m8N3iI3MAZFsTm3VuDhkSJEQ7wUC2xJTw5vHQ+X/4+ k15a8S96HK+GfWZDruxVw== X-UI-Out-Filterresults: notjunk:1; X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 16:13:00 -0000 On 11/11/14 00:39, Peter Ross wrote: > Nikos Vassiliadis wrote: > >> On 11/05/14 00:29, Manas Bhatnagar wrote: > >>> Install ports tree in a temporary location (another computer or >>> something), grab the required folder and move it to the computer that >>> you want to install the port on and do 'make install clean' ? >>> probably won't work like that, you would need to make sure the >>> directories for dependencies are there too :o) > >> Tried to do this with a full ports tree and I stopped it after gcc >> started to build. > > BTW: You need /usr/src available for the creation of the package. > Preferably matching the kernel you want to install it.. > > I do this regularly (but without VIMAGE) in a jail where I build all my > packages. ("make install" and "pkg create" in a script) > > I do all updates from source. I only update kernel, user land and ports > when security is at risk. Usually independent from each other. > > But VirtualBox is an exception. I update kernel and VirtualBox at the > same time. > > I was not brave enough to enable VIMAGE and VirtualBox on the same > server. But I may do that soon. I plan a major reconfiguration ca. end > of the year. I hope it is stable enough. Hi, Just want to add that everything works as expected with VIMAGE and VBox. From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 16:23:40 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F187DFB5 for ; Sat, 15 Nov 2014 16:23:40 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C4B7BA85 for ; Sat, 15 Nov 2014 16:23:40 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-239-104.lns20.per1.internode.on.net [121.45.239.104]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sAFGNT7q075045 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 15 Nov 2014 08:23:32 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54677DFC.70708@freebsd.org> Date: Sun, 16 Nov 2014 00:23:24 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John-Mark Gurney , freebsd-virtualization@freebsd.org Subject: Re: bhyve remote kgdb does not support kernel modules... References: <20141115093713.GY24601@funkthat.com> <20141115094340.GZ24601@funkthat.com> In-Reply-To: <20141115094340.GZ24601@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 16:23:41 -0000 On 11/15/14, 5:43 PM, John-Mark Gurney wrote: > John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800: >> So, I got remote kgdb working w/ bhyve, but kernel modules aren't >> loaded... >> >> I'm getting: >> (kgdb) bt >> #0 kdb_enter (why=0xffffffff8102ee88 "panic", msg=) >> at ../../../kern/subr_kdb.c:444 >> #1 0xffffffff8094aa79 in vpanic (fmt=, >> ap=) at ../../../kern/kern_shutdown.c:739 >> #2 0xffffffff8094a8c9 in kassert_panic (fmt=) >> at ../../../kern/kern_shutdown.c:634 >> #3 0xffffffff8093583c in __mtx_lock_flags (c=0xfffff80002919720, opts=0, >> file=0xffffffff81c3f0fa "/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c", line=442) at ../../../kern/kern_mutex.c:217 >> #4 0xffffffff81c3e3a7 in ?? () >> #5 0xfffff80002a474a0 in ?? () >> #6 0xfffff80002919720 in ?? () >> >> Where those should be in the aesni.ko module... If I run kgdb on a >> local machine, it properly finds the kernel modules... Any hits on >> how to fix this? works for me.. the modules I'm debugging are not compiled in. you have to have the modules in the standard place ONE THE GDB HOST. I make a chroot with an image of the machine being debugged, PLUS the sources. and then run kgdb from inside that chroot. > Also, any plans to support the Z2 write-watchpoint packet? > > (kgdb) watch *(char *)0xfffff80002919708 > Hardware watchpoint 1: *(char *) 18446735277659625224 > (kgdb) c > Continuing. > Can't clear hardware watchpoints without the 'Z2' (write-watchpoint) packet > From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 18:15:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4F6F67F; Sat, 15 Nov 2014 18:15:25 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A2B1467C; Sat, 15 Nov 2014 18:15:25 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sAFIFNgk095030 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 15 Nov 2014 10:15:23 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sAFIFNtO095029; Sat, 15 Nov 2014 10:15:23 -0800 (PST) (envelope-from jmg) Date: Sat, 15 Nov 2014 10:15:23 -0800 From: John-Mark Gurney To: Julian Elischer Subject: Re: bhyve remote kgdb does not support kernel modules... Message-ID: <20141115181523.GA24601@funkthat.com> References: <20141115093713.GY24601@funkthat.com> <20141115094340.GZ24601@funkthat.com> <54677DFC.70708@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54677DFC.70708@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 15 Nov 2014 10:15:24 -0800 (PST) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 18:15:26 -0000 Julian Elischer wrote this message on Sun, Nov 16, 2014 at 00:23 +0800: > On 11/15/14, 5:43 PM, John-Mark Gurney wrote: > >John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800: > >>So, I got remote kgdb working w/ bhyve, but kernel modules aren't > >>loaded... > >> > >>I'm getting: > >>(kgdb) bt > >>#0 kdb_enter (why=0xffffffff8102ee88 "panic", msg=) > >> at ../../../kern/subr_kdb.c:444 > >>#1 0xffffffff8094aa79 in vpanic (fmt=, > >> ap=) at ../../../kern/kern_shutdown.c:739 > >>#2 0xffffffff8094a8c9 in kassert_panic (fmt=) > >> at ../../../kern/kern_shutdown.c:634 > >>#3 0xffffffff8093583c in __mtx_lock_flags (c=0xfffff80002919720, opts=0, > >> file=0xffffffff81c3f0fa > >> "/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c", line=442) at ../../../kern/kern_mutex.c:217 > >>#4 0xffffffff81c3e3a7 in ?? () > >>#5 0xfffff80002a474a0 in ?? () > >>#6 0xfffff80002919720 in ?? () > >> > >>Where those should be in the aesni.ko module... If I run kgdb on a > >>local machine, it properly finds the kernel modules... Any hits on > >>how to fix this? > > works for me.. > the modules I'm debugging are not compiled in. > you have to have the modules in the standard place ONE THE GDB HOST. > I make a chroot with an image of the machine being debugged, PLUS the > sources. > and then run kgdb from inside that chroot. Ahh, that's right, kgdb isn't smart enough to know when you've pointed it to the kernel build dir whereh the modules are... :( Thanks for the reminder... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 20:58:25 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C71F3C3 for ; Sat, 15 Nov 2014 20:58:25 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D06B766 for ; Sat, 15 Nov 2014 20:58:24 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id D490B1534C0 for ; Sat, 15 Nov 2014 21:58:12 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IN8wLajxdxNO; Sat, 15 Nov 2014 21:57:54 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:806:2b43:477a:23d] (unknown [IPv6:2001:4cb8:3:1:806:2b43:477a:23d]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 57CE215340A for ; Sat, 15 Nov 2014 21:49:12 +0100 (CET) Message-ID: <5467BC49.4080308@digiware.nl> Date: Sat, 15 Nov 2014 21:49:13 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "freebsd-virtualization@freebsd.org" Subject: Trying to run an older head in a recent Bhyve. Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 20:58:25 -0000 Hi, I'm trying to run one of my older VM's and get the crash below in the VM when trying to boot... This happens both on an older BSD: FreeBSD 11.0-CURRENT (FREETEST) #1 r273066M: Sun Oct 19 00:59:06 CEST 2014 As well as on a very recent: FreeBSD 11.0-CURRENT (BHYVE00) #0 r274490M: Fri Nov 14 02:42:43 CET 2014 The older 10.0 VM's do boot normally Any suggestions on what this might be, and/or how to debug this.. --WjW DHCPDISCOVER on vtnet0 to 255.255.255.255 port 67 interval 3 DHCPOFFER from 192.168.10.75 DHCPREQUEST on vtnet0 to 255.255.255.255 port 67 DHCPACK from 192.168.10.75 panic: ffs_write: type 0xfffff8000565dce8 0 (0,62) cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00975374d0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0097537580 vpanic() at vpanic+0x126/frame 0xfffffe00975375c0 panic() at panic+0x43/frame 0xfffffe0097537620 ffs_write() at ffs_write+0x68b/frame 0xfffffe00975376c0 VOP_WRITE_APV() at VOP_WRITE_APV+0x16f/frame 0xfffffe00975377d0 vn_write() at vn_write+0x2f3/frame 0xfffffe0097537860 vn_io_fault() at vn_io_fault+0x17b/frame 0xfffffe00975379f0 dofilewrite() at dofilewrite+0x88/frame 0xfffffe0097537a40 kern_writev() at kern_writev+0x68/frame 0xfffffe0097537a90 sys_write() at sys_write+0x63/frame 0xfffffe0097537ae0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe0097537bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0097537bf0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x800de9b4a, rsp = 0x7fffffffd4c8, rbp = 0x7fffffffd510 --- KDB: enter: panic [ thread pid 551 tid 100057 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why From owner-freebsd-virtualization@FreeBSD.ORG Sat Nov 15 23:59:41 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A081A89 for ; Sat, 15 Nov 2014 23:59:41 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4CA9A8FC for ; Sat, 15 Nov 2014 23:59:41 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 0334F125BE; Sun, 16 Nov 2014 09:59:39 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id BZT48479 (AUTH peterg@ptree32.com.au); Sun, 16 Nov 2014 09:59:37 +1000 Message-ID: <5467E8E6.1000509@freebsd.org> Date: Sat, 15 Nov 2014 15:59:34 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Willem Jan Withagen , "freebsd-virtualization@freebsd.org" Subject: Re: Trying to run an older head in a recent Bhyve. References: <5467BC49.4080308@digiware.nl> In-Reply-To: <5467BC49.4080308@digiware.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2014 23:59:41 -0000 Hi Willem, > I'm trying to run one of my older VM's and get the crash below in the VM > when trying to boot... > > This happens both on an older BSD: > FreeBSD 11.0-CURRENT (FREETEST) #1 r273066M: Sun Oct 19 00:59:06 CEST 2014 > As well as on a very recent: > FreeBSD 11.0-CURRENT (BHYVE00) #0 r274490M: Fri Nov 14 02:42:43 CET 2014 > > The older 10.0 VM's do boot normally > > Any suggestions on what this might be, and/or how to debug this.. Did the disk image backing file change by any chance ? e.g. from file to zvol ? What's the version of the VM that has the issue ? later, Peter.