From owner-freebsd-virtualization@freebsd.org Sun Sep 3 03:52:00 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A1B7E17587 for ; Sun, 3 Sep 2017 03:52:00 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-yw0-x230.google.com (mail-yw0-x230.google.com [IPv6:2607:f8b0:4002:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D072E80A2F for ; Sun, 3 Sep 2017 03:51:59 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-yw0-x230.google.com with SMTP id c85so8338557ywa.0 for ; Sat, 02 Sep 2017 20:51:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=NwLxzMevKiRzf84drB1Wi/qjmi2KjP7PoQvqgQ+ezRE=; b=PI2/Op4WysWCyFhDTHgFxj8q9SAjV2Dj1BpSLLOlgBPBiVqODqt1tqR7Yhw4K9lXWg LcXPAgNTGdL2C+ho7MEX6DdREEtLKq0SugyIeAENRxh2dkFRxLf4ONRSeT1vQuW6o3/z wO0RFZXWSGwPPWzBbcYHP0fLkj2S+z4i11hRRfre+ArI3/SKz2VL2CpRwl+cQ0LOratr PLBYQH80uIwXyryq6vFcXc1ewsVXCIC1ZbuiA14BNkOdypFbDZDTD9ijZiF10tJJ7dXd qsEjropHGB+miFz1WASjQFl+wKPvvXjElPm1NOEeXjx/kHayB3Ku1/zD1SAoUHhxe5Er /0fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=NwLxzMevKiRzf84drB1Wi/qjmi2KjP7PoQvqgQ+ezRE=; b=Cx0HF1RgTDyiT/HAfX1nmve/qoQubauSl6h4prPdkvtDhjaSFA4FOajHokxJekEUwI jqME4ihmoy39Oo7CW8Ak9wEogyK5TCvB0HphNwsnVzLDv0YWrw+4wha3ivQH9BpECFeH BiCKshYzaV4ZvJG2c+n7fg9qHZQ4eEcEo3kFKj0GIpARxI61JOUs5IHj1fPzODJ11foS TC7GU3/mh6fWhbGzIrhT/fxNdxNxd0UTRZuuZUIVbMuZEauF0kPMe1Dbv0TEvCb3KzCY A1X6zJXVe1+tj0J6Bt98HaRYbzyUIC3fqfl9s6CFx3ooKWHTFq0OPdGjhAOHr7sbgaID EF0g== X-Gm-Message-State: AHPjjUjVE12fdC6yLBan5Nq+PQ4+3VHiLE76IYasUlrAiz/HW6bp6mVO MDn2mlQY9061i1txsWkeBXelX+Y5QPaq X-Google-Smtp-Source: ADKCNb5oUkshyfHGCThXFuOerCrvHDKksROPETUGgeTSFTXJ7qnZmeatuNDX9lMjUf+0f1QGMoJQgZq5ZWeq7s9oV5g= X-Received: by 10.37.224.151 with SMTP id x145mr3078000ybg.74.1504410718506; Sat, 02 Sep 2017 20:51:58 -0700 (PDT) MIME-Version: 1.0 Sender: jtubnor@gmail.com Received: by 10.37.198.10 with HTTP; Sat, 2 Sep 2017 20:51:38 -0700 (PDT) In-Reply-To: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> References: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> From: Jason Tubnor Date: Sun, 3 Sep 2017 13:51:38 +1000 X-Google-Sender-Auth: GklOG4YiV8egVGCKISv91MhoLUI Message-ID: Subject: Re: bhyve VM drive size limit? To: Ruben Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 03:52:00 -0000 Hi Ruben, On 3 September 2017 at 00:52, Ruben wrote: > > Perhaps slightly off-topic, but might you have any recommendations > regarding those blocksizes? > I use 8k zvol block sizes. This seems to work consistently for me with OpenBSD and Windows Server 2016 guests. While you could probably go larger, it really depends on your use case and how the data intends to move around in the guest. Some people have had performance issues with 8k blocks, however, I am yet to experience this. Chyves by default used 512 byte block sizes, this is sub-optimal for a guest with > 9GB storage needs, so I change the default settings back to 8k. > > I'm not really that well versed in blocksizes / filesystems and am often > wondering where my 5TB zvol (bhyves into an ext4 volume for an ubuntu > guest) exactly disappears into :P For the above, I have experienced where block sizes were defined at 512 bytes, the associated block checksums would eat up the storage, just as quick as the data being committed to the zvol (even more so as the snapshots pile up). Increasing the zvol block size increased the storage to checksum ratio, reducing how much the checksum overhead had an affect on the storage. Keep in mind, this is with a zmirror, so mileage may vary with raidz or raidz2 setups. Another thing to keep in mind, as you increase the block size, this can have performance impacts on the guest as a 4k update in the guest filesystem, would but upwards pressure on array performance if you used say 32-64k blocks (rewriting > 32x the data+checksum calcs). I suggest trying different sizes for the same workload you plan to commit to the guest storage to see which has the best storage to performance ratio. Cheers, Jason. P.S. In the above, I wouldn't commit the 5TB to the guest in one hit. Either grow it out as you need it or add additional zvols to your guest as needed. From owner-freebsd-virtualization@freebsd.org Sun Sep 3 04:01:54 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AE83E17CEF for ; Sun, 3 Sep 2017 04:01:54 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2BB0E80D08 for ; Sun, 3 Sep 2017 04:01:53 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id CDC4FDBC1 for ; Sun, 3 Sep 2017 04:01:46 +0000 (UTC) Subject: Re: bhyve VM drive size limit? To: freebsd-virtualization@freebsd.org References: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> From: Allan Jude Message-ID: Date: Sun, 3 Sep 2017 00:01:42 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cdlRI4s7lurQFlu8FEL371TCrc27otbX8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 04:01:54 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --cdlRI4s7lurQFlu8FEL371TCrc27otbX8 Content-Type: multipart/mixed; boundary="FdXbGfijjLjrD5wJswmKAO9CwsvTUaQ31"; protected-headers="v1" From: Allan Jude To: freebsd-virtualization@freebsd.org Message-ID: Subject: Re: bhyve VM drive size limit? References: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> In-Reply-To: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> --FdXbGfijjLjrD5wJswmKAO9CwsvTUaQ31 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2017-09-02 10:52, Ruben wrote: > Hi Jason, >=20 >=20 >> Make sure >> you adjust the zvol block size in the default config before creating y= our >> guest or you will wonder where your storage starts to disappear to. >> >> Cheers, >> >> Jason > Perhaps slightly off-topic, but might you have any recommendations > regarding those blocksizes? >=20 > I'm not really that well versed in blocksizes / filesystems and am ofte= n > wondering where my 5TB zvol (bhyves into an ext4 volume for an ubuntu > guest) exactly disappears into :P >=20 > Regards, >=20 > Ruben >> On 2 Sep 2017 2:46 p.m., "Dylan Williams" wrote:= >> >> Dear all, >> >> Is there a limit with respect to the size of the virtual drive I can c= reate >> for a VM using bhyve? Presumably not but I=E2=80=99m running into prob= lems with a >> larger drive. >> >> I am trying to create a 4T Ubuntu machine on a FreeBSD11.1 server usin= g >> iohyve. There=E2=80=99s plenty of drive space available so it=E2=80=99= s not a drive space >> issue. I am able to create a 40G machine without any problems but exac= tly >> the same commands to create a 4000G machine results in a server that w= on=E2=80=99t >> boot past Grub. Is this an iohyve issue perhaps? >> >> Commands used that results in a working Ubuntu server: >> >> iohyve create ubu 40G >> iohyve set ubu loader=3Dgrub-bhyve os=3Ddebian ram=3D40G cpu=3D10 >> iohyve install ubu ubuntu-16.04.3-server-amd64.iso < >> ftp://ftp.cuhk.edu.hk/pub/Linux/ubuntu-releases/16.04.3/ >> ubuntu-16.04.3-server-amd64.iso> >> >> Commands used that results in a broken Ubuntu server: >> >> iohyve create ubu 4000G >> iohyve set ubu loader=3Dgrub-bhyve os=3Ddebian ram=3D40G cpu=3D10 >> iohyve install ubu ubuntu-16.04.3-server-amd64.iso < >> ftp://ftp.cuhk.edu.hk/pub/Linux/ubuntu-releases/16.04.3/ >> ubuntu-16.04.3-server-amd64.iso> >> >> The screen I get in the console when I boot (iohyve start ubu) the 4T >> machine is: >> >> GNU GRUB version 2.00 >> >> Minimal BASH-like line editing is supported. For the first word, TA= B >> lists possible command completions. Anywhere else TAB lists possibl= e >> device or file completions. >> >> >> grub> >> >> Does anyone have any ideas? I=E2=80=99ve been at this for hours trying= different >> configurations. LVM is not selected when installing either of the abov= e >> machines as that seems to cause more problems. >> >> Many thanks, >> Dylan. >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization- >> unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@f= reebsd.org" > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 To understand why space in a ZVOL goes missing, research 'raidz padding'.= See here: https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-o= r-how-i-learned-stop-worrying-and-love-raidz For more detail, see Chapter 9 of "FreeBSD Mastery: Advanced ZFS" from http://www.zfsbook.com --=20 Allan Jude --FdXbGfijjLjrD5wJswmKAO9CwsvTUaQ31-- --cdlRI4s7lurQFlu8FEL371TCrc27otbX8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJZq36pAAoJEBmVNT4SmAt+yiwP/iaymOs2ChQLcWcYFSR4E1kT QWeLT1kFF3YGNSPPzHxwaVjr/qdrCBjMdM2swHUx9UTiUgV8u24KT6Lt+IVrgRzN xlBA2T5JAk5LY8Vox+5eh2OH3P0oFV/62OExmzETK2ul3tCxrmRVynxT7pOzgnfp L3kD1XAggI8SugdlC/Wtg63nxeqE3asGs1M5zyWE+vVoYCtZEiw7Pu8vpDes/hPJ EVEdr4oa7OdTt+jLT9VHCLRqwTQ7BXUsHvwzLj13+DD5qpyVQqyrFtOpVJhhXjcX OPzrSesGiLky98KgEOwKqbls1X2fY5cNPvWXOGVluP4PYpNyNdeCcv/tI0LjfXTB IT1ffT9upvDXYLdW9Fb7MuZbqRKkZ+Hj0438jEbzEpWUu/yWwQ+5G6EVqqFFx8c3 XKs+MV+515LwwN6dGOqAjOY6mE0Le+xi5zkgYLc9XsKe5B82TdMgMXHu+bPd4QQC iJ/imkSJbE0C7dMKTmjMBV3cH6b66KazNHej3Av9acKS/apaNvYwOsJDQLp4plps HOQZhkUMR6l9u4+7petJuu+OmhK44QYQQ4lNfb+qrltehvirZ1HIOoO6AKfi8z1e iIuFSqV8cJ6RH8uW79B7BUil2jSvfFyevfzN22ixusfdJxnvLmauxSdz0FsuniGi yDasjQdvfL+N2tgKDxHM =Qj52 -----END PGP SIGNATURE----- --cdlRI4s7lurQFlu8FEL371TCrc27otbX8-- From owner-freebsd-virtualization@freebsd.org Sun Sep 3 05:21:26 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D09D6E1B5E2 for ; Sun, 3 Sep 2017 05:21:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE8A182ACA for ; Sun, 3 Sep 2017 05:21:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v835LP0q007239 for ; Sun, 3 Sep 2017 05:21:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 213814] AWS/EC2: no egress traffic stats on ixv(4) xn(4) Date: Sun, 03 Sep 2017 05:21:25 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: IntelNetworking, needs-qa, regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: a.andersson.thn@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 05:21:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213814 --- Comment #14 from Andreas Andersson --- This still does not work. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Sep 3 11:33:28 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 667AEE06D7C for ; Sun, 3 Sep 2017 11:33:28 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 10596686F1 for ; Sun, 3 Sep 2017 11:33:27 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from [192.168.1.35] (host86-191-18-76.range86-191.btcentralplus.com [86.191.18.76]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v83BXHxY050464 for ; Sun, 3 Sep 2017 12:33:20 +0100 (BST) (envelope-from frank2@fjl.co.uk) To: "freebsd-virtualization@freebsd.org" From: Frank Leonhardt Subject: Windows XP and Xen -Mouse Trouble Message-ID: <54f439a0-e663-1fcb-6885-205e71707a19@fjl.co.uk> Date: Sun, 3 Sep 2017 12:33:18 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 11:33:28 -0000 Does anyone happen to know what's causing this? Windows XP original release (i.e. SP zero) is fine (connecting using VNC). As soon as you install SP2: 1) Windows Hardware Mangler shows two mice - a PS/2 and a QEMU HID compatible one. 2) There's a mystery PCI device. 3) After a few seconds the mouse pointer sticks. If you disable and enable the QEMU mouse it comes back for a few (~30) seconds and sticks again. (There's no option to disable the PS/2 one). It feels like Plug and Pray "finds" it and screws it up again. 4) SP3 doesn't fix it. I haven't tried it with SP1. Fortunately the keyboard is fine and I'm used to using that for most things anyway. It's bugging me - does anyone know what I need to enable/disable and how? Thanks, Frank. From owner-freebsd-virtualization@freebsd.org Sun Sep 3 19:55:01 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 890C1E1C25B for ; Sun, 3 Sep 2017 19:55:01 +0000 (UTC) (envelope-from mail@osfux.nl) Received: from vm1982.vellance.net (vm1982.vellance.net [79.99.187.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3CE37766A7 for ; Sun, 3 Sep 2017 19:55:00 +0000 (UTC) (envelope-from mail@osfux.nl) Received: from vm1982.vellance.net (localhost [127.0.0.1]) by vm1982.vellance.net (Postfix) with ESMTP id 5D1ED2018E for ; Sun, 3 Sep 2017 21:54:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=osfux.nl; s=default; t=1504468491; bh=u2IitJWMJfrOSnAqJJ8JOAkIgTaIuetS9+iRaswkRSI=; h=Subject:To:References:From:Date:In-Reply-To; b=LyWtoBK0yMStCqQU8cGAgQhUqFSI57n7QcxMKZLaUKq59mSpwQGvMn/ytu5vQxRWM cvjVEHXIHpnPgAEeeAmeEy0S/9TB9H1UcUJ1997m25UfvvsTFcaLxb/6OSENReKB/C XcRTtVmqsmgFbHNtdTM1AJqmEHIeNWIFaaP5lYW/IMUZNe6daKpxWI1rT6eFKpRcna EDXmW23tSB5IFvU0Bx1M7rceMG5q9f7VMoEBH5gAnjn+jXh4dHzCOt6b1JdcOto3Wm 7Zm9w70cmm23oPflf/PlrF5r0DDRwY5F2Dd/L2cKDIogwpEj+Fz3pc9lzEZ06CwfMQ AWQ+0Y5j/YeXplKawvws5NgiD1l/TO+OPfq/jcK0ssduXen+lXBjrlSkXIsrLewE/R QRlF1w5FTFvsAoRc/f6XQ00j+B+vVX823cu7e5dl1UJ0p9PQnrkEiTdAg7iN/cD679 /W5OBQi26vf5i/SobWllEvt43AWJkCilFFFehRID5SQJLXzSNOu+BnHKL5z3GC+4ai x1wkHrg+t/z9unth2lF6va9VR8jgZtR4ZJwjOvS1/2Yi6te4VhcGOMCD0k3AzOa63z /6ho6Dia2mKUCIA2IwInM2NG9BmgOzHNKEXxBWbY4Zhx1vFY8c8JYbIQ/nodZRasPf jbmt28CsRYG4op91vDos+fI0= Received: from vm1982.vellance.net (localhost [127.0.0.1]) by vm1982.vellance.net (Postfix) with ESMTP id 456AA20139 for ; Sun, 3 Sep 2017 21:54:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=osfux.nl; s=default; t=1504468490; bh=u2IitJWMJfrOSnAqJJ8JOAkIgTaIuetS9+iRaswkRSI=; h=Subject:To:References:From:Date:In-Reply-To; b=aV/oaLfb3zEfsK/x2t94cFRrCxP17+ZUC1EtuM0n5e3TMRiTFkMilPfCVJv4f78+P Wzqlu/SA5t5NwbJRKCyH2WFzwi1bTD9MM4+0HWsxODWHPfIIwThXzkpdcXtCW644KR jA2ovLyM/S1TpCtixW6KJ25Dahw8PMCEZY4qj8jMEQu9POmKXnaNCOYXbluAd+gVTe UOTY7yXp4FZ4QpPWG5zs+Qb0Qh73JkqhszxNe2sBqR1aTT3ZBRyRKhg4zJsJsYZw6Y A4ZXDnyVmJruAzj1QvS2ISI3sxRC8uYiqgicSEqU/nQ5vu6womM4KPzgdBPcmPyy02 GfwwTxXTmwlTyQ8sALCTcv035+FgO/5PhUbzV4XtSWBCx1fgXPUnTBJS+nZ9DdNF64 Z6VyzqLGgN6rjo/skNKVx+LOxvTJ2IiXDj136v3jtX7wvt4EZ5zcBbPMTFVP9qghS2 ZJCrTzBXLyHLuf79yareWy0q7QqEPRvy1MUJSaZhvB70orJi8dI3SJFeIxlCffyYBx mRViKrMFLAnTtFXgGejxeqAABiHQsBnhtrwxU6sUPNek9GQklXEseuNZliaT++fE30 OVkQU7hMr4XPb0joDIaizvF3TwWcHRxAajOkc2lYPXdwlw5xqHKXfbEWKYgmt3ZBN3 HA695I++Lw9h/SvX7BncrJeA= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on vm1982.vellance.net X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=ALL_TRUSTED,URIBL_BLACK autolearn=no autolearn_force=no version=3.4.1 Received: from rubens-MacBook-Air.local (ip4da7bddc.direct-adsl.nl [77.167.189.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by vm1982.vellance.net (Postfix) with ESMTPSA for ; Sun, 3 Sep 2017 21:54:37 +0200 (CEST) Subject: Re: bhyve VM drive size limit? To: freebsd-virtualization@freebsd.org References: <147e2841-bb29-1fe6-11cc-917f22bf8e87@osfux.nl> From: Ruben Message-ID: <7908ecba-d45a-5f42-744f-dfa267be8d80@osfux.nl> Date: Sun, 3 Sep 2017 21:54:39 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 19:55:01 -0000 Hi Allan/Jason, I think Ill have a go at the "missing space" situation again after some reading up. Since I went with the default block sizes (on both the zvol and the ext4 fs) I probably have some reconfiguration to do.. Thank you very much for your feedback/directions! Kind regards, Ruben On 03/09/17 06:01, Allan Jude wrote: > On 2017-09-02 10:52, Ruben wrote: >> Hi Jason, >> >> >>> Make sure >>> you adjust the zvol block size in the default config before creating your >>> guest or you will wonder where your storage starts to disappear to. >>> >>> Cheers, >>> >>> Jason >> Perhaps slightly off-topic, but might you have any recommendations >> regarding those blocksizes? >> >> I'm not really that well versed in blocksizes / filesystems and am often >> wondering where my 5TB zvol (bhyves into an ext4 volume for an ubuntu >> guest) exactly disappears into :P >> >> Regards, >> >> Ruben >>> On 2 Sep 2017 2:46 p.m., "Dylan Williams" wrote: >>> >>> Dear all, >>> >>> Is there a limit with respect to the size of the virtual drive I can create >>> for a VM using bhyve? Presumably not but I’m running into problems with a >>> larger drive. >>> >>> I am trying to create a 4T Ubuntu machine on a FreeBSD11.1 server using >>> iohyve. There’s plenty of drive space available so it’s not a drive space >>> issue. I am able to create a 40G machine without any problems but exactly >>> the same commands to create a 4000G machine results in a server that won’t >>> boot past Grub. Is this an iohyve issue perhaps? >>> >>> Commands used that results in a working Ubuntu server: >>> >>> iohyve create ubu 40G >>> iohyve set ubu loader=grub-bhyve os=debian ram=40G cpu=10 >>> iohyve install ubu ubuntu-16.04.3-server-amd64.iso < >>> ftp://ftp.cuhk.edu.hk/pub/Linux/ubuntu-releases/16.04.3/ >>> ubuntu-16.04.3-server-amd64.iso> >>> >>> Commands used that results in a broken Ubuntu server: >>> >>> iohyve create ubu 4000G >>> iohyve set ubu loader=grub-bhyve os=debian ram=40G cpu=10 >>> iohyve install ubu ubuntu-16.04.3-server-amd64.iso < >>> ftp://ftp.cuhk.edu.hk/pub/Linux/ubuntu-releases/16.04.3/ >>> ubuntu-16.04.3-server-amd64.iso> >>> >>> The screen I get in the console when I boot (iohyve start ubu) the 4T >>> machine is: >>> >>> GNU GRUB version 2.00 >>> >>> Minimal BASH-like line editing is supported. For the first word, TAB >>> lists possible command completions. Anywhere else TAB lists possible >>> device or file completions. >>> >>> >>> grub> >>> >>> Does anyone have any ideas? I’ve been at this for hours trying different >>> configurations. LVM is not selected when installing either of the above >>> machines as that seems to cause more problems. >>> >>> Many thanks, >>> Dylan. >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to "freebsd-virtualization- >>> unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" >> > To understand why space in a ZVOL goes missing, research 'raidz padding'. > > See here: > https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz > > For more detail, see Chapter 9 of "FreeBSD Mastery: Advanced ZFS" from > http://www.zfsbook.com > > From owner-freebsd-virtualization@freebsd.org Mon Sep 4 12:15:51 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1600AE040D3 for ; Mon, 4 Sep 2017 12:15:51 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CBC62717E6 for ; Mon, 4 Sep 2017 12:15:50 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: by mail-it0-x236.google.com with SMTP id k189so1373981itk.0 for ; Mon, 04 Sep 2017 05:15:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=NJ4lv+EQR8gW3DO2v/LUwQc86tyqzRc1WoNdV5ucjMk=; b=KFcFneqoO4MI88GUdShYsNath8EBXLgHYpCQooHasY78/bo3GimE2HkK93OaSmAGFe Doz5FJhGKb49nIdcpVwP9cDy8fNE+ip6uHof8n/NtMGJjUHQeOgUQ687MjlzEvBlf+Sf EciMwHpjFEInvHEJcB4O4SgD6wX1QP86BZPWfr3Raxypfz+4AHCtMlopworUYdC8BvGQ LEUAB6MwyMLowFlvc5AjVkdxByov4M3/v8nSQDBps/gV4ZfTzcWR+URDHQhtxKzNztwO 84PAyoM5AVuUN1naNlJjOTCrwPDCX70AXQ0S2v9+sNfctR68/G2kkQ2bLgSLeQBdhMyT Ppww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=NJ4lv+EQR8gW3DO2v/LUwQc86tyqzRc1WoNdV5ucjMk=; b=Z9i8qqRu9qRSnaCtzQeRs46VfcDBLhRnZAJqTMlxeSMvVudtvdFUJS96RGDtOnF0J+ jjKVQPKFxgAGP+t9Ze+fMEux267GNGEqYPC7wVTHkuqno/fakc+y0DmNEpYMKmqBBElv oiy2G4fTMgRYcnCt4+p4QWS+HJmzGNaJih17nOaQ2jozniTdqQTXNPBm4idtDvF1hRmh ATydj/F2Bgb6qG0u9q12nQNAhhw2hkRAn8oxDkiXimHF5LIcdIYy+0yj5lJ1kvmxaRd2 yvzyyLHPKsWdXoVagVl7ZgLSe9Fg0cnYMFdbUpNXleVE3D0bB18e8LQHXN8vs9LbdcIz RrjA== X-Gm-Message-State: AHPjjUjy5wF1Sj/ab1PgnJ6UUis7xRuDrNZeJ0hF3dguQf4j1uJZ47NC X1vFCGi1+P3FrgettIZ45l7pK/6PBA== X-Google-Smtp-Source: ADKCNb66NJ19naNN+P3mSyP3sqnZw+kGY+ReJrhaiXxXsL6Shr65Nwlrb1D9NosKVLP+0qn4sqoZMtjFtGDQX2kG8+0= X-Received: by 10.36.25.196 with SMTP id b187mr258065itb.110.1504527349423; Mon, 04 Sep 2017 05:15:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.180.55 with HTTP; Mon, 4 Sep 2017 05:15:48 -0700 (PDT) Received: by 10.2.180.55 with HTTP; Mon, 4 Sep 2017 05:15:48 -0700 (PDT) In-Reply-To: References: From: Sami Halabi Date: Mon, 4 Sep 2017 15:15:48 +0300 Message-ID: Subject: bhyve live migration and auto balance To: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 12:15:51 -0000 hello, I had a look at thr wiki but dont know really the state of live migration.. anyone? I also want to ask if there is any ongoing work for clusters of bhyve that auto balances resources, aka cpu and memory usage of vms between different hosts within the same cluster if they have accesd to the same storage? Thanks in advance, Sami From owner-freebsd-virtualization@freebsd.org Tue Sep 5 17:21:33 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7019EE13E23; Tue, 5 Sep 2017 17:21:33 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13A6A7707B; Tue, 5 Sep 2017 17:21:32 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 5DEB420B56; Tue, 5 Sep 2017 13:21:31 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 05 Sep 2017 13:21:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=fm1; bh=lzfsDXYawMuQsaHRp5mslDG9MNW5qi4ujGsWdWf68 /w=; b=pU/DJcxG3DAvy5JeWbRVYgrrOjIRICOACGTIlYSrHINPxI2iE5iMbKtpy Nx1+HnGbTU5rZWMd2tDf3GtxB0YATu4ysCA5iCL/Ky4cBkvIfRDVq/wNmK1J3QX0 nS45xzhLK4eGQG3QduX4du8w3EQ2K5fRaXRcv4qYNXO5hCGEPfSetSMenJJ4UrGH SZuHjjwS9qxAVpIJmjiNzw64atIjCXlVn/3sGTdM8U0YcIHzQ9hu3ZwQnae8Bhh+ Sb/zxDEhJ8eJfg0gRyg2UXeKrGnXs3yXvPHozcMK4R8up/l2xAzkUTd81hkGvajC Bq8JBNDhpfa/ZWLwWt9Ru1LT2aE3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=lzfsDXYawMuQsaHRp5 mslDG9MNW5qi4ujGsWdWf68/w=; b=JsDFNtBZAiUFK0P0H+SftuZ5FZlGLvc1N9 fj+vkKBEamEJH88881ZLLwLtfK3Eaz/7z+3kmGLm64zNKLENZ3yqxiTSorJpNal2 fzrWt1i+QH3R3aVddWse6HLjU0Y8/m7irq+ehzUQYpULjcdfc4vvXeNdWstx7JbA 2YkShtyGI9da5PzINH3uox4weFMgOyTs4Wl1IbOZAeVR8w1qrlVyqbRpOWQ45iF5 cRT561TZu6Yccnq8Pa1Q+tr5jb9yOVfTUhfl4qlMn3Nns4xOMyywyPzZmKxJx3J2 JphWa2ZgUffrveL9WiKh+l4bLe4ICVX4UbGZ04k005KDLkTWlmSw== X-ME-Sender: X-Sasl-enc: bf1x/kjbCanf+saYjY6lGjj2U/ZYd/RylKXFAqVPX+KA 1504632091 Received: from pumpkin.growveg.org (pumpkin.growveg.org [82.70.91.101]) by mail.messagingengine.com (Postfix) with ESMTPA id D8EE8240AF; Tue, 5 Sep 2017 13:21:30 -0400 (EDT) To: freebsd-virtualization@freebsd.org From: tech-lists Subject: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host Openpgp: preference=signencrypt Cc: freebsd-current@freebsd.org Message-ID: Date: Tue, 5 Sep 2017 18:21:29 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 17:21:33 -0000 Hello freebsd-virtualization@ [also cc'd to freebsd-current], I'd like to run openbsd 6.0 or 6.1 guest under a 12-current bhyve system. I'd like it to run two cpus, so to use the openbsd smp kernel. I can see, from searching various mailing lists that there have been issues in getting openbsd to boot. Have these issues been fixed? Also, is there a howto for this? I found one here: http://www.allanjude.com/bsd/virtualization-host-bhyve.html but there's no date on the document, it's not on the official freebsd sites and so have no idea if the information is still current. I realise that HardenedBSD has fixed some issues with this, but I can't easily change the host to that OS unfortunately. thanks, -- J. From owner-freebsd-virtualization@freebsd.org Tue Sep 5 19:35:35 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 037E7E1B4AC for ; Tue, 5 Sep 2017 19:35:34 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82F5B7492A for ; Tue, 5 Sep 2017 19:35:33 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.15.2/8.15.2) with ESMTP id v85J7hWZ057999; Tue, 5 Sep 2017 14:07:43 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (unknown [172.126.77.65]) by mail.shrew.net (Postfix) with ESMTPSA id B62CC18AD6E; Tue, 5 Sep 2017 14:07:38 -0500 (CDT) Subject: Re: bhyve live migration and auto balance To: Sami Halabi , FreeBSD virtualization References: From: Matthew Grooms Message-ID: <4ee198a5-392f-aed6-f171-aa12e08828d9@shrew.net> Date: Tue, 5 Sep 2017 14:07:33 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.shrew.net [10.24.10.10]); Tue, 05 Sep 2017 14:07:43 -0500 (CDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 19:35:35 -0000 Support for suspend/resume ( prerequisite for migration ) is currently a work in progress ... https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyve_save_restore This only includes support for Intel systems and VMs with virtio devices at present. Support for AMD systems is next. Support for ZFS+snapshots will come soon after that. Live migration will hopefully come later. -Matthew On 9/4/2017 7:15 AM, Sami Halabi wrote: > hello, > > I had a look at thr wiki but dont know really the state of live migration.. > anyone? > > I also want to ask if there is any ongoing work for clusters of bhyve that > auto balances resources, aka cpu and memory usage of vms between different > hosts within the same cluster if they have accesd to the same storage? > > Thanks in advance, > Sami > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Tue Sep 5 21:57:21 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7A5FE215B3; Tue, 5 Sep 2017 21:57:21 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-yw0-x22d.google.com (mail-yw0-x22d.google.com [IPv6:2607:f8b0:4002:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 382A374208; Tue, 5 Sep 2017 21:57:21 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-yw0-x22d.google.com with SMTP id x144so16998780ywg.2; Tue, 05 Sep 2017 14:57:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=IU1SOnLSM0CcJX2ij2V0mIxJ1JAwe1F9JS98/WT53DA=; b=aiVsIfzOVPD58DEd/7gsUPUYIej7aZUlnOfJ8mfaCOLTzRnSeyFUFxPddvLPtAt5L+ 9TIsON552VEMAjRwZ3bWsGQHdOndn4ljhoQVuz8PvRaGfl9ndOCMDDMA/EsRV7bXA+7n fW33EllDBfjBStZdysxS4gcoX8HoD1TIRiBMlWA5A61ricL1opbcglE4I2pQKJ+NdqqW Wsobcvpq9+TxtwXwdnR9F30fnq64MwiMUqekc49wNm7Dne6qP9jl1INXAcHfJAT1DH9R 7nNji/iBFoEPaprTkQAOr5AU73jh0w3ADmkgKEuhBgd0lMctY89a2GnEPyJ5sWcY2uwh 6KOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=IU1SOnLSM0CcJX2ij2V0mIxJ1JAwe1F9JS98/WT53DA=; b=QU3VHzFl2BeyhPG4IK+78Xe8HDThojeACkX14ac5tsKqdZ4isLui2G7jFEA3+NyVY/ jQ7AGd26nmxK57JsMo566fDsYVPEcOERTjylilMs1KNZVVEk8fHJYelgaYH5EjlAn5Cf DGvkCxiS3eOtg5dhfirN00jNh1ltxCB4cclg660FA4oLAd43PGgCIJrDcbNYQy9JBS7f XxPrWPB+SqkH/DyAjOLvqBSCbfMS6Y3yWmQ638QsEorGh/qymtTjSIhN7xO8+W7qmZG6 7QOPLP2uG1DlDV1XOkfRIY1REhWJ33puTJYNIbVsjgazL9hTz+pFavsofjO1gyaw9gui 4bag== X-Gm-Message-State: AHPjjUhY19rKq7pjMMy/Ahm9haxWA0zEz90SYWDgLGAj0OUAcC/Dip/O aAseqOfzbBcHWZwMHjGX6N7lo4uuokaf X-Google-Smtp-Source: ADKCNb6yV14k4pizer9sJXA9PgqW9BUalZGAjcA3gX5z6IaW4nsAwXNy01MusI2iA+Z1v5UTH6UNWR+U11Br4bw2s8k= X-Received: by 10.129.78.85 with SMTP id c82mr439178ywb.94.1504648640359; Tue, 05 Sep 2017 14:57:20 -0700 (PDT) MIME-Version: 1.0 Sender: jtubnor@gmail.com Received: by 10.37.198.10 with HTTP; Tue, 5 Sep 2017 14:56:59 -0700 (PDT) In-Reply-To: References: From: Jason Tubnor Date: Wed, 6 Sep 2017 07:56:59 +1000 X-Google-Sender-Auth: sC824R7Zh8SOOBovj_3XWnXB0XM Message-ID: Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: tech-lists Cc: "freebsd-virtualization@freebsd.org" , freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 21:57:22 -0000 On 6 September 2017 at 03:21, tech-lists wrote: > > > I'd like to run openbsd 6.0 or 6.1 guest under a 12-current bhyve > system. I'd like it to run two cpus, so to use the openbsd smp kernel. I > can see, from searching various mailing lists that there have been > issues in getting openbsd to boot. Have these issues been fixed? > I'm not sure of the exact issue you are referring to but I run a lot of OpenBSD 6.1 hosts under bhyve with GENERIC.MP. You can use either the grub-bhyve style boot or UEFI, both work fine. I have found the chyves framework (it is in packages) good for standing up and managing bhyve guests - with ZFS, especially for a team that are new to the platform. You have to hack the following /usr/local/lib/chyves files: chyves-guest-start chyves-properties chyves-resources to get 6.1 to install and boot (I yet to push those patches upstream). So to answer your question directly, there is currently no known issue in getting bhyve to run OpenBSD 6.1 on Intel or AMD platforms. Enjoy! Cheers, Jason. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 00:03:43 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 603E5E038CE for ; Wed, 6 Sep 2017 00:03:43 +0000 (UTC) (envelope-from lausts@laus.org) Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cdptpa-oedge", Issuer "cdptpa-oedge" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 05F9B6F13A for ; Wed, 6 Sep 2017 00:03:42 +0000 (UTC) (envelope-from lausts@laus.org) Received: from [65.29.112.189] ([65.29.112.189:20812] helo=mail.laus.org) by cdptpa-omsmta01 (envelope-from ) (ecelerity 3.6.9.48312 r(Core:3.6.9.0)) with ESMTP id 29/C0-26203-51B3FA95; Wed, 06 Sep 2017 00:02:29 +0000 Received: from mail.laus.org (localhost [127.0.0.1]) by mail.laus.org (8.15.2/8.15.2) with ESMTPS id v8602SUf031279 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 5 Sep 2017 20:02:28 -0400 (EDT) (envelope-from lausts@laus.org) Received: (from lausts@localhost) by mail.laus.org (8.15.2/8.15.2/Submit) id v8602Sp2031278 for freebsd-virtualization@freebsd.org; Tue, 5 Sep 2017 20:02:28 -0400 (EDT) (envelope-from lausts) Date: Tue, 5 Sep 2017 20:02:28 -0400 From: Thomas Laus To: freebsd-virtualization@freebsd.org Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host Message-ID: <20170906000228.GA31238@mail.laus.org> Reply-To: lausts@acm.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 11.1-RELEASE-p1 on an amd64 User-Agent: Mutt/1.8.3 (2017-05-23) X-RR-Connecting-IP: 107.14.168.6:25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 00:03:43 -0000 Jason Tubnor [jason@tubnor.net] wrote: > On 6 September 2017 at 03:21, tech-lists wrote: > > > > > > > I'd like to run openbsd 6.0 or 6.1 guest under a 12-current bhyve > > system. I'd like it to run two cpus, so to use the openbsd smp kernel. I > > can see, from searching various mailing lists that there have been > > issues in getting openbsd to boot. Have these issues been fixed? > > > There is a bug in the early Intel Skylake CPU series. The Silicon Debug (SDBG) flag is turned on as a default and this causes the OpenBSD 6.1 .iso to refuse installation. This issue was fixed for the later production Skylake and the Kabylake processors that followed. Look at the CPU flags in your computer. If one of them is SDBG, you won't be able to install OpenBSD 6.1. OpenBSD 6.0 doesn't check for this CPU feature and should install OK. The HardenedBSD project has a fix, but it has not been brought into FreeBSD CURRENT. Intel has provided a fix for the BIOS companies but mine (ASUS) has not updated their BIOS. I have a Kabylake processor and a new motherboard on order and will re-purpose my Skylake sytem. Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF From owner-freebsd-virtualization@freebsd.org Wed Sep 6 00:14:04 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E59C9E03F3A for ; Wed, 6 Sep 2017 00:14:04 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 411A771344 for ; Wed, 6 Sep 2017 00:14:03 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 46C9D20ADC05 for ; Wed, 6 Sep 2017 10:13:56 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 41315281F1F for ; Wed, 6 Sep 2017 10:13:56 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id zeahgOZNasLX for ; Wed, 6 Sep 2017 10:13:56 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 2BD792809DB; Wed, 6 Sep 2017 10:13:53 +1000 (AEST) Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: lausts@acm.org References: <20170906000228.GA31238@mail.laus.org> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: Date: Tue, 5 Sep 2017 17:13:51 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170906000228.GA31238@mail.laus.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=IkcTkHD0fZMA:10 a=2JCJgTwv5E4A:10 a=um9RVk0i1K_he2zS9LMA:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 00:14:05 -0000 > There is a bug in the early Intel Skylake CPU series. The Silicon > Debug (SDBG) flag is turned on as a default and this causes the > OpenBSD 6.1 .iso to refuse installation. The '-w' option for bhyve can be used to work around this. later, Peter. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 12:15:31 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92FE3E006FF; Wed, 6 Sep 2017 12:15:31 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13FB46C6D0; Wed, 6 Sep 2017 12:15:30 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 31F7A20DA0; Wed, 6 Sep 2017 08:15:29 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute4.internal (MEProxy); Wed, 06 Sep 2017 08:15:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=jl/lZGaozB3S+w+rJu AFTlcixMu2DLYb5xt+ivA/rok=; b=nhQp2Yveb0JxOCXLjWrksa7oORao7WeBUU +LXKme1OBiPTBMkKnbvwg3qH9RcvTIo/pZCMV1zixvaywTx3fyMmMxTcsAxYfepc sBlm9VTP/AiCMiRTuHnrFsZX7ogPUwhkVlvptilLasIPNmRJhPzSKvvcwiBKkjUd Fxd/l+RfsqTDiTDOEMAep5p+DgsoDw2JEYbbs+M6WmHE1SbSibekV8d6ZXO0qPre 95gH1DBVwsF12GgFHKaSl2OovpALU2p83hUJPqOhQfu5w9rEE364WXIJCwRgjYpF MVCkXuuERJm33PLpCEM6szr8c/EGsjhC3tipDohKE+VPY+zFZrBQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=jl/lZGaozB3S+w+rJuAFTlcixMu2DLYb5xt+ivA/rok=; b=c71OW8C7 bT846UUQXaAIaCpr98Zb0BRR+GTN5dxBv0/VLgMYmfTguWL3/8iS9H0Aq2LSRRKn taBKfJ4N3K2vX3PuqQaRexNQTIK6hMCBAqyH1jdpRx6JT+r8gkZnhuVLzKJb88KO yCFejcje5uIFgvYj98NFvjAZAoumw+2W0/4U+/1lXHkRm1wkpHo/rJJORepoprYi SaLEWHQjaAKKrhdcfCuZtk/ESrwFalOgpmET7miNX1YyTqGGLnFHUSdBlbet1iz3 dwIzxA4N7ys3hwl8veU3DSScXiPgOvhaEq+fNbADe/cla9PdkERlHKr1xs9kmhfC 5Aj2GC40yjmrjQ== X-ME-Sender: X-Sasl-enc: 5z5fbmywdMLaGZJqDccZx8fLjDU5e3Q79htf3uueLuum 1504700128 Received: from pumpkin.growveg.org (pumpkin.growveg.org [82.70.91.101]) by mail.messagingengine.com (Postfix) with ESMTPA id A2E9D7E68E; Wed, 6 Sep 2017 08:15:28 -0400 (EDT) Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host References: To: freebsd-virtualization@freebsd.org Cc: freebsd-current@freebsd.org, Jason Tubnor From: tech-lists Openpgp: preference=signencrypt Message-ID: Date: Wed, 6 Sep 2017 13:15:27 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 12:15:31 -0000 On 05/09/2017 22:56, Jason Tubnor wrote: > I'm not sure of the exact issue you are referring to but I run a lot of > OpenBSD 6.1 hosts under bhyve with GENERIC.MP .  You > can use either the grub-bhyve style boot or UEFI, both work fine. > > I have found the chyves framework (it is in packages) good for standing > up and managing bhyve guests - with ZFS, especially for a team that are > new to the platform.  You have to hack the following > /usr/local/lib/chyves files: > > chyves-guest-start > chyves-properties > chyves-resources > > to get 6.1 to install and boot (I yet to push those patches upstream). > > So to answer your question directly, there is currently no known issue > in getting bhyve to run OpenBSD 6.1 on Intel or AMD platforms.  Enjoy! Hi Jason, thanks for answering. I'll have a go at this. Up until now I've run all my VMs longhand in a screen. Good to know openbsd works. Have you encountered anything on openbsd in a bhyve that you've found not to work? thanks, -- J. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 15:23:51 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 891D6E07C2F; Wed, 6 Sep 2017 15:23:51 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [IPv6:2a01:4f8:171:f902::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E48C75768; Wed, 6 Sep 2017 15:23:51 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.local (unknown [IPv6:2a00:c380:c0d5:1:10be:4354:8df3:b2d8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id CBC82A54D; Wed, 6 Sep 2017 15:23:40 +0000 (UTC) Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: tech-lists , freebsd-virtualization@freebsd.org Cc: freebsd-current@freebsd.org References: From: Jan Bramkamp Message-ID: <638126a7-2527-5329-c952-f173c9b011b2@rlwinm.de> Date: Wed, 6 Sep 2017 17:23:39 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 15:23:51 -0000 On 05.09.17 19:21, tech-lists wrote: > Hello freebsd-virtualization@ > > [also cc'd to freebsd-current], > > I'd like to run openbsd 6.0 or 6.1 guest under a 12-current bhyve > system. I'd like it to run two cpus, so to use the openbsd smp kernel. I > can see, from searching various mailing lists that there have been > issues in getting openbsd to boot. Have these issues been fixed? > > Also, is there a howto for this? I found one here: > http://www.allanjude.com/bsd/virtualization-host-bhyve.html but there's > no date on the document, it's not on the official freebsd sites and so > have no idea if the information is still current. > > I realise that HardenedBSD has fixed some issues with this, but I can't > easily change the host to that OS unfortunately. > > thanks, I'm running OpenBSD 6.1 (with two virtual CPU cores) as bhyve guest on FreeBSD 11.1. The only problem I encountered is that it requires an external grub bootloader because the OpenBSD EFI boot code is incompatible with the bhyve EFI boot ROM. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 15:45:25 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC060E08C48; Wed, 6 Sep 2017 15:45:25 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E91B47E346; Wed, 6 Sep 2017 15:45:24 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 340B9211D3; Wed, 6 Sep 2017 11:45:23 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute4.internal (MEProxy); Wed, 06 Sep 2017 11:45:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=3a+jtSaykjATus5Wrp d7ETBRguxsieZGyFLW7DLSZAk=; b=RNHrijbDUwh0BBDuegunKMZg0h4SWP8gDf wB0XCgzgALHeMja0inJNnmItPF6Ix+3Z/2pnoClkMuKT7U16DZwbLWpwuPyb9FjY 6JSfIy2u4cIr++c3uoyGgrrEh6Vfacl3boRTRgrgATNWUjcZZVcfwfznIkp+PXvN cR+hmPFa43DR/KkVnknBCY91HjBOf717VkvmO6gszJmCquiPuR/jrmeUtasJSQNO Xyf1IJKW8+/p716NQD0M/xz/4MrIz9EaqEEkej0zqzvZ6OnWjoRwouc1394LRwDt t9s2IkEivsXh4gwuhAEDw93GjqzHc7/VIMe228TAeygncyupT9OA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=3a+jtSaykjATus5Wrpd7ETBRguxsieZGyFLW7DLSZAk=; b=fGqDwScw bvH8LWGZGAra5DoyMfiBA/thmH8tOg8VUZV28hL6qeFRoKKYL6pDpA/y4OuO8d6N +5dqZAMJ2sa4frY+nu7x5tvSbhPSy+bFHG/c1BYF2aUsU1RirydHMLWVhBVkt9ui hl1PszNNmwcKBAx/qkb1r9C6Qu/yqMFdZwLVe2f25Cu79r6iy4eSevziufRc5X0v nh7zfVLaz6JeL/n6soId6TYZUSVDRU/BU+NGhdDo6180H2XKkneyXRIn0D8GHjHB MlD1p1UtTCKepzImK+posSvy+TUxgnePLZKrn/O644HcD3GLuoAuWWlEBMRCAUGu mf1cUr3E4pPx+Q== X-ME-Sender: X-Sasl-enc: o5IhOtFJxGbnIge1KGGAaEmy4SStF13RJ+onnqU1L2NJ 1504712722 Received: from pumpkin.growveg.org (pumpkin.growveg.org [82.70.91.101]) by mail.messagingengine.com (Postfix) with ESMTPA id A24347E464; Wed, 6 Sep 2017 11:45:22 -0400 (EDT) Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: Jan Bramkamp , freebsd-virtualization@freebsd.org Cc: freebsd-current@freebsd.org References: <638126a7-2527-5329-c952-f173c9b011b2@rlwinm.de> From: tech-lists Openpgp: preference=signencrypt Message-ID: Date: Wed, 6 Sep 2017 16:45:20 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <638126a7-2527-5329-c952-f173c9b011b2@rlwinm.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 15:45:26 -0000 On 06/09/2017 16:23, Jan Bramkamp wrote: > I'm running OpenBSD 6.1 (with two virtual CPU cores) as bhyve guest on > FreeBSD 11.1. The only problem I encountered is that it requires an > external grub bootloader because the OpenBSD EFI boot code is > incompatible with the bhyve EFI boot ROM. Hi, Which external grub bootloader did you use? Was it sysutils/grub2-bhyve ? thanks, -- J. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 15:50:56 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA383E0900D; Wed, 6 Sep 2017 15:50:56 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [IPv6:2a01:4f8:171:f902::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 22C6A7F512; Wed, 6 Sep 2017 15:50:56 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.local (unknown [IPv6:2a00:c380:c0d5:1:10be:4354:8df3:b2d8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id A2A9AA5E9; Wed, 6 Sep 2017 15:50:54 +0000 (UTC) Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: tech-lists , freebsd-virtualization@freebsd.org Cc: freebsd-current@freebsd.org References: <638126a7-2527-5329-c952-f173c9b011b2@rlwinm.de> From: Jan Bramkamp Message-ID: Date: Wed, 6 Sep 2017 17:50:54 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 15:50:56 -0000 On 06.09.17 17:45, tech-lists wrote: > On 06/09/2017 16:23, Jan Bramkamp wrote: >> I'm running OpenBSD 6.1 (with two virtual CPU cores) as bhyve guest on >> FreeBSD 11.1. The only problem I encountered is that it requires an >> external grub bootloader because the OpenBSD EFI boot code is >> incompatible with the bhyve EFI boot ROM. > Hi, > > Which external grub bootloader did you use? Was it sysutils/grub2-bhyve ? I use the sysutils/grub2-bhyve port to load the OpenBSD kernel and runit for process supervision to keep the bhyve guest running unless the bhyve exit code signals an intend to perform shutdown. This allows the VMs to reboot themselves. From owner-freebsd-virtualization@freebsd.org Wed Sep 6 21:53:56 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 415C1E19553; Wed, 6 Sep 2017 21:53:56 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6BC8A69E57; Wed, 6 Sep 2017 21:53:55 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-oi0-x22c.google.com with SMTP id x190so33556980oix.3; Wed, 06 Sep 2017 14:53:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=mUS4/YqziNNsvLwvflLOeVEXXglAxPZxhn1drN3vaoc=; b=R/PTvz01Ax9wcKmS+Wd4Mv949TbrtloZyNzOyHIyu9Eg8E3lEb8RW+xXCGADc1CTyf pCo0Ld5LzFCGPjcFthhJSLAWgzExg3Mas1hvM7Cas2gbbZNPeDfvXfYKMNiQoXMtKWHx w+25056sywF7cHBgihIIiss6bHk12QCWbaflZKoPdJQhDPhYXJCS38nnfOR22xyT6UKg vKPss6cBzXru8BYykELEIKIlr0df4JhVRfNs+gfTSQuNU8DKUGSvu5pmlpxO7IBJR7B/ elAvO73YAyCVi5RyoV3Rr4TEZ7CAqIHjtaeXfclplis/VESQTfvvnOutfUvKV3iFNXYF /teA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=mUS4/YqziNNsvLwvflLOeVEXXglAxPZxhn1drN3vaoc=; b=OHj3PxRF1k+vaUKT5d/wEX7ouZoJtck/X9oCRA5ggRiCOTm+6ScStlofzpHi3TEAvk iPIMRkgRpSomUba6t1s55dxcvbsXeHaf1frKKtxnxdzpao43O3j8JM8K0HdGoQgCBSEo il0c2IEdLqF7k9NgHnNI5h2LLnRyVjqLytuNEapExtRh32gwnF4x5NmEE2e6Z1KN3acW Pxc/SYgb4PlZsScEjxieeqQhk2d6W89Sz9z+vR8yupGR3l7sf0fa6qqdTiNxdFSpqOak UW9uYKnZtGDM+92OLCYIsDyglHevSw3Bbcz9h3gMLbOf/OLZn2nm8JBxTRnH6R80EzKc Yucw== X-Gm-Message-State: AHPjjUijKA2rmRT5/R9DPd28D4MbuC0BdgOU6ZfqCd7lgsosQB5hePeU jTEbZe4jR+x4jlgUPnsy8CnnHgsjwHujx8U= X-Google-Smtp-Source: ADKCNb6wI5c63tvBIatqn32SqjrHTRrh3gsgq8HzLqjPf/lpbrxnYZMZI8L/6aNkCwvJzzWe/eEiRMhqPsUl0dQ8BX8= X-Received: by 10.202.213.5 with SMTP id m5mr771611oig.35.1504734834342; Wed, 06 Sep 2017 14:53:54 -0700 (PDT) MIME-Version: 1.0 Sender: jtubnor@gmail.com Received: by 10.157.42.114 with HTTP; Wed, 6 Sep 2017 14:53:33 -0700 (PDT) In-Reply-To: References: From: Jason Tubnor Date: Thu, 7 Sep 2017 07:53:33 +1000 X-Google-Sender-Auth: 9GeOAFquQLRldAWblzg3jTjmNb8 Message-ID: Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: tech-lists Cc: "freebsd-virtualization@freebsd.org" , freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 21:53:56 -0000 On 6 September 2017 at 22:15, tech-lists wrote: > > > Have you encountered anything on openbsd in a bhyve that you've found > not to work? > > As Thomas mentioned, there is/was a bug with certain CPUs, but this was due to the strict checking of CPU features that OpenBSD introduced ( https://marc.info/?l=openbsd-misc&m=149136173520510&w=2). Peter explained the switch to get around that issue (-w). The only other thing is Jumbo frame support, but that is really down to limits in VirtIO and not really a show stopper and doesn't affect anything in production for us. Enjoy bhyve, it has been a drop in replacement for our individual ESXi instances and has been bulletproof, which is good as we have hypervisor hosts over 6 hour drives away! (Oh the uplift from 11.0 to 11.1 was done remotely without nuking the host). Cheers, Jason. From owner-freebsd-virtualization@freebsd.org Thu Sep 7 04:30:29 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35C9CE050C5 for ; Thu, 7 Sep 2017 04:30:29 +0000 (UTC) (envelope-from karihre@gmail.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9F488274D for ; Thu, 7 Sep 2017 04:30:28 +0000 (UTC) (envelope-from karihre@gmail.com) Received: by mail-it0-x22f.google.com with SMTP id f199so16000348ita.1 for ; Wed, 06 Sep 2017 21:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=z/wFvAk71dJJgg8kIvdFbZTQC0kML6fjBhaIJaIpZqY=; b=YFHzysOUrMjkqgjegfjjBpFlEWyhKPt+JP7JiLGcTx4Loa2ZlFGR4nSWrtSuoG9P0q JwVo4s7OhHVX18A4aZqsb9x05Bz8qX1qvlO5z4q+7wtpve2h2WE8tP2IZ9SEkAGO8OwE siiU0WGsnJeQMN2lqYu/J7bxXpNRnqte1dvp4ZoOD6IZI/gzxRqqILTVnomOmwq9Bq+p L2Eo0WslkHNDunIwKuGOJ8qgF8BcRYjTA9wj94iaJzaULz48b/8daNeG1oW56A3T+W0+ vdKNi4Lp54WZQiRhE+mOjJn+j29rHLCUZT0OMhbLDZkZdagkreCl+z+ra7S/RofFuymm LLpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=z/wFvAk71dJJgg8kIvdFbZTQC0kML6fjBhaIJaIpZqY=; b=ToR0Q9bqLOFlVg9BZlX8SzbsG3OeIV++2n17Hu0Rzqb1KJLzlOd+h7cVGvHfSctj2O Ywwq1WMPDqavqRxiiPe8G/Ub8bgHbk3tlbDMf88zSM7HZgwk1QcL0PV16jtev6Ya4RBg UNkFQNBCrZnerpExepmMokTMXDxHYn+m77z7IMhoJZh/DrpkulrvbOgcEpXweS0l2/nw VhC0e4AThjQfdZdOe375Bw+9dxGw4Sk3xYlajmao5n1uugZCvLdB/0Z9U5aK+6FXutcb vg+EwejDopLWLVKmzCn9R83JFF3W5PGNrGkB9st4clrhSbsSsu++0ePFFemNDYLg/ix7 5ZJw== X-Gm-Message-State: AHPjjUhIUs3GpKZaUQlNJcg5dyr6GqqsZ7BrBZ5R4ageNOEhSpuoVoMV j/l+qjfIXyMRIPMgwRWOIsxsiU7GEIqOj7Y= X-Google-Smtp-Source: ADKCNb6p0en2QEcCyijavv/s4RDCP7cOOHDlgLJMHFsvo7bvC+Iz9Yq5bitEXvSNCYUK8HoKkxhvG65jYOUH8l0oOvI= X-Received: by 10.36.91.85 with SMTP id g82mr2593409itb.141.1504758627586; Wed, 06 Sep 2017 21:30:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.98.72 with HTTP; Wed, 6 Sep 2017 21:30:07 -0700 (PDT) From: =?UTF-8?Q?K=C3=A1ri_Hreinsson?= Date: Wed, 6 Sep 2017 21:30:07 -0700 Message-ID: Subject: (bhyve) Debian vm crashing with kernel panic To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 04:30:29 -0000 Dear all, I have been experiencing random linux kernel panics on a Debian virtual machine running under bhyve on FreeBSD 11.1, and believe it may be related to the virtualization environment. I am not an advanced FreeBSD user by any means, which is why I am turning to this mailing list for possible answers, realizing that I could be making some simple errors. I have two similar (same version and kernel) Debian VMs running on the FreeBSD host, one of them lightly loaded and running without any issues, the other one more heavily loaded and experiencing kernel panics a few days after booting. CPU: Intel(R) Xeon(R) CPU E3-1275 v6 Host system: 11.1-RELEASE-p1 VM: Debian 9 (Stretch), kernel 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 On the FreeBSD side of things I find nothing in any logs under /var/log indicating any problem (perhaps I am not looking in the right places?). On the Debian side of things an open ssh session got plenty of these leading up to the crash: kernel:[489300.648296] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:0:14902] Debian kern.log file contains this just before the crash: Sep 6 10:23:59 hostname kernel: [488456.219948] INFO: rcu_sched self-detected stall on CPU Sep 6 10:23:59 hostname kernel: [488456.220007] 0-...: (5249 ticks this GP) idle=b45/140000000000001/0 softirq=27802459/27802459 fqs=2423 Sep 6 10:23:59 hostname kernel: [488456.220062] (t=5250 jiffies g=10449032 c=10449031 q=319) Sep 6 10:23:59 hostname kernel: [488456.220093] Task dump for CPU 0: Sep 6 10:23:59 hostname kernel: [488456.220094] kworker/0:0 R running task 0 14902 2 0x00000008 Sep 6 10:23:59 hostname kernel: [488456.220108] Workqueue: rpciod rpc_async_schedule [sunrpc] Sep 6 10:23:59 hostname kernel: [488456.220109] ffffffff90713580 ffffffff8faa3bcb 0000000000000000 ffffffff90713580 Sep 6 10:23:59 hostname kernel: [488456.220111] ffffffff8fb7a4b6 ffff8a0bffc18fc0 ffffffff9064a6c0 0000000000000000 Sep 6 10:23:59 hostname kernel: [488456.220112] ffffffff90713580 00000000ffffffff ffffffff8fadee04 0000000000e746a9 Sep 6 10:23:59 hostname kernel: [488456.220113] Call Trace: Sep 6 10:23:59 hostname kernel: [488456.220114] Sep 6 10:23:59 hostname kernel: [488456.220116] [] ? sched_show_task+0xcb/0x130 Sep 6 10:23:59 hostname kernel: [488456.220118] [] ? rcu_dump_cpu_stacks+0x92/0xb2 Sep 6 10:23:59 hostname kernel: [488456.220119] [] ? rcu_check_callbacks+0x754/0x8a0 Sep 6 10:23:59 hostname kernel: [488456.220121] [] ? update_wall_time+0x473/0x790 Sep 6 10:23:59 hostname kernel: [488456.220122] [] ? tick_sched_handle.isra.12+0x50/0x50 Sep 6 10:23:59 hostname kernel: [488456.220124] [] ? update_process_times+0x28/0x50 Sep 6 10:23:59 hostname kernel: [488456.220125] [] ? tick_sched_handle.isra.12+0x20/0x50 Sep 6 10:23:59 hostname kernel: [488456.220125] [] ? tick_sched_timer+0x38/0x70 Sep 6 10:23:59 hostname kernel: [488456.220126] [] ? __hrtimer_run_queues+0xdc/0x240 Sep 6 10:23:59 hostname kernel: [488456.220127] [] ? hrtimer_interrupt+0x9c/0x1a0 Sep 6 10:23:59 hostname kernel: [488456.220128] [] ? smp_apic_timer_interrupt+0x39/0x50 Sep 6 10:23:59 hostname kernel: [488456.220129] [] ? apic_timer_interrupt+0x82/0x90 Sep 6 10:23:59 hostname kernel: [488456.220130] Sep 6 10:23:59 hostname kernel: [488456.220131] [] ? native_queued_spin_lock_slowpath+0x21/0x190 Sep 6 10:23:59 hostname kernel: [488456.220132] [] ? _raw_spin_lock+0x1d/0x20 Sep 6 10:23:59 hostname kernel: [488456.220141] [] ? nfs4_close_done+0xfa/0x400 [nfsv4] Sep 6 10:23:59 hostname kernel: [488456.220145] [] ? nfs4_xdr_dec_open_downgrade+0xf0/0xf0 [nfsv4] Sep 6 10:23:59 hostname kernel: [488456.220151] [] ? __rpc_sleep_on_priority+0x340/0x340 [sunrpc] Sep 6 10:23:59 hostname kernel: [488456.220155] [] ? __rpc_sleep_on_priority+0x340/0x340 [sunrpc] Sep 6 10:23:59 hostname kernel: [488456.220159] [] ? rpc_exit_task+0x2a/0x90 [sunrpc] Sep 6 10:23:59 hostname kernel: [488456.220163] [] ? __rpc_execute+0x86/0x420 [sunrpc] Sep 6 10:23:59 hostname kernel: [488456.220164] [] ? process_one_work+0x184/0x410 Sep 6 10:23:59 hostname kernel: [488456.220165] [] ? worker_thread+0x4d/0x480 Sep 6 10:23:59 hostname kernel: [488456.220166] [] ? process_one_work+0x410/0x410 Sep 6 10:23:59 hostname kernel: [488456.220167] [] ? do_group_exit+0x3a/0xa0 Sep 6 10:23:59 hostname kernel: [488456.220168] [] ? kthread+0xd7/0xf0 Sep 6 10:23:59 hostname kernel: [488456.220169] [] ? kthread_park+0x60/0x60 Sep 6 10:23:59 hostname kernel: [488456.220170] [] ? ret_from_fork+0x25/0x30 This seems to be all I have to go on. This is the first panic I experience after upgrading to 11.1, in the past I was experiencing similar panics on 11.0 but the log file output from those seemed different as the kernel spat out hundreds of errors in the hours leading up to finally crashing. I'm not sure those are relevant as I was running 11.0 and didn't see the same (but similar) errors this time around, but I can attach that log file if anyone is interested. The vm startup command is: bhyve -AHP \ -s 0:0,hostbridge \ -s 1:0,lpc \ -s 2:0,virtio-net,tap0 \ -s 3:0,virtio-net,tap1 \ -s 4:0,virtio-blk,/dev/zvol/tank/vms/hostname-root \ -s 5:0,virtio-blk,/dev/zvol/tank/vms/hostname-scratch \ -s 6:0,virtio-blk,/dev/zvol/tank/vms/hostname-temp \ -s 29,fbuf,tcp=127.0.0.1:5900,w=800,h=600 \ -l com1,/dev/nmdm0A \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -c 2 \ -m 32G hostname Anything that could shed some light on this issue would be much appreciated. If I can provide any additional information please let me know. Thank you, Kari Hreinsson From owner-freebsd-virtualization@freebsd.org Thu Sep 7 06:40:40 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FF80E0B082 for ; Thu, 7 Sep 2017 06:40:40 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: from mail-pg0-x231.google.com (mail-pg0-x231.google.com [IPv6:2607:f8b0:400e:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9F897CDAB for ; Thu, 7 Sep 2017 06:40:39 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: by mail-pg0-x231.google.com with SMTP id q68so3502315pgq.1 for ; Wed, 06 Sep 2017 23:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Qr5OSxuHiSqj95ULtSldnKRHssutGi6YmhSV7nwuaKA=; b=CGTYNtDLGqDokbe7/9Q520HSf0xaEfWAFVzTBw75SlhcPbfeEfbemgVOGn2BZqZ0O7 shLTbwRX6y/b62luhvWE0oD6EC8I9WiBSF07WwxeIrgKNdyiDzhBaHP25+W/PkBYXlvW svj2O1WKqQz/628U/XBV+EEE+njHHyutx77O8NRkT4G+5b0/TfdtAz/FIU23sk0uYa4G I07Fyt3j0YXD1bZeq+YjF3qQfve0IMeRlc2ZomIMur3ZKU8F1Prue6HMu9rQ4JKR96CC TaDFID3yQZhUGRM1en95wMQ54xuCq0K4HR/dO3OPbdbZ2auLi9qa6Bge/acOdQ0VsA3y sszw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Qr5OSxuHiSqj95ULtSldnKRHssutGi6YmhSV7nwuaKA=; b=R0Wbm/s9hbhF3hafO/JlvCXbL/lqWd7aPth9WtZkFyaPTXZG3bMdWKwRmTlSjc9KwX ATuOIB93pRZnAOeZkQUJJybh8ynioGktyDsbb3mWFvyM855VHeYQ4alXb5M6Y8tnP0+c nNVp872hK3Hp9YU9N/Rsk6O0iin6wZ63PaiAflzae8B3XM3HrTXLOzF0md6AxG+bl5sy V+rHb0HN/NKrTybLt5G6SmambzpnEW2hmKJAvDgDqf1+8xn5tliORLPxV3xVKFAG9U81 sgYrdm9DLHsF8UpRncAgadNvj5qKb600yo/Y9D56cnh9EY3CCxBmYamtF7AD8wX9pTgE 7jVA== X-Gm-Message-State: AHPjjUihOHebILZPpKZCHSmmrfhuo/al3U63RvDg2FcBBgA70UPhAfdQ PeOg4SUWPr5qCQFId5emMjW/G8/u5hlaz/uxEbskYlYn X-Google-Smtp-Source: ADKCNb7KelzOTvOFbzsVDqtF8DBHijwp/pt2gFS0ykqVZFoVTN6VnFQX80cXcnMVu5wd9o55usz0knRQLp7rXQSj7GE= X-Received: by 10.84.232.14 with SMTP id h14mr1879576plk.274.1504766438539; Wed, 06 Sep 2017 23:40:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.183.164 with HTTP; Wed, 6 Sep 2017 23:40:18 -0700 (PDT) From: Farhan Khan Date: Thu, 7 Sep 2017 02:40:18 -0400 Message-ID: Subject: bhyve passthrough fails To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 06:40:40 -0000 Hi, I am trying to pass an mPCIe device to a guest bhyve VM. I have the device set to use the ppt driver: # devctl set driver pci0:2:0:0 ppt pciconf -lv gives me the following output: ppt0@pci0:2:0:0: class=0x028000 card=0x197d103c chip=0x817910ec rev=0x01 hdr=0x00 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8188EE Wireless Network Adapter' class = networ But when I start a FreeBSD guest with the vmrun script (sudo sh vmrun.sh -p 2/0/0 -c 2 -m 512M -t tap0 -d freebsd12.img freebsd1), I immediately get this error: bhyve: PCI device at 2/0/0 is not using the ppt(4) driver. I have virtualization set in the BIOS and am not certain what may be causing this issue. Any ideas? Thank you, -- Farhan Khan PGP Fingerprint: 782F 342B 5B08 0D2F F4E8 82C3 FFA1 CAE1 6536 51CA From owner-freebsd-virtualization@freebsd.org Thu Sep 7 14:28:50 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E1DBE20608; Thu, 7 Sep 2017 14:28:50 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DDF780169; Thu, 7 Sep 2017 14:28:49 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x232.google.com with SMTP id q132so24910367lfe.5; Thu, 07 Sep 2017 07:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=JXfy2bTttoWSWELvgCdbmcumDSL5eBa+rRpCqFwU4lI=; b=E/b19D6wQ16iPa2OEDEpPzaDTJ8rHfNXwbcAeNK4yKsD9XHgUYhHl67uAvogEtSBHS OCR9pTw/HckA54kHhG5+PbsCkadFZiOuYV//wHO4NVaIUXFweh4zDV3nX4Njavo81/4M kH1PngQrYC9kyuvrrSxRmoZo6WGklUgcL8LnsMRZEUSTXs6gW+V1yKM2Sv1JqUNeOfh+ BR1qrgNZAR6Ku8Oa5SkDJCigkkejW501f6m0Kzbm75gp46blKScrrO/RYX/2OyRonieV E/Ct8lhXsyEPA0eLT1v8rTSZSxUeQFq1Az//+EaRot4WyuCgGFXzBPfdBCBtqUWyp5YE v9gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=JXfy2bTttoWSWELvgCdbmcumDSL5eBa+rRpCqFwU4lI=; b=InFjO896DSZ/rxHGxr4124ktIh/oj0Zo4lVPaayH3Gw5tUq7R/4p/HFzR0SiAMy/qj RGeoz5yIsTvv1SAo5IPzQ1+6CesH2/4EgKu+Mj/OUwRgmaa+qy9JEFbOXILhj6whe14J DoDaZo2IM8YtvOrjCjznh6/9yfKr+QX0wss9dnJ8c6nPDBd6jzYRjNTcsvX30Ylas07S S4D+mUNQXxKpcjXzI6qhf2YgpClqmbfR0tzwzDghpfzoFEBO5+hMMtMuHvy3vSFGEfqS qbFX1a9ootd9i4lJnBOJL+I1O+umHERx482DkIsqrdKc/zoqd71dF/zdxkYx6Vqzuap7 a/5Q== X-Gm-Message-State: AHPjjUhj8HLS2Jxe2iqNtSfJMSGBMZMe11XwLZ7mmV2OIv4/H701tzCV sbKs7MdwgyoqT46tma68n9lroIJiBQ== X-Google-Smtp-Source: AOwi7QBFVZYBhxZKWs6zUQG210tkv2IHlBhloNlOBWRtE5OBYoZIQK6P7eWPys5oG37kE8zYmvsJlGwOh2ZlgyiD2K4= X-Received: by 10.25.23.95 with SMTP id n92mr957405lfi.95.1504794527501; Thu, 07 Sep 2017 07:28:47 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.26.6 with HTTP; Thu, 7 Sep 2017 07:28:46 -0700 (PDT) In-Reply-To: References: From: Alan Somers Date: Thu, 7 Sep 2017 08:28:46 -0600 X-Google-Sender-Auth: wRVeuEN4v-HcCsPZNH93TsDbWjA Message-ID: Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: Jason Tubnor Cc: tech-lists , "freebsd-virtualization@freebsd.org" , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 14:28:50 -0000 On Wed, Sep 6, 2017 at 3:53 PM, Jason Tubnor wrote: > On 6 September 2017 at 22:15, tech-lists wrote: > >> >> >> Have you encountered anything on openbsd in a bhyve that you've found >> not to work? >> >> > As Thomas mentioned, there is/was a bug with certain CPUs, but this was due > to the strict checking of CPU features that OpenBSD introduced ( > https://marc.info/?l=openbsd-misc&m=149136173520510&w=2). Peter explained > the switch to get around that issue (-w). > > The only other thing is Jumbo frame support, but that is really down to > limits in VirtIO and not really a show stopper and doesn't affect anything > in production for us. > > Enjoy bhyve, it has been a drop in replacement for our individual ESXi > instances and has been bulletproof, which is good as we have hypervisor > hosts over 6 hour drives away! (Oh the uplift from 11.0 to 11.1 was done > remotely without nuking the host). > > Cheers, > > Jason. Do you have a semiautomated way to move bhyve instances between different hosts? Without such a feature, we can't replace all of our ESXi machines. -Alan From owner-freebsd-virtualization@freebsd.org Thu Sep 7 14:52:22 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6599DE21485 for ; Thu, 7 Sep 2017 14:52:22 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id E0FD81997 for ; Thu, 7 Sep 2017 14:52:21 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id BC9EB204093A for ; Fri, 8 Sep 2017 00:52:18 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id B715C2804DB for ; Fri, 8 Sep 2017 00:52:18 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 6yzWGLRUanef for ; Fri, 8 Sep 2017 00:52:18 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-73-170-144-137.hsd1.ca.comcast.net [73.170.144.137]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 145FE2804DC; Fri, 8 Sep 2017 00:52:16 +1000 (AEST) Subject: Re: bhyve passthrough fails To: Farhan Khan References: Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: Date: Thu, 7 Sep 2017 07:52:16 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=FAFtVJ6M21jV2uT+POhvGw==:17 a=IkcTkHD0fZMA:10 a=2JCJgTwv5E4A:10 a=Wt886vpqPyltqJ0U1T4A:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 14:52:22 -0000 Hi Farhan, > bhyve: PCI device at 2/0/0 is not using the ppt(4) driver. Just noticed this myself: looks like a regression :( I'll see if I can work it out and get a fix in. later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Sep 7 15:58:28 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16391E00DB5 for ; Thu, 7 Sep 2017 15:58:28 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9EDFE6FADE; Thu, 7 Sep 2017 15:58:27 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v87FwJup027615; Thu, 7 Sep 2017 08:58:19 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v87FwJ5x027614; Thu, 7 Sep 2017 08:58:19 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201709071558.v87FwJ5x027614@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve passthrough fails In-Reply-To: To: Peter Grehan Date: Thu, 7 Sep 2017 08:58:19 -0700 (PDT) CC: Farhan Khan , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 15:58:28 -0000 > Hi Farhan, > > bhyve: PCI device at 2/0/0 is not using the ppt(4) driver. > > Just noticed this myself: looks like a regression :( I'll see if I can > work it out and get a fix in. What is the state of bhyve in 10.4? Do we need to fire off some quick tests???? -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Sep 7 16:50:46 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89F94E02FAA for ; Thu, 7 Sep 2017 16:50:46 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: from mail-pg0-x231.google.com (mail-pg0-x231.google.com [IPv6:2607:f8b0:400e:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ACD07F187; Thu, 7 Sep 2017 16:50:45 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: by mail-pg0-x231.google.com with SMTP id m9so482725pgd.3; Thu, 07 Sep 2017 09:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KPQaIjYik+nJgopsO5WJbHrRAuwyKhyx+7P4HconWKQ=; b=hMOJBywa0uN8RfIm8jCsou99qtqIK/O8PYtm4vxejN0f9igjOyjpoP1tK4Pkys06Zb XXkvVnYlQb3acj2xE01+VGk0kV8/C6gLQnV9tSgAEQEdyb9wfvQ3cGjiHwNspzLxmvBe iCxJuz7unobcMIQJQ447ZNPc1Xfb/oAUO6ExHqZ3/QkFG3DQBArcoUlXNSZ5PjoxxWGx VdxusPd4FthQm1NkvnxQX+mQpsdzWMJQiksYuUdHuwC2fcJ/cIrh8evCZzmTJGo5/MBJ QPTEU5Z06wnzzLld4+Wn22fS8tUQTJ/jOj2vh4queAFL1111wDtae9kCo4W906HRjauj Yw3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KPQaIjYik+nJgopsO5WJbHrRAuwyKhyx+7P4HconWKQ=; b=NuHsdBp9AndXJYEkDQQ9R2EOXrrFPz0QJXWzgmMRBnHEv486/D4EZi4Zz/gJlibmsN lNhGG2Nya6OQD7CIHHcQF8v739vMyP2SiudT/kf0/Z+T2NVQMOJ8vaDSjnE0Z+IQmK1T yJ4Cd/od4XE26As2GEhhof8YgyTUGw5sfZvl/5zU/pBFQ9GMd4gFOgs0kXT4clDZsvY0 zhAAzG4SuyjbiLeNK4zmbIGMlazjZrKKhqIyPi5BldZVg9rxR/3WsH8a6e/tjnLV83Jd eAUUQsrDzTEAFLYEPEkR+hy/qkPAdJ+gjkOMyU6DzIlfU5coDrEvEHL2rgEV2mmLaJvl PncA== X-Gm-Message-State: AHPjjUjR2saBC3yc1cj5Pxaf0vIC/i5kiHqQ2gDd2/kMoWHNdRqcflS/ SbDXcqaVBh4Qxh0dSaXaqgnOnBOE/JGpFvyWfb+dhO6l X-Google-Smtp-Source: ADKCNb6uhvMb4h7PeeyflcswBdmn457YRjkOwcDR0QsE5/0EDRnwLNOvafJGNgGoaxZJUioq9OH6XCbWaeYFU6PuIoY= X-Received: by 10.99.112.94 with SMTP id a30mr3376148pgn.304.1504803044237; Thu, 07 Sep 2017 09:50:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.183.164 with HTTP; Thu, 7 Sep 2017 09:50:23 -0700 (PDT) In-Reply-To: References: From: Farhan Khan Date: Thu, 7 Sep 2017 12:50:23 -0400 Message-ID: Subject: Re: bhyve passthrough fails To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 16:50:46 -0000 I would not mind sending you on-going feedback or debugging information if it helps. Could it be a chipset issue? This is a lower-end laptop. It can do virtualization, but the passthrough issue seemed to fail. I was on FreeBSD 11.1. -- Farhan Khan PGP Fingerprint: 782F 342B 5B08 0D2F F4E8 82C3 FFA1 CAE1 6536 51CA On Thu, Sep 7, 2017 at 10:52 AM, Peter Grehan wrote: > Hi Farhan, > >> bhyve: PCI device at 2/0/0 is not using the ppt(4) driver. >> > > Just noticed this myself: looks like a regression :( I'll see if I can > work it out and get a fix in. > > later, > > Peter. > From owner-freebsd-virtualization@freebsd.org Thu Sep 7 18:05:27 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 162AAE05CFC for ; Thu, 7 Sep 2017 18:05:27 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 9C7BA6A071 for ; Thu, 7 Sep 2017 18:05:26 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 04039210A291 for ; Fri, 8 Sep 2017 04:05:23 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id E4B3B2809CC for ; Fri, 8 Sep 2017 04:05:22 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hSOY_kpzsCrI for ; Fri, 8 Sep 2017 04:05:22 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id E9AAE2809A2; Fri, 8 Sep 2017 04:05:20 +1000 (AEST) Subject: Re: bhyve passthrough fails To: Farhan Khan Cc: freebsd-virtualization@freebsd.org References: From: Peter Grehan Message-ID: <5e672671-22d6-3569-ea2b-a8105b8e3334@freebsd.org> Date: Thu, 7 Sep 2017 11:05:17 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=IkcTkHD0fZMA:10 a=2JCJgTwv5E4A:10 a=-OhNdOWJgklVJANl9ocA:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 18:05:27 -0000 > I would not mind sending you on-going feedback or debugging information > if it helps.> > Could it be a chipset issue? This is a lower-end laptop. It can do > virtualization, but the passthrough issue seemed to fail. I was on > FreeBSD 11.1. It may be due to this (and also my issue which was on AMD, may be due to a reboot where I didn't enable the iommu). If that is the case, the error handling could certainly be improved :( You can check to see if there is an IOMMU available by looking for the APCI DMAR table: acpidump -t | grep DMAR later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Sep 7 18:23:40 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4279CE06F40 for ; Thu, 7 Sep 2017 18:23:40 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 80F5B6E1C4 for ; Thu, 7 Sep 2017 18:23:39 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 9EB31210A291 for ; Fri, 8 Sep 2017 04:23:36 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 9566A281F24 for ; Fri, 8 Sep 2017 04:23:36 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ELzF5bFRFB6G for ; Fri, 8 Sep 2017 04:23:36 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id DB63D2804DB; Fri, 8 Sep 2017 04:23:34 +1000 (AEST) Subject: Re: bhyve passthrough fails To: "Rodney W. Grimes" Cc: Farhan Khan , freebsd-virtualization@freebsd.org References: <201709071558.v87FwJ5x027614@pdx.rh.CN85.dnsmgr.net> From: Peter Grehan Message-ID: <78664ef3-45f5-348d-12ec-718bfe2a41ca@freebsd.org> Date: Thu, 7 Sep 2017 11:23:31 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <201709071558.v87FwJ5x027614@pdx.rh.CN85.dnsmgr.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=IkcTkHD0fZMA:10 a=2JCJgTwv5E4A:10 a=WubxpbWX3sDj2u6O1tUA:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 18:23:40 -0000 Hi Rod, >> Hi Farhan, >>> bhyve: PCI device at 2/0/0 is not using the ppt(4) driver. >> >> Just noticed this myself: looks like a regression :( I'll see if I can >> work it out and get a fix in. > > What is the state of bhyve in 10.4? Do we need to fire off some > quick tests???? Always worth some quick tests before a release. later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Sep 7 23:55:46 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDA38E189F4; Thu, 7 Sep 2017 23:55:46 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46A7676784; Thu, 7 Sep 2017 23:55:46 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-oi0-x231.google.com with SMTP id z73so3513665oia.2; Thu, 07 Sep 2017 16:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=WEro7CWTJ7s4KA7SB2uo6o/FZGLSBFBW7YhCtmCL2uE=; b=YcK60NDGZaLfcWHTm3dH1ACWaBhAxEOHiQ5A9gduJQWyDImVxmiSTOVki5vQjZZpWQ KZLrEfMn3PGBP4zgWwkAfgsKuR59va67xkDNhSHgG0hpW2ctOb98bzD3gnJszr5f0Qg5 8w6XjNz3zeaGI7aS5ryc370TGjn2oL8SYSMFrz2DrCidQh+3tD9fhHDv7W8Zfzcu5n4g 9SV3q9kkviNcLNwvS0kTwIH/FHIwC4rd0UQ0nOKqCeRQOEfx5kWQYtK/BSdR3hG7hVTG HuKyEWDUy5JAOZsI7LsTb7K3ZuLU4yYEuJiD8nHSPeXln3xu/Bb/BXylG/npjzUiqZyo ksYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=WEro7CWTJ7s4KA7SB2uo6o/FZGLSBFBW7YhCtmCL2uE=; b=K0C3DBtOGGneb53aVk99y7WYqLG31qBlKQ7zyh+WgXiUvThFaIf6jIoHnlRHJO3ggY eu1xx5JOlPvn5HZ0UOjqvixUbcTW/9bJt7D5sF6vNw3rWCDHKlnffmLPFvtKWr7SsmGf j8XdHakxm65COBum3AT6aGIoIlMOinYdUdpwFjPbwYEo04o/FZ+p5R58gxyxaebEh7EH +LSuvOpV9Qfjg852NFHU0B9KP4yuVuW846GqaBHYbFGkcZ9p44vSpLL1Ky3MLvOGMiUU jpzAGBPh2d7z2g3UpCkorA+nihdceo1pwLCE/N497mpvEJVVt43jjecGq/WDQ06fCWb9 +xkw== X-Gm-Message-State: AHPjjUjntG6Awl9KcE4RhCG6IyR7wmJKdU+EoLY3XT1UpqAw7WeS3gTM +6k1RGaeFIes5X5lEDSTfXM+i5jBD3B0 X-Google-Smtp-Source: ADKCNb5uQzRFfteIlIvorDZjUNZU/DUmPll3ZkE7gXxarVetMUbVxdFcWnNf17Vld5Zwv8UfpfrfoUya6LTvzu1HIa0= X-Received: by 10.202.208.92 with SMTP id h89mr1379542oig.90.1504828545247; Thu, 07 Sep 2017 16:55:45 -0700 (PDT) MIME-Version: 1.0 Sender: jtubnor@gmail.com Received: by 10.157.42.114 with HTTP; Thu, 7 Sep 2017 16:55:24 -0700 (PDT) In-Reply-To: References: From: Jason Tubnor Date: Fri, 8 Sep 2017 09:55:24 +1000 X-Google-Sender-Auth: ZfIgWrZt1QxaeA1bSC0y0y9RZXU Message-ID: Subject: Re: openbsd 6.0 or 6.1 guest on freebsd-12 (current) host To: Alan Somers Cc: tech-lists , "freebsd-virtualization@freebsd.org" , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 23:55:47 -0000 On 8 September 2017 at 00:28, Alan Somers wrote: > > > Do you have a semiautomated way to move bhyve instances between > different hosts? Without such a feature, we can't replace all of our > ESXi machines. > We are not using bhyve for those workloads (yet). Our vsphere environment still has life left in it but we never want to bet on one horse (I've been in too many positions where vendor lock has occurred and its been painful and expensive). However, where there are single hosts instances out at remote branches, it is bhyve all the way where we use to use individual instances of ESXi. So for your use case, it isn't there yet, but, if you don't need live migration, using zfs and/or iSCSI you could tool something together for offline migration of guests between hosts. Allowing you to get closer to the operating system (unlike the abstraction that is vsphere/ESXi), gives you the ability to take the excellent tools that FreeBSD has to offer and make them fit your application. From owner-freebsd-virtualization@freebsd.org Fri Sep 8 16:45:13 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 52C27E21C8F for ; Fri, 8 Sep 2017 16:45:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4141C27B5 for ; Fri, 8 Sep 2017 16:45:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v88GjDrW084394 for ; Fri, 8 Sep 2017 16:45:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222115] vagrant image sudo segfaults Date: Fri, 08 Sep 2017 16:45:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 16:45:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222115 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Sep 9 23:03:15 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BEF4E1047C for ; Sat, 9 Sep 2017 23:03:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 19E51813B4 for ; Sat, 9 Sep 2017 23:03:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v89N3EKN057898 for ; Sat, 9 Sep 2017 23:03:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Sat, 09 Sep 2017 23:03:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: greg@unrelenting.technology X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Sep 2017 23:03:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 Greg V changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |greg@unrelenting.technology --- Comment #5 from Greg V --- (In reply to Sepherosa Ziehau from comment #4) Gen2 VM is based on UEFI and SCSI instead of BIOS and IDE. As a desktop use= r, I only want to use modern systems, not legacy :) The thought of using emulated IDE drives is terrifying. (VirtualBox emulates NVMe, dammit!) I actually recommend using VcXsrv and SSH X forwarding via PuTTY. I have a little PowerShell script to launch the VM and run X apps from the VM on the Windows desktop: https://github.com/myfreeweb/xvmmgr Still, why exactly is the Hyper-V console "_not_ suitable for X related stuffs"? --=20 You are receiving this mail because: You are the assignee for the bug.=