From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 5 01:34:22 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31C24FB3 for ; Sun, 5 Apr 2015 01:34:22 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD9F26B for ; Sun, 5 Apr 2015 01:34:21 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id A5012280F98 for ; Sun, 5 Apr 2015 11:34:12 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fCIcqRL1KCsU for ; Sun, 5 Apr 2015 11:34:12 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-76-126-65-88.hsd1.ca.comcast.net [76.126.65.88]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id DCB0F280F90; Sun, 5 Apr 2015 11:34:10 +1000 (EST) Message-ID: <5520910F.3050607@freebsd.org> Date: Sat, 04 Apr 2015 18:34:07 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Aryeh Friedman Subject: Re: odd output from bhyve References: <5520193C.2010101@freebsd.org> <55203A7B.9020703@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 01:34:22 -0000 Hi Aryeh, > /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () > Remark 2095 - > Effective AML package length is zero ^ This looks like a bug in bhyve if there is no PCI-ISA bridge. This can be fixed by changing the syntax for the serial port from a PCI serial device: > -s 31,uart ... to a serial port behind the PCI-ISA bridge: -s 31,lpc -l com1,stdio ... which is now the recommended way to configure a serial console. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sun Apr 5 03:20:47 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4816045D for ; Sun, 5 Apr 2015 03:20:47 +0000 (UTC) Received: from fallback8.mail.ru (fallback8.mail.ru [94.100.181.110]) by mx1.freebsd.org (Postfix) with ESMTP id 855CFD6A for ; Sun, 5 Apr 2015 03:20:45 +0000 (UTC) Received: from f409.i.mail.ru (f409.i.mail.ru [185.5.136.80]) by fallback8.mail.ru (mPOP.Fallback_MX) with ESMTP id 86CC980F0440 for ; Sun, 5 Apr 2015 06:20:36 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ua; s=mail; h=Content-Type:Message-ID:Reply-To:Date:MIME-Version:Subject:To:From; bh=pJiWryqAhAoc4gJCsHBcfepaTC9beFKl/bPCWempQKk=; b=paHWf4+1kbP2Vabkn9qZjRlVDadY5FJbmW10mNAgQbsmATP3xyYUi9IdevN/ICedzDoxA1wFtASukrZbe0lsF2s9TYpO4SZQpyGqmw8kpPxlvhfsYRDoTKc3cG/32CrD3yYdoO8I6uXxbsIufM6YosKtFXHyNdTksZB2vZUOt2g=; Received: from [195.20.154.36] (ident=mail) by f409.i.mail.ru with local (envelope-from ) id 1Yeb6p-0000KC-Iz for freebsd-virtualization@freebsd.org; Sun, 05 Apr 2015 06:20:27 +0300 Received: from [195.20.154.36] by e.mail.ru with HTTP; Sun, 05 Apr 2015 06:20:27 +0300 From: =?UTF-8?B?0JXQstCz0LXQvdC40LkgSVJPTg==?= To: freebsd-virtualization@freebsd.org Subject: =?UTF-8?B?Ymh5dmU6IEFib3J0IHRyYXA=?= MIME-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [195.20.154.36] Date: Sun, 05 Apr 2015 06:20:27 +0300 Reply-To: =?UTF-8?B?0JXQstCz0LXQvdC40LkgSVJPTg==?= X-Priority: 3 (Normal) Message-ID: <1428204027.328665395@f409.i.mail.ru> X-Mras: Ok X-Spam: undefined Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 03:20:47 -0000 IEhlbGxvIGFsbCwKCkFmdGVyIEkgdXBncmFkZWQgRnJlZUJTRCBTVEFCTEUgdXAgdG/CoHIyODEw OTUsIEkgY2FuJ3QgYm9vdCB1cCBWTSB3aXRoIENlbnRPUyA2LgoKIyB1bmFtZSAtYQpGcmVlQlNE IGNvYnJhIDEwLjEtU1RBQkxFIEZyZWVCU0QgMTAuMS1TVEFCTEUgIzggcjI4MTA5NTogU3VuIEFw ciA1IDA1OjIwOjU3IEVFU1QgMjAxNSByb290QGNvYnJhOi91c3Ivb2JqL3Vzci9zcmMvc3lzL0NP QlJBIGFtZDY0CiMgY2F0IGJpdHJpeC5ncnViCmxpbnV4IC92bWxpbnV6LTIuNi4zMi01MDQuMTIu Mi5lbDYueDg2XzY0IHJvIHJvb3Q9L2Rldi9tYXBwZXIvVm9sR3JvdXAtbHZfcm9vdCByZF9OT19M VUtTIExBTkc9ZW5fVVMuVVRGLTggcmRfTk9fTUQgS0VZVEFCTEU9dXMgcmRfTFZNX0xWPVZvbEdy b3VwL2x2X3N3YXAgU1lTRk9OVD1sYXRhcmN5cmhlYi1zdW4xNiBjb25zb2xlPXR0eVMwIGNyYXNo a2VybmVsPWF1dG8gcmRfTFZNX0xWPVZvbEdyb3VwL2x2X3Jvb3QgcmRfTk9fRE0KaW5pdHJkIC9p bml0cmFtZnMtMi42LjMyLTUwNC4xMi4yLmVsNi54ODZfNjQuaW1nCmJvb3QKCiMgY2F0IGJpdHJp eF9zdGFydC5zaAojIS91c3IvbG9jYWwvYmluL2Jhc2gKwqAKaWZjb25maWcgdGFwMCBtdHUgOTAw MAppZmNvbmZpZyBicmlkZ2UwIGRlbGV0ZW0gZW0wCmlmY29uZmlnIGJyaWRnZTAgYWRkbSBlbTAg YWRkbSB0YXAwCmJoeXZlY3RsIC0tZGVzdHJveSAtLXZtPWJpdHJpeCAmPiAvZGV2L251bGwKZ3J1 Yi1iaHl2ZSAtbSBkZXZpY2UubWFwIC1yIGhkMCxtc2RvczEgLU0gNDA5Nk0gYml0cml4IDwgYml0 cml4LmdydWIKYmh5dmUgLWMgNCAtQUkgLUggLVAgLXMgMDowLGhvc3RicmlkZ2UgLXMgMTowLGxw YyAtcyAyOjAsdmlydGlvLW5ldCx0YXAwIC1zIDM6MCx2aXJ0aW8tYmxrLC9kZXYvenZvbC96cm9v dC9iaXRyaXggLWwgY29tMSxzdGRpbyAtbSA0MDk2TSBiaXRyaXgKCgpDb25zb2xlIG91dHB1dDoK Li4uLi4uLgpDUFU6IFBoeXNpY2FsIFByb2Nlc3NvciBJRDogMAphbHRlcm5hdGl2ZXM6IHN3aXRj aGluZyB0byB1bmZhaXIgc3BpbmxvY2sKQUNQSTogQ29yZSByZXZpc2lvbiAyMDA5MDkwMwpBQ1BJ IEV4Y2VwdGlvbjogQUVfTk9fQUNQSV9UQUJMRVMsIFdoaWxlIGxvYWRpbmcgbmFtZXNwYWNlIGZy b20gQUNQSSB0YWJsZXMgKDIwMDkwOTAzL3RieGZhY2UtNjA1KQpBQ1BJOiBVbmFibGUgdG8gbG9h ZCB0aGUgU3lzdGVtIERlc2NyaXB0aW9uIFRhYmxlcwpmdHJhY2U6IGNvbnZlcnRpbmcgbWNvdW50 IGNhbGxzIHRvIDBmIDFmIDQ0IDAwIDAwCmZ0cmFjZTogYWxsb2NhdGluZyAyMTkxOSBlbnRyaWVz IGluIDg2IHBhZ2VzCkFQSUMgcm91dGluZyBmaW5hbGl6ZWQgdG8gZmxhdC4KLi5USU1FUjogdmVj dG9yPTB4MzAgYXBpYzE9MCBwaW4xPTAgYXBpYzI9LTEgcGluMj0tMQouLk1QLUJJT1MgYnVnOiA4 MjU0IHRpbWVyIG5vdCBjb25uZWN0ZWQgdG8gSU8tQVBJQwouLi50cnlpbmcgdG8gc2V0IHVwIHRp bWVyIChJUlEwKSB0aHJvdWdoIHRoZSA4MjU5QSAuLi4KLi4uLi4gKGZvdW5kIGFwaWMgMCBwaW4g MCkgLi4uCi4uLi4uLi4gZmFpbGVkLgouLi50cnlpbmcgdG8gc2V0IHVwIHRpbWVyIGFzIFZpcnR1 YWwgV2lyZSBJUlEuLi4KLi4uLi4gZmFpbGVkLgouLi50cnlpbmcgdG8gc2V0IHVwIHRpbWVyIGFz IEV4dElOVCBJUlEuLi4KVW5oYW5kbGVkIG91dGIgMHgwMDcxCi4vYml0cml4X3N0YXJ0LnNoOiBs aW5lIDc6IDUyMDIgQWJvcnQgdHJhcCAoY29yZSBkdW1wZWQpIGJoeXZlIC1jIDQgLUFJIC1IIC1Q IC1zIDA6MCxob3N0YnJpZGdlIC1zIDE6MCxscGMgLXMgMjowLHZpcnRpby1uZXQsdGFwMCAtcyAz OjAsdmlydGlvLWJsaywvZGV2L3p2b2wvenJvb3QvYml0cml4IC1sIGNvbTEsc3RkaW8gLW0gNDA5 Nk0gYml0cml4ClRlbGwgbWUgcGxlYXNlLCBob3cgY2FuIEkgZml4IHRoaXMgcHJvYmxlbT8KSSBj YW4gcmVidWlsZCBrZXJuZWwgd2l0aCBkZWJ1ZyBlbmFibGVkIGFuZCBtYWtlIGJhY2t0cmFjZSBv ZiBjb3JlIGR1bXAgaWYgaXQgd2lsbCBoZWxwIHNvbWVob3cgdG8gZmluZCBhIHJlYXNvbi4gSSBn dWVzc8KgIDI4MTA3NcKgIGNvbW1pdCBpcyB0aGUgcmVhc29uLgoKVGhhbmtzIGEgbG90IQ== From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 6 03:25:40 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5333FC4 for ; Mon, 6 Apr 2015 03:25:39 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 799B87BB for ; Mon, 6 Apr 2015 03:25:39 +0000 (UTC) Received: by wgyo15 with SMTP id o15so6452858wgy.2 for ; Sun, 05 Apr 2015 20:25:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=bN2NpYvkTLLT77PmCBtDoIeL0k7TdbB9dt2zKm0mBeo=; b=NaPXcHk23ayCurhFlG37BkPM+xRcKWbIX0ir9EhVoC3eDja61Bdga9OFDms6vWdV/w VSaARQdpMTdhxJyzbsGLBh79pKheu7cuha9uPGkvFk14TYmJQW3vVDVueRTv2OZoKKS4 t5lpLtvd8fAbd/MIr8N5g1wtAvwU/baSPtChnFbBtf5iPpsisuqtZAr/9caTlb/TJHqR qoE+WLrFl9kbnIUjLFNh1zky84vNbUdVUlbcQUwKcx3L96JBH36eC0K8Zp06vd6cKpyw 3KDEqK9f05cKnQdUouxdAWwogpqd726dq6Wc5T+az1WbRbPu4+0yBU6qedSQfIiBcpj2 YT4g== MIME-Version: 1.0 X-Received: by 10.194.94.1 with SMTP id cy1mr26470264wjb.127.1428290738009; Sun, 05 Apr 2015 20:25:38 -0700 (PDT) Received: by 10.27.9.9 with HTTP; Sun, 5 Apr 2015 20:25:37 -0700 (PDT) In-Reply-To: <1428204027.328665395@f409.i.mail.ru> References: <1428204027.328665395@f409.i.mail.ru> Date: Sun, 5 Apr 2015 20:25:37 -0700 Message-ID: Subject: Re: bhyve: Abort trap From: Neel Natu To: =?UTF-8?B?0JXQstCz0LXQvdC40LkgSVJPTg==?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 03:25:40 -0000 Hi, On Sat, Apr 4, 2015 at 8:20 PM, =D0=95=D0=B2=D0=B3=D0=B5=D0=BD=D0=B8=D0=B9 = IRON wrote: > Hello all, > > After I upgraded FreeBSD STABLE up to r281095, I can't boot up VM with Ce= ntOS 6. > > # uname -a > FreeBSD cobra 10.1-STABLE FreeBSD 10.1-STABLE #8 r281095: Sun Apr 5 05:20= :57 EEST 2015 root@cobra:/usr/obj/usr/src/sys/COBRA amd64 > # cat bitrix.grub > linux /vmlinuz-2.6.32-504.12.2.el6.x86_64 ro root=3D/dev/mapper/VolGroup-= lv_root rd_NO_LUKS LANG=3Den_US.UTF-8 rd_NO_MD KEYTABLE=3Dus rd_LVM_LV=3DVo= lGroup/lv_swap SYSFONT=3Dlatarcyrheb-sun16 console=3DttyS0 crashkernel=3Dau= to rd_LVM_LV=3DVolGroup/lv_root rd_NO_DM > initrd /initramfs-2.6.32-504.12.2.el6.x86_64.img > boot > > # cat bitrix_start.sh > #!/usr/local/bin/bash > > ifconfig tap0 mtu 9000 > ifconfig bridge0 deletem em0 > ifconfig bridge0 addm em0 addm tap0 > bhyvectl --destroy --vm=3Dbitrix &> /dev/null > grub-bhyve -m device.map -r hd0,msdos1 -M 4096M bitrix < bitrix.grub > bhyve -c 4 -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 = -s 3:0,virtio-blk,/dev/zvol/zroot/bitrix -l com1,stdio -m 4096M bitrix > > > Console output: > ....... > CPU: Physical Processor ID: 0 > alternatives: switching to unfair spinlock > ACPI: Core revision 20090903 > ACPI Exception: AE_NO_ACPI_TABLES, While loading namespace from ACPI tabl= es (20090903/tbxface-605) > ACPI: Unable to load the System Description Tables > ftrace: converting mcount calls to 0f 1f 44 00 00 > ftrace: allocating 21919 entries in 86 pages > APIC routing finalized to flat. > ..TIMER: vector=3D0x30 apic1=3D0 pin1=3D0 apic2=3D-1 pin2=3D-1 > ..MP-BIOS bug: 8254 timer not connected to IO-APIC > ...trying to set up timer (IRQ0) through the 8259A ... > ..... (found apic 0 pin 0) ... > ....... failed. > ...trying to set up timer as Virtual Wire IRQ... > ..... failed. > ...trying to set up timer as ExtINT IRQ... > Unhandled outb 0x0071 > ./bitrix_start.sh: line 7: 5202 Abort trap (core dumped) bhyve -c 4 -AI -= H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,= /dev/zvol/zroot/bitrix -l com1,stdio -m 4096M bitrix > Tell me please, how can I fix this problem? > I can rebuild kernel with debug enabled and make backtrace of core dump i= f it will help somehow to find a reason. I guess 281075 commit is the rea= son. > > Thanks a lot! I am surprised that you even got that far. I was getting ACPI compilation failures in bhyve after r281075 upgraded the ACPICA in stable/10. This is fixed in r281134. Can you try running your guest after syncing past this change? best Neel > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 6 05:11:55 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0CE4B929 for ; Mon, 6 Apr 2015 05:11:55 +0000 (UTC) Received: from fallback3.mail.ru (fallback3.mail.ru [94.100.181.189]) by mx1.freebsd.org (Postfix) with ESMTP id 5652829 for ; Mon, 6 Apr 2015 05:11:53 +0000 (UTC) Received: from f177.i.mail.ru (f177.i.mail.ru [94.100.178.94]) by fallback3.mail.ru (mPOP.Fallback_MX) with ESMTP id 8D3B614AD57D4 for ; Mon, 6 Apr 2015 08:09:01 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ua; s=mail; h=References:In-Reply-To:Content-Type:Message-ID:Reply-To:Date:MIME-Version:Subject:Cc:To:From; bh=DPrDZmyxkkQnj1Zqjqjvkj7hHXbDfKbrELoIih84Yzc=; b=Q6Si5R1Nq7ssNKO8G+ulew68Bw1fUlzQtTCPxgJ6xKi1BTMVGsDMTi4QOS3iB+AJmHtGbhUCdx6hfojyF0xSYRgGnmuRv7KFN+pzFevRPrU9nDDt4REqeayPRWmx+X+39PmXJ05dvRG0uJj6eObQIk/VXi1h48Q56DEumHpB4yk=; Received: from [195.20.154.36] (ident=mail) by f177.i.mail.ru with local (envelope-from ) id 1YezHI-0006Wt-Bd; Mon, 06 Apr 2015 08:08:52 +0300 Received: from [195.20.154.36] by e.mail.ru with HTTP; Mon, 06 Apr 2015 08:08:52 +0300 From: =?UTF-8?B?0JXQstCz0LXQvdC40LkgSVJPTg==?= To: =?UTF-8?B?TmVlbCBOYXR1?= Subject: =?UTF-8?B?UmVbMl06IGJoeXZlOiBBYm9ydCB0cmFw?= MIME-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [195.20.154.36] Date: Mon, 06 Apr 2015 08:08:52 +0300 Reply-To: =?UTF-8?B?0JXQstCz0LXQvdC40LkgSVJPTg==?= X-Priority: 3 (Normal) Message-ID: <1428296932.527195236@f177.i.mail.ru> X-Mras: Ok X-Spam: undefined In-Reply-To: References: <1428204027.328665395@f409.i.mail.ru> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: =?UTF-8?B?ZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBmcmVlYnNkLm9yZw==?= X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 05:11:55 -0000 IEhlbGxvIE5lZWwsCgpDYW4gY29uZmlybSB0aGF0IHRoZSBpc3N1dWUgZml4ZWQgaW7CoHIyODEx MzQuCgpUaGFua3MgeW91IQoKCtCS0L7RgdC60YDQtdGB0LXQvdGM0LUsICA1INCw0L/RgNC10LvR jyAyMDE1LCAyMDoyNSAtMDc6MDAg0L7RgiBOZWVsIE5hdHUgPG5lZWxuYXR1QGdtYWlsLmNvbT46 Cj5IaSwKPgo+T24gU2F0LCBBcHIgNCwgMjAxNSBhdCA4OjIwIFBNLCDQldCy0LPQtdC90LjQuSBJ Uk9OIDwgaXJvbkBtYWlsLnVhID4gd3JvdGU6Cj4+ICBIZWxsbyBhbGwsCj4+Cj4+IEFmdGVyIEkg dXBncmFkZWQgRnJlZUJTRCBTVEFCTEUgdXAgdG8gcjI4MTA5NSwgSSBjYW4ndCBib290IHVwIFZN IHdpdGggQ2VudE9TIDYuCj4+Cj4+ICMgdW5hbWUgLWEKPj4gRnJlZUJTRCBjb2JyYSAxMC4xLVNU QUJMRSBGcmVlQlNEIDEwLjEtU1RBQkxFICM4IHIyODEwOTU6IFN1biBBcHIgNSAwNToyMDo1NyBF RVNUIDIwMTUgcm9vdEBjb2JyYTovdXNyL29iai91c3Ivc3JjL3N5cy9DT0JSQSBhbWQ2NAo+PiAj IGNhdCBiaXRyaXguZ3J1Ygo+PiBsaW51eCAvdm1saW51ei0yLjYuMzItNTA0LjEyLjIuZWw2Lng4 Nl82NCBybyByb290PS9kZXYvbWFwcGVyL1ZvbEdyb3VwLWx2X3Jvb3QgcmRfTk9fTFVLUyBMQU5H PWVuX1VTLlVURi04IHJkX05PX01EIEtFWVRBQkxFPXVzIHJkX0xWTV9MVj1Wb2xHcm91cC9sdl9z d2FwIFNZU0ZPTlQ9bGF0YXJjeXJoZWItc3VuMTYgY29uc29sZT10dHlTMCBjcmFzaGtlcm5lbD1h dXRvIHJkX0xWTV9MVj1Wb2xHcm91cC9sdl9yb290IHJkX05PX0RNCj4+IGluaXRyZCAvaW5pdHJh bWZzLTIuNi4zMi01MDQuMTIuMi5lbDYueDg2XzY0LmltZwo+PiBib290Cj4+Cj4+ICMgY2F0IGJp dHJpeF9zdGFydC5zaAo+PiAjIS91c3IvbG9jYWwvYmluL2Jhc2gKPj4KPj4gaWZjb25maWcgdGFw MCBtdHUgOTAwMAo+PiBpZmNvbmZpZyBicmlkZ2UwIGRlbGV0ZW0gZW0wCj4+IGlmY29uZmlnIGJy aWRnZTAgYWRkbSBlbTAgYWRkbSB0YXAwCj4+IGJoeXZlY3RsIC0tZGVzdHJveSAtLXZtPWJpdHJp eCAmPiAvZGV2L251bGwKPj4gZ3J1Yi1iaHl2ZSAtbSBkZXZpY2UubWFwIC1yIGhkMCxtc2RvczEg LU0gNDA5Nk0gYml0cml4IDwgYml0cml4LmdydWIKPj4gYmh5dmUgLWMgNCAtQUkgLUggLVAgLXMg MDowLGhvc3RicmlkZ2UgLXMgMTowLGxwYyAtcyAyOjAsdmlydGlvLW5ldCx0YXAwIC1zIDM6MCx2 aXJ0aW8tYmxrLC9kZXYvenZvbC96cm9vdC9iaXRyaXggLWwgY29tMSxzdGRpbyAtbSA0MDk2TSBi aXRyaXgKPj4KPj4KPj4gQ29uc29sZSBvdXRwdXQ6Cj4+IC4uLi4uLi4KPj4gQ1BVOiBQaHlzaWNh bCBQcm9jZXNzb3IgSUQ6IDAKPj4gYWx0ZXJuYXRpdmVzOiBzd2l0Y2hpbmcgdG8gdW5mYWlyIHNw aW5sb2NrCj4+IEFDUEk6IENvcmUgcmV2aXNpb24gMjAwOTA5MDMKPj4gQUNQSSBFeGNlcHRpb246 IEFFX05PX0FDUElfVEFCTEVTLCBXaGlsZSBsb2FkaW5nIG5hbWVzcGFjZSBmcm9tIEFDUEkgdGFi bGVzICgyMDA5MDkwMy90YnhmYWNlLTYwNSkKPj4gQUNQSTogVW5hYmxlIHRvIGxvYWQgdGhlIFN5 c3RlbSBEZXNjcmlwdGlvbiBUYWJsZXMKPj4gZnRyYWNlOiBjb252ZXJ0aW5nIG1jb3VudCBjYWxs cyB0byAwZiAxZiA0NCAwMCAwMAo+PiBmdHJhY2U6IGFsbG9jYXRpbmcgMjE5MTkgZW50cmllcyBp biA4NiBwYWdlcwo+PiBBUElDIHJvdXRpbmcgZmluYWxpemVkIHRvIGZsYXQuCj4+IC4uVElNRVI6 IHZlY3Rvcj0weDMwIGFwaWMxPTAgcGluMT0wIGFwaWMyPS0xIHBpbjI9LTEKPj4gLi5NUC1CSU9T IGJ1ZzogODI1NCB0aW1lciBub3QgY29ubmVjdGVkIHRvIElPLUFQSUMKPj4gLi4udHJ5aW5nIHRv IHNldCB1cCB0aW1lciAoSVJRMCkgdGhyb3VnaCB0aGUgODI1OUEgLi4uCj4+IC4uLi4uIChmb3Vu ZCBhcGljIDAgcGluIDApIC4uLgo+PiAuLi4uLi4uIGZhaWxlZC4KPj4gLi4udHJ5aW5nIHRvIHNl dCB1cCB0aW1lciBhcyBWaXJ0dWFsIFdpcmUgSVJRLi4uCj4+IC4uLi4uIGZhaWxlZC4KPj4gLi4u dHJ5aW5nIHRvIHNldCB1cCB0aW1lciBhcyBFeHRJTlQgSVJRLi4uCj4+IFVuaGFuZGxlZCBvdXRi IDB4MDA3MQo+PiAuL2JpdHJpeF9zdGFydC5zaDogbGluZSA3OiA1MjAyIEFib3J0IHRyYXAgKGNv cmUgZHVtcGVkKSBiaHl2ZSAtYyA0IC1BSSAtSCAtUCAtcyAwOjAsaG9zdGJyaWRnZSAtcyAxOjAs bHBjIC1zIDI6MCx2aXJ0aW8tbmV0LHRhcDAgLXMgMzowLHZpcnRpby1ibGssL2Rldi96dm9sL3py b290L2JpdHJpeCAtbCBjb20xLHN0ZGlvIC1tIDQwOTZNIGJpdHJpeAo+PiBUZWxsIG1lIHBsZWFz ZSwgaG93IGNhbiBJIGZpeCB0aGlzIHByb2JsZW0/Cj4+IEkgY2FuIHJlYnVpbGQga2VybmVsIHdp dGggZGVidWcgZW5hYmxlZCBhbmQgbWFrZSBiYWNrdHJhY2Ugb2YgY29yZSBkdW1wIGlmIGl0IHdp bGwgaGVscCBzb21laG93IHRvIGZpbmQgYSByZWFzb24uIEkgZ3Vlc3MgIDI4MTA3NSAgY29tbWl0 IGlzIHRoZSByZWFzb24uCj4+Cj4+IFRoYW5rcyBhIGxvdCEKPgo+SSBhbSBzdXJwcmlzZWQgdGhh dCB5b3UgZXZlbiBnb3QgdGhhdCBmYXIuIEkgd2FzIGdldHRpbmcgQUNQSQo+Y29tcGlsYXRpb24g ZmFpbHVyZXMgaW4gYmh5dmUgYWZ0ZXIgcjI4MTA3NSB1cGdyYWRlZCB0aGUgQUNQSUNBIGluCj5z dGFibGUvMTAuCj4KPlRoaXMgaXMgZml4ZWQgaW4gcjI4MTEzNC4gQ2FuIHlvdSB0cnkgcnVubmlu ZyB5b3VyIGd1ZXN0IGFmdGVyIHN5bmNpbmcKPnBhc3QgdGhpcyBjaGFuZ2U/Cj4KPmJlc3QKPk5l ZWwKPgo+PiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+ PiAgZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QKPj4gIGh0 dHA6Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtdmlydHVhbGl6 YXRpb24KPj4gVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55IG1haWwgdG8gIiBmcmVlYnNkLXZpcnR1 YWxpemF0aW9uLXVuc3Vic2NyaWJlQGZyZWVic2Qub3JnICIKCgoBCg== From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 6 20:39:02 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 382BEA1F for ; Mon, 6 Apr 2015 20:39:02 +0000 (UTC) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB09D136 for ; Mon, 6 Apr 2015 20:39:01 +0000 (UTC) Received: by lboc7 with SMTP id c7so27922033lbo.1 for ; Mon, 06 Apr 2015 13:38:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=EcCvFXgzPpdHZ5qZNjsyxHQ52pOKIhTbuyXXzAVsj/Q=; b=pEX/kn4QfFeTzls54Z8lRb9O9N+hCrUwwM8WayCI0aBw6CjNdGoxqTBQ+Jfq5ipdzS gQsKHtil7GzqdSudvdA62/AzvaLtu5ancmF5AclATbT5T7b7LdWQOvb125WLgDvsgTB3 +/SXXG0MVX+SU/LleutbgK2WnLu9keZluBH3uO2zQnJdKyjXBosN7/3aHk0PNMscBzZm I11XECXPcSYKgZ88wtjqcyaBbqDuTloZtFbdSue26GbIhl/y9QKcjAKNy1gK2ayUCd8h PO49xJSW8ZTAbgXdlucjU8LW8P1PIfoH9AGAv2qHSIvmx9C0qVIdK57+BbUIoeyKE8Z7 dwkg== X-Received: by 10.112.171.65 with SMTP id as1mr15094322lbc.45.1428352739758; Mon, 06 Apr 2015 13:38:59 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id i9sm1259756lbs.26.2015.04.06.13.38.57 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Apr 2015 13:38:58 -0700 (PDT) Sender: Alexander Motin Message-ID: <5522EEE0.5010807@FreeBSD.org> Date: Mon, 06 Apr 2015 23:38:56 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Julian Hsiao , "freebsd-virtualization@freebsd.org" Subject: Re: Bhyve storage improvements (was: Several bhyve quirks) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 20:39:02 -0000 Hi, Julian. > I had some time to try it out today, but I'm still having issues: I've just made experiment alike to your with making bhyve to work on top of GEOM device instead of preferable "dev" mode of ZVOL. And I indeed reproduced the problem. But the problem that I see is not related to the block size. The block size is reported to the guest correctly as 4K, and as I can see it works as such at least in FreeBSD guest. The problem is in the way how bhyve inter-operates with block/GEOM devices. bhyve sends requests to the kernel with preadv()/pwritev() calls, specifying scatter/gather lists of buffer addresses provided by the guest. But GEOM code can not handle scatter/gather lists, only sequential buffer, and so single request is split into several. The problem is that splitting happens according to scatter/gather elements, and those elements in general case may not be multiple to the block size, that is fatal for GEOM and any block device. I am not yet sure how to fix this problem. The most straightforward way is to copy the data at some point to collect elements of scatter/gather list into something sequential to pass to GEOM, but that requires additional memory allocation, and the copying is not free. May be some cases could be optimized to work without copying but with some clever page mapping, but that seems absolutely not trivial. -- Alexander Motin From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 7 07:32:03 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6FAC85D for ; Tue, 7 Apr 2015 07:32:03 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D551F63 for ; Tue, 7 Apr 2015 07:32:03 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,536,1422921600"; d="scan'208";a="250714949" Message-ID: <552387EA.5020505@citrix.com> Date: Tue, 7 Apr 2015 09:31:54 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Gustau_P=E9rez?= , Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551D892E.3090309@gmail.com> In-Reply-To: <551D892E.3090309@gmail.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-DLP: MIA2 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 07:32:03 -0000 Hello, El 02/04/15 a les 20.23, Gustau Pérez ha escrit: > > > On 01/04/2015 12:41, Roger Pau Monné wrote: >> El 01/04/15 a les 12.30, Udo Rader ha escrit: >>> Hi all, >>> >>> first please excuse if this may be a FAQ, but even though I am a long >>> time linux admin (~1996), I am quite new to the *BSD world and I am >>> trying to evaluate if FreeBSD fits our virtualization needs. >>> >>> So, for my many questions: >>> >>> As far as my homework digging revealed, FreeBSD supports four hypervisors: >>> >>> * bhyve >>> * KVM >>> * QEMU >>> * VirtualBox >> Make that 5: >> * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 > > Hi, > > reading the wiki, I see EPT seems to be mandatory, is there any way > to run without the EPT extension in legacy hardware? Not really, FreeBSD Dom0 runs in PVH mode, which requires EPT at the moment. We could maybe get rid of the EPT requirement, but the IOMMU support is mandatory, and IIRC there's no hardware with IOMMUs but without EPT. Roger. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 7 12:23:11 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 053255B2 for ; Tue, 7 Apr 2015 12:23:11 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C62CB8B0 for ; Tue, 7 Apr 2015 12:23:10 +0000 (UTC) Received: by pddn5 with SMTP id n5so77389105pdd.2 for ; Tue, 07 Apr 2015 05:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=UHBAal154Bj2laG05F2iWYyJPZMKhPnIaLJLmx5nTiU=; b=g1nVUXFzcnPNVP5pUDyBTk5ZwDFOKDahSSThaipod8u5FdIbKlTBIj9GLYtqKavSDw yFiGFJoqTQU5JQj8/esIpG7RfY3OmMvkY8pZGeCP7DL7ERNeiaBGMSUIPAy6TJKHOp0I nWSkyr49F60bl5WpkkqIqGLaVPD1vJXlXhzmFXakvbGI3D8Vbmvm8Nzzo4BrpYbgg2k4 CkwSCwE6fveOcPFVyTo1dj+v2XbdoJ41Mb4kmYjhc6YjQI0Pmg4Tv70IZ59URgMMALw4 m/t3HnL7nqvo0m/IBOAFHBOX8I3feO4cdbmMP54L80XKxmUQ9RbLYu4rv8vlEVttFjNJ hEeQ== X-Received: by 10.70.134.198 with SMTP id pm6mr36418205pdb.17.1428409390456; Tue, 07 Apr 2015 05:23:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.183.41 with HTTP; Tue, 7 Apr 2015 05:22:50 -0700 (PDT) In-Reply-To: References: From: Stacey Pellegrino Date: Tue, 7 Apr 2015 13:22:50 +0100 Message-ID: Subject: Fwd: Bhyve port to Gentoo Linux/FreeBSD and support for AMD64 with PCI pass through... To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 12:23:11 -0000 Folks... please help. ---------- Forwarded message ---------- From: Stacey Pellegrino Date: 7 April 2015 at 13:04 Subject: Bhyve port to Gentoo Linux/FreeBSD and support for AMD64 with PCI pass through... To: bsd , recruiters , "Justin (jlec)" , Javier Villavicencio , zfs-discuss@zfsonlinux.org Hey folks, Can someone provide me (or forward onto the relevant person/team) an update on the Bhyve development? I have been trying ever so hard to find out about the latest AMD Catalyst graphics drivers for FreeBSD (rumours on the internet that they exist and/or are in development) so that I can run multiple displays from my Sapphire Radeon R9 290X. It could still be possible to use this graphics card with FreeBSD 10.1 by means of PCI pass through with Bhyve, although (as I found out with extensive testing with Xen) it is Intel processor favored and supported. When is AMD64 support coming up to the level of development that Intel processors are getting? When will Windows and Mac OS X Snow Leopard onwards be supported? Are installations headless and require something like VNC? Is someone working on a port of Bhyve to Gentoo Linux and also Gentoo FreeBSD too? As always, thoughts/comments most welcome. -stacepellegrino From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 7 21:58:35 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B576DD2 for ; Tue, 7 Apr 2015 21:58:35 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F83CF4 for ; Tue, 7 Apr 2015 21:58:35 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 9E271280F9C for ; Wed, 8 Apr 2015 07:58:26 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UdnjJ1oSL3ZT for ; Wed, 8 Apr 2015 07:58:26 +1000 (EST) Received: from Peters-MacBook-Pro.local (unknown [64.245.0.210]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 78C842810AB; Wed, 8 Apr 2015 07:58:24 +1000 (EST) Message-ID: <552452FE.4040209@freebsd.org> Date: Tue, 07 Apr 2015 14:58:22 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Stacey Pellegrino Subject: Re: Fwd: Bhyve port to Gentoo Linux/FreeBSD and support for AMD64 with PCI pass through... References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 21:58:35 -0000 Hi Stacey, > Can someone provide me (or forward onto the relevant person/team) an update > on the Bhyve development? I have been trying ever so hard to find out about > the latest AMD Catalyst graphics drivers for FreeBSD (rumours on the > internet that they exist and/or are in development) so that I can run > multiple displays from my Sapphire Radeon R9 290X. It could still be > possible to use this graphics card with FreeBSD 10.1 by means of PCI pass > through with Bhyve, although (as I found out with extensive testing with > Xen) it is Intel processor favored and supported. When is AMD64 support > coming up to the level of development that Intel processors are getting? AMD processor support in bhyve is almost at the level of Intel processor support, with one notable gap being the lack of IOMMU support which is required for PCI passthru. > When will Windows and Mac OS X Snow Leopard onwards be supported? Windows is (slowly) being worked on. Nothing is happening with OSX to my knowledge. > Are installations headless and require something like VNC? Installations are currently via the guest's serial console. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 8 08:35:32 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A91C9CEF; Wed, 8 Apr 2015 08:35:32 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 757C4AE; Wed, 8 Apr 2015 08:35:32 +0000 (UTC) Received: by pdbnk13 with SMTP id nk13so108216996pdb.0; Wed, 08 Apr 2015 01:35:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=g8vvMgUtzLyb77hjX4+3nRIRwD4xhIXV4zjOG29Fs8A=; b=D5TsSC2D3ms7vD2dyLF1CVu1nBaFOtS4jbh8OvfNoL+Qcksy4xKsMVf8spq0CYDEDp yPZolN1GURnXowEAwUERU3eNxMCrAFzM4RnB3BbFoiEEc0X6F6dW1mJuGtmIUw4+rrPh tG55sBhXYIDjtqT5n8TslkAbZf8vEw21YRL75vVyvZB5YlQqP+WhwjekrpAhAL5JBD5O QWW7i8MUaK1jIU7vs5OiyFa+GGpd9h+6s+YPausl1kl0dIXBRuH+jjrR3gHQcfkEdPQa ta0rGGrj3nw1S/LQHBIFHk+vyHhYaFaox4bw3F707N3jCLX7X13NBO3PqyTUjqPLaSzF zTWQ== X-Received: by 10.66.154.162 with SMTP id vp2mr43731172pab.73.1428482131993; Wed, 08 Apr 2015 01:35:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.183.41 with HTTP; Wed, 8 Apr 2015 01:35:11 -0700 (PDT) In-Reply-To: <552452FE.4040209@freebsd.org> References: <552452FE.4040209@freebsd.org> From: Stacey Pellegrino Date: Wed, 8 Apr 2015 09:35:11 +0100 Message-ID: Subject: Re: Fwd: Bhyve port to Gentoo Linux/FreeBSD and support for AMD64 with PCI pass through... To: Peter Grehan Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 08:35:32 -0000 Thanks for the response Peter. Please email me and/or put me on a mailing list when AMD PCI passthru is supported as this would be much needed for my existing hardware investment and currently unusable graphics card on FreeBSD. Thanks. On 7 April 2015 at 22:58, Peter Grehan wrote: > Hi Stacey, > > Can someone provide me (or forward onto the relevant person/team) an >> update >> on the Bhyve development? I have been trying ever so hard to find out >> about >> the latest AMD Catalyst graphics drivers for FreeBSD (rumours on the >> internet that they exist and/or are in development) so that I can run >> multiple displays from my Sapphire Radeon R9 290X. It could still be >> possible to use this graphics card with FreeBSD 10.1 by means of PCI pass >> through with Bhyve, although (as I found out with extensive testing with >> Xen) it is Intel processor favored and supported. When is AMD64 support >> coming up to the level of development that Intel processors are getting? >> > > AMD processor support in bhyve is almost at the level of Intel processor > support, with one notable gap being the lack of IOMMU support which is > required for PCI passthru. > > When will Windows and Mac OS X Snow Leopard onwards be supported? >> > > Windows is (slowly) being worked on. Nothing is happening with OSX to my > knowledge. > > Are installations headless and require something like VNC? >> > > Installations are currently via the guest's serial console. > > later, > > Peter. > > From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 8 10:40:59 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACB13AFB; Wed, 8 Apr 2015 10:40:59 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 784D5BB; Wed, 8 Apr 2015 10:40:59 +0000 (UTC) Received: by pdbnk13 with SMTP id nk13so111429597pdb.0; Wed, 08 Apr 2015 03:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=H1FFK6B4TBUL4hrCE/1Xq8VdL9NXvA9EtcdXszrDVL8=; b=mi94i38utE8RbrSgMqNVpxhQcO+6bjI73aRmBdQPJZ0449JffpXs+rN5S2SIBKALoQ on1ctjtvCqHRtlTcWzFBjpp6gN73fxUu6LAYq1s0H8RDI13tdOBjGIfDU4zES1nR/T/B 2DbvYbMz3AMT8lQL5t6h19YzoQgCJN5j2aNiaqH59zUrSxH7tw8QmVKKIHwkQ23aIy+M Ij5dlPRA0xqKV0UEFDEOPNdBitUW0F8aA1DCzkyA5I7HrI0fb2qhePZPMN1siJowOi2z XMriSQMzA1j+FI2gg+Xxl155KdtFNlTs3IqZTRdlVaRtH4Dd17iWHSlM1CcTL/B4u8SX Nnww== X-Received: by 10.66.55.42 with SMTP id o10mr44962499pap.148.1428489659039; Wed, 08 Apr 2015 03:40:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.183.41 with HTTP; Wed, 8 Apr 2015 03:40:38 -0700 (PDT) In-Reply-To: References: <552452FE.4040209@freebsd.org> From: Stacey Pellegrino Date: Wed, 8 Apr 2015 11:40:38 +0100 Message-ID: Subject: Re: Fwd: Bhyve port to Gentoo Linux/FreeBSD and support for AMD64 with PCI pass through... To: Peter Grehan Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 10:40:59 -0000 When is work on IOMMU support for AMD processors to enable PCI passthru expected to be completed? On 8 April 2015 at 09:35, Stacey Pellegrino wrote: > Thanks for the response Peter. Please email me and/or put me on a mailing > list when AMD PCI passthru is supported as this would be much needed for my > existing hardware investment and currently unusable graphics card on > FreeBSD. Thanks. > > On 7 April 2015 at 22:58, Peter Grehan wrote: > >> Hi Stacey, >> >> Can someone provide me (or forward onto the relevant person/team) an >>> update >>> on the Bhyve development? I have been trying ever so hard to find out >>> about >>> the latest AMD Catalyst graphics drivers for FreeBSD (rumours on the >>> internet that they exist and/or are in development) so that I can run >>> multiple displays from my Sapphire Radeon R9 290X. It could still be >>> possible to use this graphics card with FreeBSD 10.1 by means of PCI pass >>> through with Bhyve, although (as I found out with extensive testing with >>> Xen) it is Intel processor favored and supported. When is AMD64 support >>> coming up to the level of development that Intel processors are getting? >>> >> >> AMD processor support in bhyve is almost at the level of Intel processor >> support, with one notable gap being the lack of IOMMU support which is >> required for PCI passthru. >> >> When will Windows and Mac OS X Snow Leopard onwards be supported? >>> >> >> Windows is (slowly) being worked on. Nothing is happening with OSX to my >> knowledge. >> >> Are installations headless and require something like VNC? >>> >> >> Installations are currently via the guest's serial console. >> >> later, >> >> Peter. >> >> > From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 9 20:43:16 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02012D78 for ; Thu, 9 Apr 2015 20:43:16 +0000 (UTC) Received: from mail-vn0-x22e.google.com (mail-vn0-x22e.google.com [IPv6:2607:f8b0:400c:c0f::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF28D898 for ; Thu, 9 Apr 2015 20:43:15 +0000 (UTC) Received: by vnbg1 with SMTP id g1so22108531vnb.2 for ; Thu, 09 Apr 2015 13:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=friNfvRfGvLt+56sD29WRaRmOZA1c4FSQl3j7Dvy1Ug=; b=AF0Cdqs1b6fxhnELVQxOPehHrplBIlnVp/EPaSKlQOUq6NK5jUraBPZFbtsAVG+RRa ZfwnKEzLU21PXVfTxJH8EigdRIiWw2vjFfxZYTAfAR8VxXdAjPKzOUKhvusd/hkDmh3c 17Oeemdq8EK+LSvMqRq4YCmn/tqcQSbnBQATvsVEJZMLpUwtbSgnp4J3l1NQ/s6nJfAx febfIH/ux8GogqLRITwomFxwQftsJBW0HfUYubBg8uBr4hFa/m/FNQF8r+fI5mq+8JXz pQGTrYBtGCRt7Q1owxjdzrTZOpycYcCv5WzUH4ImPlWRFbUqBxGuDu4LOReOb3jxVkCK DAlg== MIME-Version: 1.0 X-Received: by 10.140.98.245 with SMTP id o108mr35020275qge.36.1428612194908; Thu, 09 Apr 2015 13:43:14 -0700 (PDT) Received: by 10.140.175.11 with HTTP; Thu, 9 Apr 2015 13:43:14 -0700 (PDT) Date: Thu, 9 Apr 2015 21:43:14 +0100 Message-ID: Subject: Problem hiding USB host controller From: "Luis P. Mendes" To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 20:43:16 -0000 Hi, I'm trying to passthrough a USB host controller (2.0 or 3.0) but there seems to be some kind of problem. # uname -a FreeBSD leao 10.1-STABLE FreeBSD 10.1-STABLE #0 r279201: Mon Feb 23 18:55:46 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 VT-d is enabled in bios. CPU is a Intel Xeon E3-1246v3 Haswell # cat /boot/loader.conf # load radeon firmware not presented pptdevs="0/26/0 1/0/0 0/20/0" blackhole_load="YES" # pciconf -lv # only relevant ehci0@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = 'Lynx Point USB Enhanced Host Controller' class = serial bus subclass = USB xhci0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = 'Lynx Point USB xHCI Host Controller' class = serial bus subclass = USB ppt0@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b rev=0x11 hdr=0x00 Only the 1:0:0 disk controler is hidden. The two USB controllers, one ehci and one xhci are not. Why don't the controllers get hidden? Is the blackhole_load="YES" still relevant? I tried with and without that line to no avail. Is my STABLE version too old? Luis PS. Keep up the excellent work bringing From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 9 21:25:17 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42C0B809 for ; Thu, 9 Apr 2015 21:25:17 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAD48DAC for ; Thu, 9 Apr 2015 21:25:16 +0000 (UTC) Received: by wiax7 with SMTP id x7so3363961wia.0 for ; Thu, 09 Apr 2015 14:25:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=khWMPMHilEyNqRBCxEsz9onlWsv3tsxz/JTq4sKH1MQ=; b=yns325hElyc4H4JqVytl8xNljxkPHN79wlCYqvtbwsbTrHWz6CdoL3PLJmyrwJOwil nOueZUElhIHCWhusX5q7XyxSvWI+zNcjDW2zPJRJK0wMYWIhPtLS+snANmz6X+MsNOaz RqaSP2+amCTnvs1uy394y3L4hPf7B2l4M4CMWez3IHFa1/agNZ3ittirP1DCgOS7e1nn Q4jUqPBubs87ORVU+cWJlRZlKfPh9e47iscYCwNxNuziAUyqqT8r9gx49K1qpRvxJmYu XQcDIV7GhY8le08sASwQM152VWVIkDQV3FfiTymK99+ZAQxOXYY0uJ66BVfQybuPDucj rN9Q== MIME-Version: 1.0 X-Received: by 10.194.23.197 with SMTP id o5mr3595478wjf.75.1428614715292; Thu, 09 Apr 2015 14:25:15 -0700 (PDT) Received: by 10.27.9.9 with HTTP; Thu, 9 Apr 2015 14:25:15 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Apr 2015 14:25:15 -0700 Message-ID: Subject: Re: Problem hiding USB host controller From: Neel Natu To: "Luis P. Mendes" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 21:25:17 -0000 Hi Luis, On Thu, Apr 9, 2015 at 1:43 PM, Luis P. Mendes wrote: > Hi, > > I'm trying to passthrough a USB host controller (2.0 or 3.0) but there > seems to be some kind of problem. > > # uname -a > FreeBSD leao 10.1-STABLE FreeBSD 10.1-STABLE #0 r279201: Mon Feb 23 > 18:55:46 UTC 2015 > root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > VT-d is enabled in bios. > CPU is a Intel Xeon E3-1246v3 Haswell > > > # cat /boot/loader.conf > # load radeon firmware not presented > pptdevs="0/26/0 1/0/0 0/20/0" > blackhole_load="YES" > > # pciconf -lv > # only relevant > ehci0@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 > rev=0x05 hdr=0x00 > vendor = 'Intel Corporation' > device = 'Lynx Point USB Enhanced Host Controller' > class = serial bus > subclass = USB > > xhci0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 > rev=0x05 hdr=0x00 > vendor = 'Intel Corporation' > device = 'Lynx Point USB xHCI Host Controller' > class = serial bus > subclass = USB > > ppt0@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b > rev=0x11 hdr=0x00 > > > Only the 1:0:0 disk controler is hidden. The two USB controllers, one > ehci and one xhci are not. > > Why don't the controllers get hidden? > Is the blackhole_load="YES" still relevant? I tried with and without > that line to no avail. > Is my STABLE version too old? > The blackhole driver is not needed anymore since that functionality was moved into vmm.ko. Can you replace the line blackhole_load="YES" to vmm_load="YES" in loader.conf? best Neel > > Luis > > PS. Keep up the excellent work bringing > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 9 22:32:59 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 74504163 for ; Thu, 9 Apr 2015 22:32:59 +0000 (UTC) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32FCC836 for ; Thu, 9 Apr 2015 22:32:59 +0000 (UTC) Received: by qkhg7 with SMTP id g7so2225981qkh.2 for ; Thu, 09 Apr 2015 15:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bKdvvmZz08ctAB7rLSrqfL6y+hUolWEpgjMYX3WSroo=; b=DFPbx76F72yx+R3V+duxkSpG/sdPh8y5xq2HQ63oKGNQZFS/zOMRpM5tTXgpYLoGIC ZU7TJFTnaxz1w7zSFMhdEpslszbHLBRsdEMX6MPylofsKU0FZABsez/HNGJQgu170t3j 2NIDwxfakvAblaU4hnvBkD3IrgMDCEEl3KcGtAq1grFZyEsCcpBF59H1Ewsly0vFXyCl HYpONGPBTodMnfZpIToq4brJQLeeJQeCj7eXr6j/IVWpNyM17Cl6a53jczukMH4OcVDs KRVwMM74JwZ6pY5osjEhGYbWDA8sqjoX/5EFBJB+naZlk4iOR0uGMzgfnKvglL1Cs2Jx pshw== MIME-Version: 1.0 X-Received: by 10.55.19.80 with SMTP id d77mr43208556qkh.92.1428618777928; Thu, 09 Apr 2015 15:32:57 -0700 (PDT) Received: by 10.140.175.11 with HTTP; Thu, 9 Apr 2015 15:32:57 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Apr 2015 23:32:57 +0100 Message-ID: Subject: Re: Problem hiding USB host controller From: "Luis P. Mendes" To: Neel Natu Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 22:32:59 -0000 Hi, 2015-04-09 22:25 GMT+01:00 Neel Natu : > Hi Luis, > > On Thu, Apr 9, 2015 at 1:43 PM, Luis P. Mendes wrote: >> Hi, >> >> I'm trying to passthrough a USB host controller (2.0 or 3.0) but there >> seems to be some kind of problem. >> >> # uname -a >> FreeBSD leao 10.1-STABLE FreeBSD 10.1-STABLE #0 r279201: Mon Feb 23 >> 18:55:46 UTC 2015 >> root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >> >> VT-d is enabled in bios. >> CPU is a Intel Xeon E3-1246v3 Haswell >> >> >> # cat /boot/loader.conf >> # load radeon firmware not presented >> pptdevs="0/26/0 1/0/0 0/20/0" >> blackhole_load="YES" >> >> # pciconf -lv >> # only relevant >> ehci0@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 >> rev=0x05 hdr=0x00 >> vendor = 'Intel Corporation' >> device = 'Lynx Point USB Enhanced Host Controller' >> class = serial bus >> subclass = USB >> >> xhci0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 >> rev=0x05 hdr=0x00 >> vendor = 'Intel Corporation' >> device = 'Lynx Point USB xHCI Host Controller' >> class = serial bus >> subclass = USB >> >> ppt0@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b >> rev=0x11 hdr=0x00 >> >> >> Only the 1:0:0 disk controler is hidden. The two USB controllers, one >> ehci and one xhci are not. >> >> Why don't the controllers get hidden? >> Is the blackhole_load="YES" still relevant? I tried with and without >> that line to no avail. >> Is my STABLE version too old? >> > > The blackhole driver is not needed anymore since that functionality > was moved into vmm.ko. > > Can you replace the line blackhole_load="YES" to vmm_load="YES" in loader.conf? The vmm_load="YES" line was already present. The full contents of the file are: % cat /boot/loader.conf net.fibs=4 radeonkmsfw_CAICOS_pfp_load="YES" radeonkmsfw_CAICOS_me_load="YES" radeonkmsfw_BTC_rlc_load="YES" radeonkmsfw_CAICOS_mc_load="YES" radeonkms_load="YES" vmm_load="YES" nmdm_load="YES" if_bridge_load="YES" if_tap_load="YES" pptdevs="1/0/0 0/20/0 0/26/0" Thanks, Luis From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 05:30:18 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7142C7A for ; Fri, 10 Apr 2015 05:30:18 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72D6985B for ; Fri, 10 Apr 2015 05:30:18 +0000 (UTC) Received: by wgin8 with SMTP id n8so6904608wgi.0 for ; Thu, 09 Apr 2015 22:30:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=E1+VOrLz4bfsblnHtrKMHXO/CL8pIX/HThfyO8AMyIo=; b=drcgkNxZbYHTCydRqqKXgWKad8IQwx4OixtNCMuZRVzmUOdZEsr4GaeOlNRo7o1OQV HKBF2kGAk+OA/7jcOoVFJlNfFldphtK3fnr+Q7xGdj08/qjgsdJmRg7b7/BTe0ZnYbi5 wGa9dZU2mM3BP2OBakVugQkbsKpKkykW8x5GWJ/qONhzE93qlITQ0GRctKAeZfscA4j6 LUWVbopjgn3m6wxVJkb+tY1FVTyqTEPd+Oz38M+eb+y5Z5we6FArIW8rDMf4xy/2vOGm Mb2JIs8g7o2WSCbu7JXDLg7qQOylTiK89s/jS8JaTfEGpNH6axU77ksq5uaG8sBsW/28 IuEw== MIME-Version: 1.0 X-Received: by 10.194.89.7 with SMTP id bk7mr7037967wjb.127.1428643815840; Thu, 09 Apr 2015 22:30:15 -0700 (PDT) Received: by 10.27.9.9 with HTTP; Thu, 9 Apr 2015 22:30:15 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Apr 2015 22:30:15 -0700 Message-ID: Subject: Re: Problem hiding USB host controller From: Neel Natu To: "Luis P. Mendes" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 05:30:19 -0000 Hi Luis, This following patch should do the trick: https://people.freebsd.org/~neel/patches/usb_pci_probe_default.patch The patch is against -current although it applies cleanly to 10-stable as well. best Neel On Thu, Apr 9, 2015 at 3:32 PM, Luis P. Mendes wrote: > Hi, > > 2015-04-09 22:25 GMT+01:00 Neel Natu : >> Hi Luis, >> >> On Thu, Apr 9, 2015 at 1:43 PM, Luis P. Mendes wrote: >>> Hi, >>> >>> I'm trying to passthrough a USB host controller (2.0 or 3.0) but there >>> seems to be some kind of problem. >>> >>> # uname -a >>> FreeBSD leao 10.1-STABLE FreeBSD 10.1-STABLE #0 r279201: Mon Feb 23 >>> 18:55:46 UTC 2015 >>> root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >>> >>> VT-d is enabled in bios. >>> CPU is a Intel Xeon E3-1246v3 Haswell >>> >>> >>> # cat /boot/loader.conf >>> # load radeon firmware not presented >>> pptdevs="0/26/0 1/0/0 0/20/0" >>> blackhole_load="YES" >>> >>> # pciconf -lv >>> # only relevant >>> ehci0@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 >>> rev=0x05 hdr=0x00 >>> vendor = 'Intel Corporation' >>> device = 'Lynx Point USB Enhanced Host Controller' >>> class = serial bus >>> subclass = USB >>> >>> xhci0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 >>> rev=0x05 hdr=0x00 >>> vendor = 'Intel Corporation' >>> device = 'Lynx Point USB xHCI Host Controller' >>> class = serial bus >>> subclass = USB >>> >>> ppt0@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b >>> rev=0x11 hdr=0x00 >>> >>> >>> Only the 1:0:0 disk controler is hidden. The two USB controllers, one >>> ehci and one xhci are not. >>> >>> Why don't the controllers get hidden? >>> Is the blackhole_load="YES" still relevant? I tried with and without >>> that line to no avail. >>> Is my STABLE version too old? >>> >> >> The blackhole driver is not needed anymore since that functionality >> was moved into vmm.ko. >> >> Can you replace the line blackhole_load="YES" to vmm_load="YES" in loader.conf? > > The vmm_load="YES" line was already present. > The full contents of the file are: > > % cat /boot/loader.conf > net.fibs=4 > > radeonkmsfw_CAICOS_pfp_load="YES" > radeonkmsfw_CAICOS_me_load="YES" > radeonkmsfw_BTC_rlc_load="YES" > radeonkmsfw_CAICOS_mc_load="YES" > radeonkms_load="YES" > > vmm_load="YES" > nmdm_load="YES" > if_bridge_load="YES" > if_tap_load="YES" > > pptdevs="1/0/0 0/20/0 0/26/0" > > > > Thanks, > > > Luis From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 08:14:25 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDBCDDA5 for ; Fri, 10 Apr 2015 08:14:25 +0000 (UTC) Received: from relay.netserv.kiev.ua (relay.netserv.kiev.ua [77.88.208.26]) by mx1.freebsd.org (Postfix) with ESMTP id 4186EC4F for ; Fri, 10 Apr 2015 08:14:24 +0000 (UTC) Received: from 00-w01.mgcnet.local (sa1.mcnet [172.26.139.41]) by relay.netserv.kiev.ua (8.13.5/8.13.5) with ESMTP id t3A83jpq010402 for ; Fri, 10 Apr 2015 11:03:45 +0300 Date: Fri, 10 Apr 2015 11:01:11 +0300 From: Subscriber Reply-To: Subscriber X-Priority: 3 (Normal) Message-ID: <346471698.20150410110111@agoris.net.ua> To: freebsd-virtualization@freebsd.org Subject: Support CPU without EPT? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 08:14:25 -0000 Hi. Question for developers of bhyve: Do you planning add support bhyve for processors without Extended Page Tables (EPT)? I want to try bhyve but my Xeon E5410 have only VT-x without EPT. Thx. From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 13:36:22 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10A81FC7 for ; Fri, 10 Apr 2015 13:36:22 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C08F4901 for ; Fri, 10 Apr 2015 13:36:21 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id BAB2428116D for ; Fri, 10 Apr 2015 23:36:12 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 67oCxD3hoiZu for ; Fri, 10 Apr 2015 23:36:12 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-76-126-65-88.hsd1.ca.comcast.net [76.126.65.88]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 367A5281124; Fri, 10 Apr 2015 23:36:09 +1000 (EST) Message-ID: <5527D1C8.3040600@freebsd.org> Date: Fri, 10 Apr 2015 06:36:08 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Subscriber Subject: Re: Support CPU without EPT? References: <346471698.20150410110111@agoris.net.ua> In-Reply-To: <346471698.20150410110111@agoris.net.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 13:36:22 -0000 Hi, > Question for developers of bhyve: > Do you planning add support bhyve for processors without Extended Page > Tables (EPT)? There are no plans to add support for non-EPT processors. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 17:35:57 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6821B36 for ; Fri, 10 Apr 2015 17:35:57 +0000 (UTC) Received: from family.redbarn.org (family.redbarn.org [IPv6:2001:559:8000:cd::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D07399EE for ; Fri, 10 Apr 2015 17:35:57 +0000 (UTC) Received: from [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd] (unknown [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id 1452B18149 for ; Fri, 10 Apr 2015 17:35:50 +0000 (UTC) Message-ID: <552809F4.6070206@redbarn.org> Date: Fri, 10 Apr 2015 10:35:48 -0700 From: Paul Vixie User-Agent: Postbox 3.0.11 (Windows/20140602) MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf X-Enigmail-Version: 1.2.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 17:35:58 -0000 here is a representative one-hour sample of ntp messages from /var/log/messages on a freebsd 10.1 bhyve guest: > Apr 10 12:00:46 family ntpd[634]: time reset -0.613057 s > Apr 10 12:17:02 family ntpd[634]: time reset -0.604933 s > Apr 10 12:33:51 family ntpd[634]: time reset -0.650622 s > Apr 10 12:55:06 family ntpd[634]: time reset -0.743930 s that's with kern.timecounter.hardware: HPET, which is the default. a "time reset" from ntpd means the clock either won't slew at all or can't slew fast enough. in other words a normal freebsd 10.1 server can run for months or even years without ever outputting one of these messages. google reports that some folks have had good luck with kern.timecounter.hardware: TSC-low. so i tried that on another freebsd 10.1 bhyve guest on the same bhyve server, with dramatic results -- no more ntpd messages. can we make TSC-low the default? vixie From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 17:51:34 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85297CE for ; Fri, 10 Apr 2015 17:51:34 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B1F5C26 for ; Fri, 10 Apr 2015 17:51:34 +0000 (UTC) Received: by qgeb100 with SMTP id b100so1467618qge.3 for ; Fri, 10 Apr 2015 10:51:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yQaefOSydaHc8rKPrBiHwyBtx1oPcalUvLKvSavNO9I=; b=DjbFmQVHB2IdxOT9Gj8xBo7ECLtJxqK9krIB4APfYd7Ki1HGDxXAdUh284dv14pHZO NhQvdNMJsE/zP6HreUcLD94BMx9EQsVJYIn6mXS4wsmwK4p5/Ruv77wW0c4y6ZyDY4d9 q2aGBZiXVnCI/+0P6q9qiiQB333hH9XFuf/pLnqU+/4FkodZlVrU2rbCTysGpuUzRqlY y7q2J0vDKUcQ0n72lkRXz+nz2loNErfpiLda7kedXfeTOS+2G4Oa//RtkJLlAx/nBAM+ vAKvJexUjU3mZNijQrW064+AECAy1Tz7PT1FtyglDsXWtiOnXufeTSkQkM9NozIoPcvw +fIg== MIME-Version: 1.0 X-Received: by 10.140.235.77 with SMTP id g74mr3461458qhc.64.1428688293510; Fri, 10 Apr 2015 10:51:33 -0700 (PDT) Received: by 10.140.104.44 with HTTP; Fri, 10 Apr 2015 10:51:33 -0700 (PDT) In-Reply-To: <552809F4.6070206@redbarn.org> References: <552809F4.6070206@redbarn.org> Date: Fri, 10 Apr 2015 10:51:33 -0700 Message-ID: Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf From: Stephen Stuart To: Paul Vixie Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 17:51:34 -0000 I think the right thing is to set (in /boot/loader.conf) kern.timecounter.tc.TSC-low.quality to a value higher than that for HPET, to force the clock choice over what the kernel decides for the hardware that you're running on. See timecounters(4). Stephen On Fri, Apr 10, 2015 at 10:35 AM, Paul Vixie wrote: > here is a representative one-hour sample of ntp messages from > /var/log/messages on a freebsd 10.1 bhyve guest: > > > Apr 10 12:00:46 family ntpd[634]: time reset -0.613057 s > > Apr 10 12:17:02 family ntpd[634]: time reset -0.604933 s > > Apr 10 12:33:51 family ntpd[634]: time reset -0.650622 s > > Apr 10 12:55:06 family ntpd[634]: time reset -0.743930 s > > > that's with kern.timecounter.hardware: HPET, which is the default. > > a "time reset" from ntpd means the clock either won't slew at all or > can't slew fast enough. in other words a normal freebsd 10.1 server can > run for months or even years without ever outputting one of these messages. > > google reports that some folks have had good luck with > kern.timecounter.hardware: TSC-low. so i tried that on another freebsd > 10.1 bhyve guest on the same bhyve server, with dramatic results -- no > more ntpd messages. > > can we make TSC-low the default? > > vixie > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 18:05:05 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5C256DD for ; Fri, 10 Apr 2015 18:05:05 +0000 (UTC) Received: from family.redbarn.org (family.redbarn.org [24.104.150.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC0ABD86 for ; Fri, 10 Apr 2015 18:05:05 +0000 (UTC) Received: from [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd] (unknown [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id DA4C518149; Fri, 10 Apr 2015 18:05:03 +0000 (UTC) Message-ID: <552810CE.7020502@redbarn.org> Date: Fri, 10 Apr 2015 11:05:02 -0700 From: Paul Vixie User-Agent: Postbox 3.0.11 (Windows/20140602) MIME-Version: 1.0 To: Stephen Stuart Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf References: <552809F4.6070206@redbarn.org> In-Reply-To: X-Enigmail-Version: 1.2.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 18:05:06 -0000 Stephen Stuart wrote: > I think the right thing is to set (in /boot/loader.conf) > kern.timecounter.tc.TSC-low.quality to a value higher than that for > HPET, to force the clock choice over what the kernel decides for the > hardware that you're running on. See timecounters(4). if every bhyve guest ever is going to need this, then the interface offered to the guest kernel ought to present the right set of defaults to cause the right choice to be made, or the guest kernel must simply "know better". in other words me adding something to /boot/loader.conf or /etc/sysctl.conf in every single bhyve guest ever is an indication that we're doing something wrong elsewhere. -- Paul Vixie From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 18:07:25 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2551375E for ; Fri, 10 Apr 2015 18:07:25 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id EF34DDA7 for ; Fri, 10 Apr 2015 18:07:24 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 6BBBCA1185 for ; Fri, 10 Apr 2015 18:07:18 +0000 (UTC) Message-ID: <5528115B.7060509@freebsd.org> Date: Fri, 10 Apr 2015 14:07:23 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf References: <552809F4.6070206@redbarn.org> <552810CE.7020502@redbarn.org> In-Reply-To: <552810CE.7020502@redbarn.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="don8dOrT8WfC5bAnsStggNXeqXN5ApsXS" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 18:07:25 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --don8dOrT8WfC5bAnsStggNXeqXN5ApsXS Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-04-10 14:05, Paul Vixie wrote: >=20 >=20 > Stephen Stuart wrote: >> I think the right thing is to set (in /boot/loader.conf) >> kern.timecounter.tc.TSC-low.quality to a value higher than that for >> HPET, to force the clock choice over what the kernel decides for the >> hardware that you're running on. See timecounters(4). >=20 > if every bhyve guest ever is going to need this, then the interface > offered to the guest kernel ought to present the right set of defaults > to cause the right choice to be made, or the guest kernel must simply > "know better". in other words me adding something to /boot/loader.conf > or /etc/sysctl.conf in every single bhyve guest ever is an indication > that we're doing something wrong elsewhere. >=20 I think that was the suggestion, to change the login in timecounter(4) to detect bhyve (or all virtualization?) and change the quality values for the different times, so that the best one for the environment (virtualized, rather than physical) is selected automatically. Temporarily, as a workaround, /boot/loader.conf is better because it is applied sooner than /etc/sysctl.conf --=20 Allan Jude --don8dOrT8WfC5bAnsStggNXeqXN5ApsXS Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJVKBFeAAoJEJrBFpNRJZKf8JcP/A17yfZk3bHA5OcbkDQW0CSP 6EEBxnvi4KPqX4fCo/9z8SSysN9YXECc6lJMN2Gn5pC+VWdiNQZnLvKzzH5emNk+ XWCuXplTv7Zr3CsPQXsZ4tdaxiqGtPW53vwoVhqyp6IINtY5inZnFnBOyMYN92oe exSUHR5woZJVBRnQ+8RS6z2Q3iprjipLLh4uEckTyUOUjHF2cQQVL+yhBPMepWq0 AAkl7O6icz2jmgnKTvSkdirqL98HF+XFwws/vI7bx5JpCTq2dxf1aq44OdI99rMq P4NIQMBwYy3+5uG9rtIjghGBQQV91EsowWK00qUus6V9S208/OACzIg1ufUgl8QT 8anj11LuwAsgaNwV55+ArVaryGitF8sGRtA5NKlnrorOEAprmvsXpM7GDL9CPyIK E3/F8rPYwRymwkSXc5VZdtl4eEm/8kl3S84dWSkBJ1FrZV8p1ncxdVEcvMFnzNBz 3RcoI0B1lXQ9txDru9hD3Y/W51j1IviN6V0K8xupjQoRp09B5d5+5iuJjWcTk1sK 0G05utWvy2YejC2nbJQgF5w6+CgIk5szBO6L1rbgRaooFHa9wp2Ait5emF5Cjz9E 7XpFkfcD+quCSOeXvLFg6htPeJpINv2k+p4rhiN6knODmo7nLDH0CnXrL5m4k1Ox ruwVCKsK3ErEvJe0gUfI =qudz -----END PGP SIGNATURE----- --don8dOrT8WfC5bAnsStggNXeqXN5ApsXS-- From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 21:36:49 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE5285FE; Fri, 10 Apr 2015 21:36:49 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1BD7AB5; Fri, 10 Apr 2015 21:36:49 +0000 (UTC) Received: by iebmp1 with SMTP id mp1so27873782ieb.0; Fri, 10 Apr 2015 14:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=m9Q9ElWaULpeiW86ZkcFIPe6/mzAcbdGLWuiBNoeBt0=; b=Rp3uXkcJXpKjxKft3Xg7b3nr6RWJIF5c/sla/Sv0mrHgdgAhoDrJMxITILUvLo4ErI A4IKjOFEl2fM/tkPEGwR/N6PnQHRoFOX+qPDZLIoe/g1GYbgVuNzOKOuJlRXRDIPD9Gf LwwsPrBITMR5FuM2dGVFoDZRrsyzeEujhtfqECTL8BAMIhGLOaCCF/TfkYSTtQv72on4 6+s30FAfbD3tH6mf1iKG13ZKXQSxDMz1tVsa5s0E92/YfJe4SBisPlHUOMj9fyEHQCgU CdUd/ZaIQ7+LvEqmUwFnjtLBxqgvgLwekeK4v/Q6ZBQVYESNXE66VR5e8ETEsw9VZRWM ULEg== MIME-Version: 1.0 X-Received: by 10.42.207.206 with SMTP id fz14mr6426070icb.34.1428701809109; Fri, 10 Apr 2015 14:36:49 -0700 (PDT) Received: by 10.107.156.75 with HTTP; Fri, 10 Apr 2015 14:36:49 -0700 (PDT) In-Reply-To: <5528115B.7060509@freebsd.org> References: <552809F4.6070206@redbarn.org> <552810CE.7020502@redbarn.org> <5528115B.7060509@freebsd.org> Date: Fri, 10 Apr 2015 17:36:49 -0400 Message-ID: Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf From: Ryan Stone To: Allan Jude Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 21:36:50 -0000 Using the TSC as the default timecounter in a VM is dangerous. On some hardware, the TSC is not synchronized across all CPU cores. This means that if a VM migrates from one core to another, it could see the timecounter value go backwards. Time jumping backwards can cause all kinds of hilarity. What's really needed is for bhyve to support KVM clock or a similar scheme (brianv@ might also still need to commit the guest side of KVM clock as well). From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 23:14:56 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40D6A587 for ; Fri, 10 Apr 2015 23:14:56 +0000 (UTC) Received: from mx2.shrew.net (mx2.shrew.net [38.97.5.132]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9B356B2 for ; Fri, 10 Apr 2015 23:14:55 +0000 (UTC) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx2.shrew.net (8.14.7/8.14.7) with ESMTP id t3AMk8IM019973 for ; Fri, 10 Apr 2015 17:46:08 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (72-48-144-84.static.grandenetworks.net [72.48.144.84]) by mail.shrew.net (Postfix) with ESMTPSA id 8763F18A82C for ; Fri, 10 Apr 2015 17:45:57 -0500 (CDT) Message-ID: <552852FB.1060901@shrew.net> Date: Fri, 10 Apr 2015 17:47:23 -0500 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf References: <552809F4.6070206@redbarn.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx2.shrew.net [10.24.10.11]); Fri, 10 Apr 2015 17:46:08 -0500 (CDT) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 23:14:56 -0000 Perhaps applying this commit would have some positive effect ( HPET vs LAPIC )? https://docs.freebsd.org/cgi/getmsg.cgi?fetch=885358+0+archive/2015/svn-src-head/20150322.svn-src-head -Matthew On 4/10/2015 12:51 PM, Stephen Stuart wrote: > I think the right thing is to set (in /boot/loader.conf) > kern.timecounter.tc.TSC-low.quality to a value higher than that for HPET, > to force the clock choice over what the kernel decides for the hardware > that you're running on. See timecounters(4). > > Stephen > > On Fri, Apr 10, 2015 at 10:35 AM, Paul Vixie wrote: > >> here is a representative one-hour sample of ntp messages from >> /var/log/messages on a freebsd 10.1 bhyve guest: >> >>> Apr 10 12:00:46 family ntpd[634]: time reset -0.613057 s >>> Apr 10 12:17:02 family ntpd[634]: time reset -0.604933 s >>> Apr 10 12:33:51 family ntpd[634]: time reset -0.650622 s >>> Apr 10 12:55:06 family ntpd[634]: time reset -0.743930 s >> >> that's with kern.timecounter.hardware: HPET, which is the default. >> >> a "time reset" from ntpd means the clock either won't slew at all or >> can't slew fast enough. in other words a normal freebsd 10.1 server can >> run for months or even years without ever outputting one of these messages. >> >> google reports that some folks have had good luck with >> kern.timecounter.hardware: TSC-low. so i tried that on another freebsd >> 10.1 bhyve guest on the same bhyve server, with dramatic results -- no >> more ntpd messages. >> >> can we make TSC-low the default? >> >> vixie >> >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to " >> freebsd-virtualization-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 23:27:10 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 451DC666 for ; Fri, 10 Apr 2015 23:27:10 +0000 (UTC) Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF7AB7E9 for ; Fri, 10 Apr 2015 23:27:09 +0000 (UTC) Received: by wizk4 with SMTP id k4so12613020wiz.1 for ; Fri, 10 Apr 2015 16:27:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=I0o5NJctYqE7Y7E53pT4GXBigzHe9BOnBASbRH0ZwPU=; b=cdBxbfAZ78gn7g8rZd4wQgtb5UXHfE14t5XHuBsvbvx9cz3DEZlu0GwXZOL8esQ50U GeDt3QV44zS/Iodh7JZJSPak05PTaVZC2tPlKu+788si0i4CwERO0JGPXSH98oNbpb6c f0EioMtS5CIQH5qHpGFGWk7tOuTgF+d6wVL1bBGtthHXuhx9hAJzoYTvw006sLl2+vTS /m7jYvQosRTG4qJ7j0uZCiORuuYZUBdVLaqAqA0tMHWAI9Sm/+PNA6M90rLHuVwimgPW EWWArB/Cop/lYQbvWaMtU2UvMQ6wQjLJRS62edMZbnryj8WzsPxHBM620uZ6r6EcPqLI GyZw== X-Gm-Message-State: ALoCoQk3AL2F8/AtZioCeWvDbA5OW5lmwMCH9you0eu5u7qIukvv8NOQ3ml3rCs8ck9VycQ7KTW/ X-Received: by 10.194.77.44 with SMTP id p12mr7154820wjw.1.1428708422263; Fri, 10 Apr 2015 16:27:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.97.135 with HTTP; Fri, 10 Apr 2015 16:26:42 -0700 (PDT) In-Reply-To: References: <552809F4.6070206@redbarn.org> <552810CE.7020502@redbarn.org> <5528115B.7060509@freebsd.org> From: Bryan Venteicher Date: Fri, 10 Apr 2015 18:26:42 -0500 Message-ID: Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-virtualization@freebsd.org" , Allan Jude X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 23:27:10 -0000 On Fri, Apr 10, 2015 at 4:36 PM, Ryan Stone wrote: > Using the TSC as the default timecounter in a VM is dangerous. On some > hardware, the TSC is not synchronized across all CPU cores. This means > that if a VM migrates from one core to another, it could see the > timecounter value go backwards. Time jumping backwards can cause all kin= ds > of hilarity. > > What's really needed is for bhyve to support KVM clock or a similar schem= e > (brianv@ might also still need to commit the guest side of KVM clock as > well). > =E2=80=8BI still need to after addressing some review comments first, but I= 've been perpetually busy with my job. I hope to get to it RealSoon.=E2=80=8B _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 10 23:50:33 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0001DAD4 for ; Fri, 10 Apr 2015 23:50:32 +0000 (UTC) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAC54A4F for ; Fri, 10 Apr 2015 23:50:32 +0000 (UTC) Received: by qkgx75 with SMTP id x75so51491127qkg.1 for ; Fri, 10 Apr 2015 16:50:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uRhGxJYc4Us+3GgVwxZIyoqKycl66ibI2Mr1Q63CtYc=; b=MEHnYJIEFysfFzDvPKvO4EUs6iyJv7jjQ1ei8inCtgwvvBTJdwGL3KAGzlm0kApZQJ XjLTMHC67ZZ83wwcDrwS+pBI/n3vQdM+gdjxdqrelyV/mSatO+c4H8T62nu7GouaiOjj /5wfCc3w4Wzb3RaNkb7VK5tmcGgvOPfCaviHU95eSZLCzuI7Ua7SoAnCUSIWLjgyL+Rj 558NgF970sKSlz8a5xrYb4Auw2aUb1LjcaNHDflOKzn5ZfwOF3iufoYti9q59I1jTlWN 2BnygBKimkp9WZi8emqcdnxpqasDAn85FwLZ148Vq7nOh6mXEtngGL6zQT6w3pCrlshN FTRw== MIME-Version: 1.0 X-Received: by 10.140.233.146 with SMTP id e140mr5166688qhc.26.1428709831744; Fri, 10 Apr 2015 16:50:31 -0700 (PDT) Received: by 10.140.199.138 with HTTP; Fri, 10 Apr 2015 16:50:31 -0700 (PDT) In-Reply-To: References: Date: Sat, 11 Apr 2015 00:50:31 +0100 Message-ID: Subject: Re: Problem hiding USB host controller From: "Luis P. Mendes" To: Neel Natu Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 23:50:33 -0000 Hi Neel, I updated my STABLE to r281384M and applied the patch. It seems that the patch solved the issue: # pciconf -vl | grep ppt ppt0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 rev=0x05 hdr=0x00 ppt1@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 rev=0x05 hdr=0x00 ppt2@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b rev=0x11 hdr=0x00 As I'm not a developer, please tell me whether this patch will be added to STABLE or if I need to patch again when I update the system in the future. Thank you very much, Luis 2015-04-10 6:30 GMT+01:00 Neel Natu : > Hi Luis, > > This following patch should do the trick: > https://people.freebsd.org/~neel/patches/usb_pci_probe_default.patch > > The patch is against -current although it applies cleanly to 10-stable as well. > > best > Neel > > On Thu, Apr 9, 2015 at 3:32 PM, Luis P. Mendes wrote: >> Hi, >> >> 2015-04-09 22:25 GMT+01:00 Neel Natu : >>> Hi Luis, >>> >>> On Thu, Apr 9, 2015 at 1:43 PM, Luis P. Mendes wrote: >>>> Hi, >>>> >>>> I'm trying to passthrough a USB host controller (2.0 or 3.0) but there >>>> seems to be some kind of problem. >>>> >>>> # uname -a >>>> FreeBSD leao 10.1-STABLE FreeBSD 10.1-STABLE #0 r279201: Mon Feb 23 >>>> 18:55:46 UTC 2015 >>>> root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >>>> >>>> VT-d is enabled in bios. >>>> CPU is a Intel Xeon E3-1246v3 Haswell >>>> >>>> >>>> # cat /boot/loader.conf >>>> # load radeon firmware not presented >>>> pptdevs="0/26/0 1/0/0 0/20/0" >>>> blackhole_load="YES" >>>> >>>> # pciconf -lv >>>> # only relevant >>>> ehci0@pci0:0:26:0: class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 >>>> rev=0x05 hdr=0x00 >>>> vendor = 'Intel Corporation' >>>> device = 'Lynx Point USB Enhanced Host Controller' >>>> class = serial bus >>>> subclass = USB >>>> >>>> xhci0@pci0:0:20:0: class=0x0c0330 card=0x8c311849 chip=0x8c318086 >>>> rev=0x05 hdr=0x00 >>>> vendor = 'Intel Corporation' >>>> device = 'Lynx Point USB xHCI Host Controller' >>>> class = serial bus >>>> subclass = USB >>>> >>>> ppt0@pci0:1:0:0: class=0x010601 card=0x91721849 chip=0x91721b4b >>>> rev=0x11 hdr=0x00 >>>> >>>> >>>> Only the 1:0:0 disk controler is hidden. The two USB controllers, one >>>> ehci and one xhci are not. >>>> >>>> Why don't the controllers get hidden? >>>> Is the blackhole_load="YES" still relevant? I tried with and without >>>> that line to no avail. >>>> Is my STABLE version too old? >>>> >>> >>> The blackhole driver is not needed anymore since that functionality >>> was moved into vmm.ko. >>> >>> Can you replace the line blackhole_load="YES" to vmm_load="YES" in loader.conf? >> >> The vmm_load="YES" line was already present. >> The full contents of the file are: >> >> % cat /boot/loader.conf >> net.fibs=4 >> >> radeonkmsfw_CAICOS_pfp_load="YES" >> radeonkmsfw_CAICOS_me_load="YES" >> radeonkmsfw_BTC_rlc_load="YES" >> radeonkmsfw_CAICOS_mc_load="YES" >> radeonkms_load="YES" >> >> vmm_load="YES" >> nmdm_load="YES" >> if_bridge_load="YES" >> if_tap_load="YES" >> >> pptdevs="1/0/0 0/20/0 0/26/0" >> >> >> >> Thanks, >> >> >> Luis From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 11 04:27:20 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2606B297 for ; Sat, 11 Apr 2015 04:27:20 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ADB3F945 for ; Sat, 11 Apr 2015 04:27:19 +0000 (UTC) Received: by wgso17 with SMTP id o17so34377359wgs.1 for ; Fri, 10 Apr 2015 21:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GROS7CLt59fWs26K+2D+CsmAyYjX6VJMHsIUACw7MvY=; b=tciIFOhdpP4zIWtLWeWZykCxbgM8GrDA8AgD2wIlKHepE+PIMCS61NOaGt99Ld238m CsP4W6WzCC3KC4KrQAKbUpg6Gxd2SCA3Zh6Ae8wPS397XjZ7JAyGQeyTYtGgZ98KvTp+ rbuHyxBr8e/b8rp5QBKbne6Lyhwo1RYBlEMGDBpL0gFqdLh3+ekDGxxmZ3CRsR4HH40e OD52F0PjSTCdF3DlIsZF5BrYpelh1FV8wKGIF8UNd6HPE3Qw0vsiE0/Sj/KXbzIk7GpS e1S/Mnuf0mUmKdo0AXhnBjaLHRo0jGkYjat6pfLMhstzkNG6p4P1whtbb+ODmXF08EJP Vdlw== MIME-Version: 1.0 X-Received: by 10.194.6.228 with SMTP id e4mr8387159wja.63.1428726438125; Fri, 10 Apr 2015 21:27:18 -0700 (PDT) Received: by 10.27.9.9 with HTTP; Fri, 10 Apr 2015 21:27:18 -0700 (PDT) In-Reply-To: <552852FB.1060901@shrew.net> References: <552809F4.6070206@redbarn.org> <552852FB.1060901@shrew.net> Date: Fri, 10 Apr 2015 21:27:18 -0700 Message-ID: Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf From: Neel Natu To: Matthew Grooms Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 04:27:20 -0000 Hi, On Fri, Apr 10, 2015 at 3:47 PM, Matthew Grooms wrote: > Perhaps applying this commit would have some positive effect ( HPET vs LAPIC > )? > > https://docs.freebsd.org/cgi/getmsg.cgi?fetch=885358+0+archive/2015/svn-src-head/20150322.svn-src-head > No, it's an orthogonal change. The change above only causes the APIC to be preferred as an event timer (i.e. nothing to do with the choice of timecounter). best Neel > -Matthew > > > On 4/10/2015 12:51 PM, Stephen Stuart wrote: >> >> I think the right thing is to set (in /boot/loader.conf) >> kern.timecounter.tc.TSC-low.quality to a value higher than that for HPET, >> to force the clock choice over what the kernel decides for the hardware >> that you're running on. See timecounters(4). >> >> Stephen >> >> On Fri, Apr 10, 2015 at 10:35 AM, Paul Vixie wrote: >> >>> here is a representative one-hour sample of ntp messages from >>> /var/log/messages on a freebsd 10.1 bhyve guest: >>> >>>> Apr 10 12:00:46 family ntpd[634]: time reset -0.613057 s >>>> Apr 10 12:17:02 family ntpd[634]: time reset -0.604933 s >>>> Apr 10 12:33:51 family ntpd[634]: time reset -0.650622 s >>>> Apr 10 12:55:06 family ntpd[634]: time reset -0.743930 s >>> >>> >>> that's with kern.timecounter.hardware: HPET, which is the default. >>> >>> a "time reset" from ntpd means the clock either won't slew at all or >>> can't slew fast enough. in other words a normal freebsd 10.1 server can >>> run for months or even years without ever outputting one of these >>> messages. >>> >>> google reports that some folks have had good luck with >>> kern.timecounter.hardware: TSC-low. so i tried that on another freebsd >>> 10.1 bhyve guest on the same bhyve server, with dramatic results -- no >>> more ntpd messages. >>> >>> can we make TSC-low the default? >>> >>> vixie >>> >>> >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to " >>> freebsd-virtualization-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 11 04:41:31 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B28E40B for ; Sat, 11 Apr 2015 04:41:31 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4FE4AD0 for ; Sat, 11 Apr 2015 04:41:30 +0000 (UTC) Received: by wizk4 with SMTP id k4so16583953wiz.1 for ; Fri, 10 Apr 2015 21:41:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7GtDz0QvfB7mzZyOJ4OcM0iR6W03cHTqK2f/dj6qNn8=; b=o3VHErMqNn+FhrSK44ICxIYNKfM+OowqVtJLQH7tSBnIyaWFvXeHEibIGCI/ywdPNg K/ljH6xB1VFc50w/lg0wUAX4AWnh0qrpGUpGnQ7lpWdbmpxPVZ+RywVN9Mh8ydeV+K8A pz1ZvcpLipEmsprYrK7nyIH6IXyOfqcZV63zqlKjRH7jggLLFE9c6acv2C9HGdy+Gk/R Kfo5k+4Dt2ce6WxCvnUUFgmZm2OfioAcoesueD0k4bnDA3E1/qr7ef9vq6brQXKt4wFc oYpchkryKmQroElEcZ9QUrUT86YfBLc68xe9dLVRihGZwLdq/V1qrZZJF0JUWT64/L8Q kIVA== MIME-Version: 1.0 X-Received: by 10.194.89.7 with SMTP id bk7mr8355991wjb.127.1428727289195; Fri, 10 Apr 2015 21:41:29 -0700 (PDT) Received: by 10.27.9.9 with HTTP; Fri, 10 Apr 2015 21:41:29 -0700 (PDT) In-Reply-To: <552809F4.6070206@redbarn.org> References: <552809F4.6070206@redbarn.org> Date: Fri, 10 Apr 2015 21:41:29 -0700 Message-ID: Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf From: Neel Natu To: Paul Vixie Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 04:41:31 -0000 Hi Paul, On Fri, Apr 10, 2015 at 10:35 AM, Paul Vixie wrote: > here is a representative one-hour sample of ntp messages from > /var/log/messages on a freebsd 10.1 bhyve guest: > >> Apr 10 12:00:46 family ntpd[634]: time reset -0.613057 s >> Apr 10 12:17:02 family ntpd[634]: time reset -0.604933 s >> Apr 10 12:33:51 family ntpd[634]: time reset -0.650622 s >> Apr 10 12:55:06 family ntpd[634]: time reset -0.743930 s > > > that's with kern.timecounter.hardware: HPET, which is the default. > > a "time reset" from ntpd means the clock either won't slew at all or > can't slew fast enough. in other words a normal freebsd 10.1 server can > run for months or even years without ever outputting one of these messages. > > google reports that some folks have had good luck with > kern.timecounter.hardware: TSC-low. so i tried that on another freebsd > 10.1 bhyve guest on the same bhyve server, with dramatic results -- no > more ntpd messages. > > can we make TSC-low the default? > The choice of using the TSC is not without issues: - As rstone@ points out the TSCs need to be synchronized across physical cpus. - Depending on system load the guest's estimate of the TSC frequency might be way off the mark. So, in a way the HPET or the ACPI time counters are better since both the host and guest agree on the frequency. But it seems there is either an issue with the emulation or an artifact due to the guest-to-host round trip time. I have been doing experiments but don't have anything conclusive yet. best Neel > vixie > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 11 06:41:57 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48275B75 for ; Sat, 11 Apr 2015 06:41:57 +0000 (UTC) Received: from family.redbarn.org (family.redbarn.org [24.104.150.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E3E284E for ; Sat, 11 Apr 2015 06:41:56 +0000 (UTC) Received: from [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd] (unknown [IPv6:2001:559:8000:cb:447:ba78:32f4:47fd]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id 87EA31813C; Sat, 11 Apr 2015 06:41:54 +0000 (UTC) Message-ID: <5528C230.8070406@redbarn.org> Date: Fri, 10 Apr 2015 23:41:52 -0700 From: Paul Vixie User-Agent: Postbox 3.0.11 (Windows/20140602) MIME-Version: 1.0 To: Neel Natu Subject: Re: bhyve clock problem, solved by kern.timecounter.hardware="TSC-low" in /etc/sysctl.conf References: <552809F4.6070206@redbarn.org> In-Reply-To: X-Enigmail-Version: 1.2.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 06:41:57 -0000 Neel Natu wrote: > Hi Paul, > > On Fri, Apr 10, 2015 at 10:35 AM, Paul Vixie wrote: >> ... >> >> can we make TSC-low the default? >> > > The choice of using the TSC is not without issues: > > - As rstone@ points out the TSCs need to be synchronized across physical cpus. ok, then the reason i'm not seeing this is i have a single 6-core CPU. (ntpd would complain about clocks going backward.) > - Depending on system load the guest's estimate of the TSC frequency > might be way off the mark. i see what you mean. > So, in a way the HPET or the ACPI time counters are better since both > the host and guest agree on the frequency. But it seems there is > either an issue with the emulation or an artifact due to the > guest-to-host round trip time. to be fair, some of my bhyve's ran fast (so, positive adjustments by ntpd), some ran slow (negative adjustments). i did not try every possible timecounter hardware; TSC-low fixed it, so i stopped. > I have been doing experiments but don't have anything conclusive yet. thanks for looking into it. paravirtualization (kvm-clock or some equivalent) seems desirable. -- Paul Vixie