From owner-freebsd-xen@FreeBSD.ORG Wed May 13 08:47:17 2015 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66F8E6A6 for ; Wed, 13 May 2015 08:47:17 +0000 (UTC) Received: from forward3l.mail.yandex.net (forward3l.mail.yandex.net [IPv6:2a02:6b8:0:1819::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0AA8134D for ; Wed, 13 May 2015 08:47:16 +0000 (UTC) Received: from smtp18.mail.yandex.net (smtp18.mail.yandex.net [IPv6:2a02:6b8:0:1402::119]) by forward3l.mail.yandex.net (Yandex) with ESMTP id 524A9150163B; Wed, 13 May 2015 11:47:10 +0300 (MSK) Received: from smtp18.mail.yandex.net (localhost [127.0.0.1]) by smtp18.mail.yandex.net (Yandex) with ESMTP id CFCB118A00EA; Wed, 13 May 2015 11:47:09 +0300 (MSK) Received: from unknown (unknown [77.66.213.135]) by smtp18.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id BuojZ1tiMf-l9m4ME41; Wed, 13 May 2015 11:47:09 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1431506829; bh=VWtDi9Lz4smrjOcXJcKfjpwvtnooGSltbEf27d8kok4=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=TUXJtEwtB/UcKymOYT9xV5bNKfz6/zGr/L+5u642d7WTOBHX3WHVWdCQsiZT9eJiU 4HRXVHuEX7e9Joq+Ei8+ntevHzq49fOg+RF9QDbEhhFrCd/318iDkPA9GmU5zVddCv +bL33H5BKPu3GnjGVSogEUgZ7nUhpgzCnqispbAo= Authentication-Results: smtp18.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <55530F6B.5090604@yandex.ru> Date: Wed, 13 May 2015 11:46:35 +0300 From: Ruslan Makhmatkhanov User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: =?UTF-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= , freebsd-xen@freebsd.org Subject: Re: iommu must be enabled for PVH hardware domain References: <55521A29.9080409@yandex.ru> <5552212D.8090308@citrix.com> <55530582.2070008@yandex.ru> <555307DA.2060303@citrix.com> In-Reply-To: <555307DA.2060303@citrix.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 08:47:17 -0000 Roger Pau Monné wrote on 05/13/15 11:14: > Hello, > > El 13/05/15 a les 10.04, Ruslan Makhmatkhanov ha escrit: >> Roger Pau Monné wrote on 05/12/15 18:50: >>> Hello, >>> >>> El 12/05/15 a les 17.20, Ruslan Makhmatkhanov ha escrit: >>>> Hello, >>>> >>>> we have machine with VT-d and EPT-capable CPU: >>>> Intel(R) Xeon(R) CPU E5504 [1] >>>> >>>> The system is FreeBSD 11.0-CURRENT r282694 installed from >>>> ftp.freebsd.org iso. >>>> >>>> But dom0 fails to boot at this hardware with "iommu must be enabled for >>>> PVH hardware domain" [2]. VT-d support is indeed not detected at kernel >>>> level. Here is dmesg with VT-d enabled in BIOS [3] and dmesg with VT-d >>>> disabled in BIOS [4]. It's look like there is no difference on both >>>> outputs. Here is the proofpic of that the system is VT-d capable and the >>>> support is enabled in BIOS [5]. Here is acpidump -t output [6] >>> >>> My bet right now is be that Xen detects that your IOMMU contains erratas >>> and disables it. Can you add "iommu=debug,force" to your xen_cmdline and >>> try again? >>> >>> Also, can you paste the contents of your /boot/loader.conf? >>> >>> Roger. >> >> You are right: >> >> [VT-D] Disabling IOMMU due to Intel 5500/5520/X58 Chipset errata #47, #53 >> >> and after that: >> >> ---------------------------------------------- >> Panic on CPU 0: >> Couldn't enable IOMMU and iommu=required/force >> ---------------------------------------------- >> >> Reboot in five seconds. >> >> But what this means practically? Some chipset bugs preventing us to use >> this feature. Can this circumvented somehow? Thanks. > > AFAIK iommu=force should force the usage of the IOMMU even if erratas > are detected. You can also try with iommu=force,no-intremap. > > Roger. Yes, the system is now booted successfully! But `xl list` doesn't work with: root@:~ # xl list libxl: error: libxl.c:669:libxl_list_domain: getting domain info list: Cannot allocate memory libxl_list_domain failed. I googled for this error, and make ktrace/kdump on `xl list`. Here is the output: https://dpaste.de/mAyX/raw It referencing many shared libraries, that is not available on this system. Dunno if this is cause of problem. And here is the `xl info` and `xl dmesg`: https://dpaste.de/H1dV/raw Max wired pages is unlimited: root@:~ # sysctl vm.max_wired vm.max_wired: -1 Also tried to set: vm.max_wired=1048576, but error is the same. Shell memory values are unlimited too: root@:~ # ulimit -a cpu time (seconds, -t) unlimited file size (512-blocks, -f) unlimited data seg size (kbytes, -d) 33554432 stack size (kbytes, -s) 524288 core file size (512-blocks, -c) unlimited max memory size (kbytes, -m) unlimited locked memory (kbytes, -l) 64 max user processes (-u) 6656 open files (-n) 57987 virtual mem size (kbytes, -v) unlimited swap limit (kbytes, -w) unlimited sbsize (bytes, -b) unlimited pseudo-terminals (-p) unlimited kqueues (-k) unlimited The same error when we trying to create new guest domain: http://dpaste.com/1PD63M9 Domain configuration is taken here: http://wiki.xen.org/wiki/FreeBSD_Dom0 Can anything be done to solve this? Thank you. -- Regards, Ruslan T.O.S. Of Reality