From owner-freebsd-xen@FreeBSD.ORG Sun Dec 14 06:51:49 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72B98CCF for ; Sun, 14 Dec 2014 06:51:49 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E61382CC for ; Sun, 14 Dec 2014 06:51:48 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id n12so12196431wgh.34 for ; Sat, 13 Dec 2014 22:51:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=8TZBbixBZjdL01fWz4l1UEkkfcnr1qxU7apgCBFjAr0=; b=MpPo0yp4E6WMVXD2Aq6JyqRNHTvgLnBsrksjB4RBLvJaAeGGxCoq/5vgVy9GdFtEue J5TPsmiH9V9wFS8BBk8Tach04mkQGhylxZEiR6NBgrInPndwwVhN5h3nFlWuwbP0wC0f Q/Rnv+yBUmpAbupO4tIq3ohOr1n8w0yqTd5790wdqQHcFSnNiPnWE7p1+J9hsOeumvJs zd4vraZUohvONWcG4v5Mn1WbGYCEEauVzVLZyD2+gY+5k6W6VK1aDACDa0YTZsllI7N0 OREUaI7wt65l3kdCpduT0PMR1PGySB8jBLDTh7FZ2x3vtby9pZgZnWvLct/rjw4v8RX+ Dhfw== X-Received: by 10.180.38.98 with SMTP id f2mr20909264wik.55.1418539906431; Sat, 13 Dec 2014 22:51:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.16.98 with HTTP; Sat, 13 Dec 2014 22:51:25 -0800 (PST) In-Reply-To: <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <54819D6A.4050603@citrix.com> <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> <5486B287.7070302@citrix.com> <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> From: Miguel Clara Date: Sun, 14 Dec 2014 06:51:25 +0000 Message-ID: Subject: Re: Attempting to Get Xen FreeBSD Dom0 working To: "David P. Discher" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Dec 2014 06:51:49 -0000 I was just trying too boot Freebsd Xen dom0 on a laptop but I just get a black screen after the boot process.... any idea what it could be? the system boots fine wihtout loading "boot/xen", I'm not sure how to get more info with the black scren! thanks Melhores Cumprimentos // Best Regards ----------------------------------------------- *Miguel Clara* *IT - Sys Admin & Developer* *E-mail: *miguelmclara@gmail.com www.linkedin.com/in/miguelmclara/ On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher wrote: > > ah, sorry missed that. Will try that today. > > AHCI lasted over night with MSI off. Something I noticed, is that when > the AHCI bus was timing out, it looked like the Xen Kernel was re-scannin= g > the PCI bus. (Sorry, didn=E2=80=99t save these logs). I=E2=80=99ve love = to dig into this > further. > > Please let me know what/where to add some debugging code. > > - > David P. Discher > http://davidpdischer.com/ > AIM: DavidDPD | Y!M: daviddpdz > Mobile: 408.368.3725 > > > > On Dec 9, 2014, at 12:27 AM, Roger Pau Monn=C3=A9 = wrote: > > > Hello, > > > > El 08/12/14 a les 23.45, David P. Discher ha escrit: > > > >> > >> Sent SIGTERM to all processes > =E2=94=82 > >> Sent SIGKILL to all > processes=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 > >> Requesting system reboot > >> [ 1157.299205] Restarting system. > >> root@borg:/zdata/debian # > >> root@borg:/zdata/debian # > >> root@borg:/zdata/debian # xl create -c debian.cfg > >> root@borg:/zdata/debian # xl destroy debian > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D > Resource temporarily unavailabl): Internal error > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain info > list: Resource temporarily unavailable > >> debian is an invalid domain identifier (rc=3D-5) > >> root@borg:/zdata/debian # > >> > >> I=E2=80=99m running AHCI with MSI off in the FreeBSD kernel, and so fa= r, so > good on that front. The great thing is now I got the Xen console working= , > so can get the debug output. However the bounce buffer/hypercall issue = i > would think is far more important than MSI interrupts at the monument. > > > > Glad to know you got it working at the end! I've already pointed this > > out in my last email, but did you try to increase vm.max_wired even > further? > > > > This usually happens when mlock in > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails to > > wire down the memory that would be used by the hypercalls. > > > > Roger. > > > > From owner-freebsd-xen@FreeBSD.ORG Sun Dec 14 23:34:05 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A03E296A for ; Sun, 14 Dec 2014 23:34:05 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EB2EFC1 for ; Sun, 14 Dec 2014 23:34:05 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id kx10so10672113pab.2 for ; Sun, 14 Dec 2014 15:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dpdtech.com; s=google; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=BKB9pHSLgThTid6XeA9VWi8aaEBQ2ZQsSnhr7hW1d+U=; b=QH2HjJrn4ZpOWRldV2Bqxyo4uBbwlOBwfQR2fcghRjfd324m6A+17XQuNqEGrkxWsl QZwW7TAMrAdDRubDOSQgMXVDcDn4oQm5P8xVBgKI89ZZsdpjB6j96c1ixDthlsJZZfAM Qbdu5nSpvLMDo7363pzbYyLxalGo1KjjraCQk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=BKB9pHSLgThTid6XeA9VWi8aaEBQ2ZQsSnhr7hW1d+U=; b=M8w1WV1luXTey8BEF/Y2e5lUXmZ3h9t0whhvJASdR4u7pCY0NEd07+yyRTW0yPjtRN 95+npvDfcQ/sS070rf1cBZOz2n1P8FFQ9AE7uBvL3OPo70en52rZFkRvciUScWW7Tl5x YVNdbhllwSgBr1/VJOI1dVyCLGsRDgopJtpl5jLHe7nbgFqqMfAr/6qjFOo50QVcl/GE 0Tv5EAC90TFdZ5dSfvx1KSdhbHD+Yz7ee/fVvDdzURVJ8sVCv0WyAG/ZHbplutdDIssG 7WLiBbW1NGFsgCY7Ww8fMPVxLNmfisk3LevN0O/0/QpKYTDEKJYvylzOmzXpQblzosnb /YAg== X-Gm-Message-State: ALoCoQlSWlqFUrdVSpS2eCegOJRlmLSXLN04WW41PEJqyc3+hZ75ufNTJnpfsNg0iTkif2ISd8pr X-Received: by 10.68.221.162 with SMTP id qf2mr46439640pbc.148.1418600044584; Sun, 14 Dec 2014 15:34:04 -0800 (PST) Received: from 173-13-188-41-sfba.hfc.comcastbusiness.net (173-13-188-41-sfba.hfc.comcastbusiness.net. [173.13.188.41]) by mx.google.com with ESMTPSA id tr8sm7389421pbc.30.2014.12.14.15.34.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 14 Dec 2014 15:34:03 -0800 (PST) Content-Type: multipart/signed; boundary="Apple-Mail=_346844B5-152E-47ED-BCC3-914B8EB83C46"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Attempting to Get Xen FreeBSD Dom0 working From: "David P. Discher" In-Reply-To: Date: Sun, 14 Dec 2014 15:34:08 -0800 Message-Id: References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <54819D6A.4050603@citrix.com> <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> <5486B287.7070302@citrix.com> <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> To: Miguel Clara X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Dec 2014 23:34:05 -0000 --Apple-Mail=_346844B5-152E-47ED-BCC3-914B8EB83C46 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 First make sure you CPU supports both EPT and has a IOMMU. It sounds like you have Xen to use the serial console.=20 To /etc/ttys, you need to add: xc0 "/usr/libexec/getty Pc" vt100 on secure Also, make sure all the ttyu[0-4] are off. Recent versions of FreeBSD = maybe putting onifconsole =E2=80=A6 and I can't remember all the = combinations I used, but I had a lot of issues with Xen and FreeBSD = fighting for the serial ports. Also, make sure /boot/loader.conf is configured for = console=3D=E2=80=9Cvidconsole" on the freebsd side, and that the = xen_cmdline should have =E2=80=9Cconsole=3Dvga=E2=80=9D and not =3Dcom1. = It=E2=80=99s not explicit, but the Xen kernel creates the = =E2=80=9Cconsole=E2=80=9D. The xc0 device (and Xen itself) seems to take = care of piping the Dom0 console over to whatever Xen is using as = console.=20 Other tips, if you are running ZFS, you will probably need to add = "vm.max_wired=3D-1=E2=80=9D to /etc/sysctl.conf (I=E2=80=99m actually = not sure this is valid, but if you don=E2=80=99t you=E2=80=99ll run out = of wired memory and all the =E2=80=9Cxl=E2=80=9D commands will fail. Or = limit the size of ARC. ) You might need to turn off MSI interrupts on AHCI, "hint.ahci.0.msi=3D0=E2= =80=9D in /boot/loader.conf. However, try both ways (default I believe = is =3D1). I=E2=80=99m running Intel ICH10 and have to disable MSI. = Roger has ICH8 and doesn=E2=80=99t seem to have the issue. The latest issue is very poor network performance (with Intel 82574L) = from the DomU (guests) over the bridge and to the network. However, = Guests -> Dom0 seem ok.=20 - David P. Discher http://davidpdischer.com/ AIM: DavidDPD | Y!M: daviddpdz=20 On Dec 13, 2014, at 10:51 PM, Miguel Clara = wrote: > I was just trying too boot Freebsd Xen dom0 on a laptop but I just = get a black screen after the boot process.... >=20 > any idea what it could be? >=20 > the system boots fine wihtout loading "boot/xen", I'm not sure how to = get more info with the black scren! >=20 > thanks >=20 >=20 > Melhores Cumprimentos // Best Regards > ----------------------------------------------- > Miguel Clara > IT - Sys Admin & Developer > E-mail: miguelmclara@gmail.com > www.linkedin.com/in/miguelmclara/ >=20 > On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher = wrote: > ah, sorry missed that. Will try that today. >=20 > AHCI lasted over night with MSI off. Something I noticed, is that = when the AHCI bus was timing out, it looked like the Xen Kernel was = re-scanning the PCI bus. (Sorry, didn=E2=80=99t save these logs). = I=E2=80=99ve love to dig into this further. >=20 > Please let me know what/where to add some debugging code. >=20 > - > David P. Discher > http://davidpdischer.com/ > AIM: DavidDPD | Y!M: daviddpdz > Mobile: 408.368.3725 >=20 >=20 >=20 > On Dec 9, 2014, at 12:27 AM, Roger Pau Monn=C3=A9 = wrote: >=20 > > Hello, > > > > El 08/12/14 a les 23.45, David P. Discher ha escrit: > > > >> > >> Sent SIGTERM to all processes = =E2=94=82 > >> Sent SIGKILL to all = processes=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 > >> Requesting system reboot > >> [ 1157.299205] Restarting system. > >> root@borg:/zdata/debian # > >> root@borg:/zdata/debian # > >> root@borg:/zdata/debian # xl create -c debian.cfg > >> root@borg:/zdata/debian # xl destroy debian > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> xc: error: Could not bounce buffer for version hypercall (35 =3D= Resource temporarily unavailabl): Internal error > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain = info list: Resource temporarily unavailable > >> debian is an invalid domain identifier (rc=3D-5) > >> root@borg:/zdata/debian # > >> > >> I=E2=80=99m running AHCI with MSI off in the FreeBSD kernel, and so = far, so good on that front. The great thing is now I got the Xen = console working, so can get the debug output. However the bounce = buffer/hypercall issue i would think is far more important than MSI = interrupts at the monument. > > > > Glad to know you got it working at the end! I've already pointed = this > > out in my last email, but did you try to increase vm.max_wired even = further? > > > > This usually happens when mlock in > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails to > > wire down the memory that would be used by the hypercalls. > > > > Roger. > > >=20 --Apple-Mail=_346844B5-152E-47ED-BCC3-914B8EB83C46 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUjh5xAAoJEEmwU6XuhYWOfb8IAK3lYNpE3bOHBlklrlse5K9l 9s7NH0kLclP/ka0WZB4+ZKAu15dB4Ci4IPFbIw0oolqRjb05dcOg95ImJ0JR6MiO roI7x1XAplrKbhm75GLfwst5xW76HDuxh1Toyi8l0n+7Zhi8zFkff1twhWe7aIuo Iq9Ci8+z3Yvw4IcVpx9P3lcP62By4/Me6/WvOJ+WKF/Rqrvuwag9XVx4SL3STMVj QBZPwppFWWAKLQI58yX5fPk+if9GLjvAyCBHn1CdbRCTB+ZYw7PVgxhnIIB1/jv1 GHfAxHlzQoNYBex/NPEYd/wfZ7Q/zhSKIEiULo0x9RTjB+4tkxdZOrby59ya1Jo= =jHBQ -----END PGP SIGNATURE----- --Apple-Mail=_346844B5-152E-47ED-BCC3-914B8EB83C46-- From owner-freebsd-xen@FreeBSD.ORG Mon Dec 15 00:30:30 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C41F8E3B for ; Mon, 15 Dec 2014 00:30:30 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4047369C for ; Mon, 15 Dec 2014 00:30:30 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id r20so7336242wiv.6 for ; Sun, 14 Dec 2014 16:30:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=u5hP58WN7uItXixI+XQ3lUx+qwk1IU8CAGjsNGRJ/S4=; b=wG0atrLVXx2raYfl/VoRgsyA06eXvRJKiKBBKWhOfJN/+3kDL7EWHLe4PQFaV67lxb ya7K/60ZAmlVQ9jLkG/5LgZEusxChC0NJypo7yupXEr/9vy2S+q6yR/Qw3nubNoMIFnG Vl4bAZyCbwYIxQfhxxEElZ9a6roQe+W2KG0/nC17EXX0s8t39+iPNovRs9iaf3JEX14N 435elURIySDtz5JHLqXtHNaZaOMy1iMxMxS/8tsGorBJZHKT6YTAkNdLNZeR+i53Avqi u7f4s5icJ/e81Bc5Ko0HAMyOe3sW2JQQZiPh8TJgfXI73+CajJGhgIA6M22FvdlFC4IG wVYg== X-Received: by 10.180.218.74 with SMTP id pe10mr27045389wic.48.1418603428647; Sun, 14 Dec 2014 16:30:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.16.98 with HTTP; Sun, 14 Dec 2014 16:30:08 -0800 (PST) In-Reply-To: References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <54819D6A.4050603@citrix.com> <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> <5486B287.7070302@citrix.com> <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> From: Miguel Clara Date: Mon, 15 Dec 2014 00:30:08 +0000 Message-ID: Subject: Re: Attempting to Get Xen FreeBSD Dom0 working To: "David P. Discher" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2014 00:30:30 -0000 On Sun, Dec 14, 2014 at 11:34 PM, David P. Discher wrote: > > First make sure you CPU supports both EPT and has a IOMMU. > Yes it does. I should note theat I've already tested with Xen in this laptop using Debian. > > It sounds like you have Xen to use the serial console. > > To /etc/ttys, you need to add: > > xc0 "/usr/libexec/getty Pc" vt100 on secure > > Also, make sure all the ttyu[0-4] are off. Recent versions of FreeBSD > maybe putting onifconsole =E2=80=A6 and I can't remember all the combinat= ions I > used, but I had a lot of issues with Xen and FreeBSD fighting for the > serial ports. > Those this make sense: [miguelc@hpbsd]~% cat /etc/ttys|eg 'ttyuv|xc0' xc0 "/usr/libexec/getty Pc" vt100 on secure [miguelc@hpbsd]~% cat /etc/ttys | eg 'ttyu|xc0' ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure ttyu1 "/usr/libexec/getty std.9600" dialup off secure ttyu2 "/usr/libexec/getty std.9600" dialup off secure ttyu3 "/usr/libexec/getty std.9600" dialup off secure xc0 "/usr/libexec/getty Pc" vt100 on secure (I've just added xc0 following the guide, not sure if ttyu0 onifconsole is correct or should just be off) > > Also, make sure /boot/loader.conf is configured for console=3D=E2=80=9Cvi= dconsole" > on the freebsd side, and that the xen_cmdline should have =E2=80=9Cconsol= e=3Dvga=E2=80=9D and > not =3Dcom1. It=E2=80=99s not explicit, but the Xen kernel creates the = =E2=80=9Cconsole=E2=80=9D. > The xc0 device (and Xen itself) seems to take care of piping the Dom0 > console over to whatever Xen is using as console. > Oh wait... com1 and vga ... I copy pasted from the guide and left it to "com1".. this must be the issue, let me retry and I'll post some feedback. > > > Other tips, if you are running ZFS, you will probably need to add > "vm.max_wired=3D-1=E2=80=9D to /etc/sysctl.conf (I=E2=80=99m actually not= sure this is valid, > but if you don=E2=80=99t you=E2=80=99ll run out of wired memory and all t= he =E2=80=9Cxl=E2=80=9D commands > will fail. Or limit the size of ARC. ) > My arc_max is limitted to 2G atm but in any case I've set the vm.max_wired to -1 and as you say we will see. > > You might need to turn off MSI interrupts on AHCI, "hint.ahci.0.msi=3D0= =E2=80=9D in > /boot/loader.conf. However, try both ways (default I believe is =3D1). I= =E2=80=99m > running Intel ICH10 and have to disable MSI. Roger has ICH8 and doesn=E2= =80=99t > seem to have the issue. > > Do I have a way to see what ICH is it in freebsd... (I can ofc lookit up online) its a I7 so I think its ICH10, but not sure. > The latest issue is very poor network performance (with Intel 82574L) fro= m > the DomU (guests) over the bridge and to the network. However, Guests -> > Dom0 seem ok. > > > - > David P. Discher > http://davidpdischer.com/ > AIM: DavidDPD | Y!M: daviddpdz > > > > On Dec 13, 2014, at 10:51 PM, Miguel Clara wrote= : > > > I was just trying too boot Freebsd Xen dom0 on a laptop but I just get > a black screen after the boot process.... > > > > any idea what it could be? > > > > the system boots fine wihtout loading "boot/xen", I'm not sure how to > get more info with the black scren! > > > > thanks > > > > > > Melhores Cumprimentos // Best Regards > > ----------------------------------------------- > > Miguel Clara > > IT - Sys Admin & Developer > > E-mail: miguelmclara@gmail.com > > www.linkedin.com/in/miguelmclara/ > > > > On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher > wrote: > > ah, sorry missed that. Will try that today. > > > > AHCI lasted over night with MSI off. Something I noticed, is that whe= n > the AHCI bus was timing out, it looked like the Xen Kernel was re-scannin= g > the PCI bus. (Sorry, didn=E2=80=99t save these logs). I=E2=80=99ve love = to dig into this > further. > > > > Please let me know what/where to add some debugging code. > > > > - > > David P. Discher > > http://davidpdischer.com/ > > AIM: DavidDPD | Y!M: daviddpdz > > Mobile: 408.368.3725 > > > > > > > > On Dec 9, 2014, at 12:27 AM, Roger Pau Monn=C3=A9 > wrote: > > > > > Hello, > > > > > > El 08/12/14 a les 23.45, David P. Discher ha escrit: > > > > > >> > > >> Sent SIGTERM to all processes > =E2=94=82 > > >> Sent SIGKILL to all > processes=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 > > >> Requesting system reboot > > >> [ 1157.299205] Restarting system. > > >> root@borg:/zdata/debian # > > >> root@borg:/zdata/debian # > > >> root@borg:/zdata/debian # xl create -c debian.cfg > > >> root@borg:/zdata/debian # xl destroy debian > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D > Resource temporarily unavailabl): Internal error > > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain inf= o > list: Resource temporarily unavailable > > >> debian is an invalid domain identifier (rc=3D-5) > > >> root@borg:/zdata/debian # > > >> > > >> I=E2=80=99m running AHCI with MSI off in the FreeBSD kernel, and so = far, so > good on that front. The great thing is now I got the Xen console working= , > so can get the debug output. However the bounce buffer/hypercall issue = i > would think is far more important than MSI interrupts at the monument. > > > > > > Glad to know you got it working at the end! I've already pointed this > > > out in my last email, but did you try to increase vm.max_wired even > further? > > > > > > This usually happens when mlock in > > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails to > > > wire down the memory that would be used by the hypercalls. > > > > > > Roger. > > > > > > > From owner-freebsd-xen@FreeBSD.ORG Mon Dec 15 01:24:46 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A3CC58D for ; Mon, 15 Dec 2014 01:24:46 +0000 (UTC) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9D91BB3 for ; Mon, 15 Dec 2014 01:24:45 +0000 (UTC) Received: by mail-wg0-f54.google.com with SMTP id l2so13398228wgh.41 for ; Sun, 14 Dec 2014 17:24:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc :content-type; bh=ycEOzFGLCWNAsc2fa3lEVSOEcYLr5V4Jq3v9H2svA90=; b=D7geXO0/IMOmP9RX5orT/DZww2D+cwrtaLHAGafDNPdZl0IRQ/L1b+l3+eWUfCoUrO vvAyNBxszKQ0OsBNZVwIgw37qo4lX+NVINhvFf8E1XFaxmwzQ+z6RdENeZVKq5EtoG2I M58vr2/Ub46nI6rHtsk4UiQN7/BxOmqycT5QApUSWRdmkwnhcHFP7cr/g+ruT97pK7Ij wNVgSUQgqiYtdpyu1hRgoHbnvRh3zqqghbQiUEpnMAX9u0M3bfeOaYyNL3r11rztLGjw /VZAIvHxhL+S76crKB8fr30qn94PGFOz9rgVPx/2ETOP8ca7B+XYhSrna8ZnI8wYd8Fh teBw== X-Received: by 10.180.82.98 with SMTP id h2mt17604163wiy.7.1418606684257; Sun, 14 Dec 2014 17:24:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.16.98 with HTTP; Sun, 14 Dec 2014 17:24:24 -0800 (PST) In-Reply-To: References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <54819D6A.4050603@citrix.com> <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> <5486B287.7070302@citrix.com> <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> From: Miguel Clara Date: Mon, 15 Dec 2014 01:24:24 +0000 Message-ID: Subject: Re: Attempting to Get Xen FreeBSD Dom0 working Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2014 01:24:46 -0000 On Mon, Dec 15, 2014 at 12:30 AM, Miguel Clara wrote: > > > On Sun, Dec 14, 2014 at 11:34 PM, David P. Discher > wrote: >> >> First make sure you CPU supports both EPT and has a IOMMU. >> > > Yes it does. > > I should note theat I've already tested with Xen in this laptop using > Debian. > > >> >> It sounds like you have Xen to use the serial console. >> >> To /etc/ttys, you need to add: >> >> xc0 "/usr/libexec/getty Pc" vt100 on secure >> >> Also, make sure all the ttyu[0-4] are off. Recent versions of FreeBSD >> maybe putting onifconsole =E2=80=A6 and I can't remember all the combina= tions I >> used, but I had a lot of issues with Xen and FreeBSD fighting for the >> serial ports. >> > > Those this make sense: > [miguelc@hpbsd]~% cat /etc/ttys|eg 'ttyuv|xc0' > xc0 "/usr/libexec/getty Pc" vt100 on secure > [miguelc@hpbsd]~% cat /etc/ttys | eg 'ttyu|xc0' > ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure > ttyu1 "/usr/libexec/getty std.9600" dialup off secure > ttyu2 "/usr/libexec/getty std.9600" dialup off secure > ttyu3 "/usr/libexec/getty std.9600" dialup off secure > xc0 "/usr/libexec/getty Pc" vt100 on secure > > > (I've just added xc0 following the guide, not sure if ttyu0 onifconsole > is correct or should just be off) > >> >> Also, make sure /boot/loader.conf is configured for console=3D=E2=80=9Cv= idconsole" >> on the freebsd side, and that the xen_cmdline should have =E2=80=9Cconso= le=3Dvga=E2=80=9D and >> not =3Dcom1. It=E2=80=99s not explicit, but the Xen kernel creates the = =E2=80=9Cconsole=E2=80=9D. >> The xc0 device (and Xen itself) seems to take care of piping the Dom0 >> console over to whatever Xen is using as console. >> > > Oh wait... com1 and vga ... I copy pasted from the guide and left it to > "com1".. this must be the issue, let me retry and I'll post some feedback= . > >> >> >> Other tips, if you are running ZFS, you will probably need to add >> "vm.max_wired=3D-1=E2=80=9D to /etc/sysctl.conf (I=E2=80=99m actually no= t sure this is valid, >> but if you don=E2=80=99t you=E2=80=99ll run out of wired memory and all = the =E2=80=9Cxl=E2=80=9D commands >> will fail. Or limit the size of ARC. ) >> > > My arc_max is limitted to 2G atm but in any case I've set the vm.max_wire= d > to -1 and as you say we will see. > > >> >> You might need to turn off MSI interrupts on AHCI, "hint.ahci.0.msi=3D0= =E2=80=9D in >> /boot/loader.conf. However, try both ways (default I believe is =3D1). = I=E2=80=99m >> running Intel ICH10 and have to disable MSI. Roger has ICH8 and doesn= =E2=80=99t >> seem to have the issue. >> >> Do I have a way to see what ICH is it in freebsd... (I can ofc lookit up > online) its a I7 so I think its ICH10, but not sure. > > >> The latest issue is very poor network performance (with Intel 82574L) >> from the DomU (guests) over the bridge and to the network. However, Gue= sts >> -> Dom0 seem ok. >> >> >> Console issue solved, however I'm getting a panic message with Presently, iommu must be enabled for pvh Looking the model up online I see: Intel=C2=AE Virtualization Technology (VT-x) =E2=80=A1 Yes Intel=C2=AE Virtualization Technology for Directed I/O (VT-d) =E2=80=A1 No Intel=C2=AE VT-x with Extended Page Tables (EPT) =E2=80=A1 Yes So I guess it does have EPT and vtx but not vtd, so I don't have ioemu :( - >> David P. Discher >> http://davidpdischer.com/ >> AIM: DavidDPD | Y!M: daviddpdz >> >> >> >> On Dec 13, 2014, at 10:51 PM, Miguel Clara >> wrote: >> >> > I was just trying too boot Freebsd Xen dom0 on a laptop but I just ge= t >> a black screen after the boot process.... >> > >> > any idea what it could be? >> > >> > the system boots fine wihtout loading "boot/xen", I'm not sure how to >> get more info with the black scren! >> > >> > thanks >> > >> > >> > Melhores Cumprimentos // Best Regards >> > ----------------------------------------------- >> > Miguel Clara >> > IT - Sys Admin & Developer >> > E-mail: miguelmclara@gmail.com >> > www.linkedin.com/in/miguelmclara/ >> > >> > On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher >> wrote: >> > ah, sorry missed that. Will try that today. >> > >> > AHCI lasted over night with MSI off. Something I noticed, is that >> when the AHCI bus was timing out, it looked like the Xen Kernel was >> re-scanning the PCI bus. (Sorry, didn=E2=80=99t save these logs). I=E2= =80=99ve love to dig >> into this further. >> > >> > Please let me know what/where to add some debugging code. >> > >> > - >> > David P. Discher >> > http://davidpdischer.com/ >> > AIM: DavidDPD | Y!M: daviddpdz >> > Mobile: 408.368.3725 >> > >> > >> > >> > On Dec 9, 2014, at 12:27 AM, Roger Pau Monn=C3=A9 >> wrote: >> > >> > > Hello, >> > > >> > > El 08/12/14 a les 23.45, David P. Discher ha escrit: >> > > >> > >> >> > >> Sent SIGTERM to all processes >> =E2=94=82 >> > >> Sent SIGKILL to all >> processes=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 >> > >> Requesting system reboot >> > >> [ 1157.299205] Restarting system. >> > >> root@borg:/zdata/debian # >> > >> root@borg:/zdata/debian # >> > >> root@borg:/zdata/debian # xl create -c debian.cfg >> > >> root@borg:/zdata/debian # xl destroy debian >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >> Resource temporarily unavailabl): Internal error >> > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain >> info list: Resource temporarily unavailable >> > >> debian is an invalid domain identifier (rc=3D-5) >> > >> root@borg:/zdata/debian # >> > >> >> > >> I=E2=80=99m running AHCI with MSI off in the FreeBSD kernel, and so= far, so >> good on that front. The great thing is now I got the Xen console workin= g, >> so can get the debug output. However the bounce buffer/hypercall issue= i >> would think is far more important than MSI interrupts at the monument. >> > > >> > > Glad to know you got it working at the end! I've already pointed thi= s >> > > out in my last email, but did you try to increase vm.max_wired even >> further? >> > > >> > > This usually happens when mlock in >> > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails to >> > > wire down the memory that would be used by the hypercalls. >> > > >> > > Roger. >> > > >> > >> >> From owner-freebsd-xen@FreeBSD.ORG Mon Dec 15 01:26:39 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE9D9618 for ; Mon, 15 Dec 2014 01:26:39 +0000 (UTC) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48C90BC7 for ; Mon, 15 Dec 2014 01:26:39 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id l18so13135187wgh.26 for ; Sun, 14 Dec 2014 17:26:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=hS1/DmSGoqnyNe/Q+pSOJAvhMIyncvWvXtQb8P4AObo=; b=ImKTtCklEyfSWeWT7IUAaxnMp1CnYWaQ86wMFBed7mt7GSNFfARKJdytdp7u0918is a5C3SZSIoQFGowa2AO1ivu5XfsxQTdhzKlAM/h6n44VqK7Z7cM637X9m5s1w+XBjgRMJ w4/oq//6WJhR0qVsV7Gr2VYVGwvBeB3y2C9ofrcNOIBYPNEPhzSwWorER6VzRf9egEe8 JH+/vFYad3qYeLEPHurcxoKPQ1v+OBIDgBudrHzt0o9okxcRgi4mhvJdfBitw/yhLBu/ my8jXyXYsVoFj9afRputTezH9cKaT0YMqlAo2PNC2fD4MHPxQZWwMTb6WB7vbUcjV/zo qDmg== X-Received: by 10.180.218.74 with SMTP id pe10mr27311354wic.48.1418606797662; Sun, 14 Dec 2014 17:26:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.16.98 with HTTP; Sun, 14 Dec 2014 17:26:17 -0800 (PST) In-Reply-To: References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <54819D6A.4050603@citrix.com> <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> <5486B287.7070302@citrix.com> <55E2C419-E537-4324-B7AB-A21E94863E17@dpdtech.com> From: Miguel Clara Date: Mon, 15 Dec 2014 01:26:17 +0000 Message-ID: Subject: Re: Attempting to Get Xen FreeBSD Dom0 working To: Miguel Clara Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2014 01:26:39 -0000 On Mon, Dec 15, 2014 at 1:24 AM, Miguel Clara wrote: > > > On Mon, Dec 15, 2014 at 12:30 AM, Miguel Clara > wrote: >> >> >> On Sun, Dec 14, 2014 at 11:34 PM, David P. Discher >> wrote: >>> >>> First make sure you CPU supports both EPT and has a IOMMU. >>> >> >> Yes it does. >> >> I should note theat I've already tested with Xen in this laptop using >> Debian. >> >> >>> >>> It sounds like you have Xen to use the serial console. >>> >>> To /etc/ttys, you need to add: >>> >>> xc0 "/usr/libexec/getty Pc" vt100 on secure >>> >>> Also, make sure all the ttyu[0-4] are off. Recent versions of FreeBSD >>> maybe putting onifconsole =E2=80=A6 and I can't remember all the combin= ations I >>> used, but I had a lot of issues with Xen and FreeBSD fighting for the >>> serial ports. >>> >> >> Those this make sense: >> [miguelc@hpbsd]~% cat /etc/ttys|eg 'ttyuv|xc0' >> xc0 "/usr/libexec/getty Pc" vt100 on secure >> [miguelc@hpbsd]~% cat /etc/ttys | eg 'ttyu|xc0' >> ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure >> ttyu1 "/usr/libexec/getty std.9600" dialup off secure >> ttyu2 "/usr/libexec/getty std.9600" dialup off secure >> ttyu3 "/usr/libexec/getty std.9600" dialup off secure >> xc0 "/usr/libexec/getty Pc" vt100 on secure >> >> >> (I've just added xc0 following the guide, not sure if ttyu0 onifconsole >> is correct or should just be off) >> >>> >>> Also, make sure /boot/loader.conf is configured for console=3D=E2=80=9C= vidconsole" >>> on the freebsd side, and that the xen_cmdline should have =E2=80=9Ccons= ole=3Dvga=E2=80=9D and >>> not =3Dcom1. It=E2=80=99s not explicit, but the Xen kernel creates the= =E2=80=9Cconsole=E2=80=9D. >>> The xc0 device (and Xen itself) seems to take care of piping the Dom0 >>> console over to whatever Xen is using as console. >>> >> >> Oh wait... com1 and vga ... I copy pasted from the guide and left it to >> "com1".. this must be the issue, let me retry and I'll post some feedbac= k. >> >>> >>> >>> Other tips, if you are running ZFS, you will probably need to add >>> "vm.max_wired=3D-1=E2=80=9D to /etc/sysctl.conf (I=E2=80=99m actually n= ot sure this is valid, >>> but if you don=E2=80=99t you=E2=80=99ll run out of wired memory and all= the =E2=80=9Cxl=E2=80=9D commands >>> will fail. Or limit the size of ARC. ) >>> >> >> My arc_max is limitted to 2G atm but in any case I've set the >> vm.max_wired to -1 and as you say we will see. >> >> >>> >>> You might need to turn off MSI interrupts on AHCI, "hint.ahci.0.msi=3D0= =E2=80=9D >>> in /boot/loader.conf. However, try both ways (default I believe is =3D1= ). >>> I=E2=80=99m running Intel ICH10 and have to disable MSI. Roger has ICH= 8 and >>> doesn=E2=80=99t seem to have the issue. >>> >>> Do I have a way to see what ICH is it in freebsd... (I can ofc lookit u= p >> online) its a I7 so I think its ICH10, but not sure. >> >> >>> The latest issue is very poor network performance (with Intel 82574L) >>> from the DomU (guests) over the bridge and to the network. However, Gu= ests >>> -> Dom0 seem ok. >>> >>> >>> Console issue solved, however I'm getting a panic message with > > Presently, iommu must be enabled for pvh > > Looking the model up online I see: > Intel=C2=AE Virtualization Technology (VT-x) =E2=80=A1 Yes > Intel=C2=AE Virtualization Technology for Directed I/O (VT-d) =E2=80=A1 N= o > Intel=C2=AE VT-x with Extended Page Tables (EPT) =E2=80=A1 Yes > > So I guess it does have EPT and vtx but not vtd, so I don't have ioemu :( > *IOMMU > > > - >>> David P. Discher >>> http://davidpdischer.com/ >>> AIM: DavidDPD | Y!M: daviddpdz >>> >>> >>> >>> On Dec 13, 2014, at 10:51 PM, Miguel Clara >>> wrote: >>> >>> > I was just trying too boot Freebsd Xen dom0 on a laptop but I just >>> get a black screen after the boot process.... >>> > >>> > any idea what it could be? >>> > >>> > the system boots fine wihtout loading "boot/xen", I'm not sure how to >>> get more info with the black scren! >>> > >>> > thanks >>> > >>> > >>> > Melhores Cumprimentos // Best Regards >>> > ----------------------------------------------- >>> > Miguel Clara >>> > IT - Sys Admin & Developer >>> > E-mail: miguelmclara@gmail.com >>> > www.linkedin.com/in/miguelmclara/ >>> > >>> > On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher >>> wrote: >>> > ah, sorry missed that. Will try that today. >>> > >>> > AHCI lasted over night with MSI off. Something I noticed, is that >>> when the AHCI bus was timing out, it looked like the Xen Kernel was >>> re-scanning the PCI bus. (Sorry, didn=E2=80=99t save these logs). I=E2= =80=99ve love to dig >>> into this further. >>> > >>> > Please let me know what/where to add some debugging code. >>> > >>> > - >>> > David P. Discher >>> > http://davidpdischer.com/ >>> > AIM: DavidDPD | Y!M: daviddpdz >>> > Mobile: 408.368.3725 >>> > >>> > >>> > >>> > On Dec 9, 2014, at 12:27 AM, Roger Pau Monn=C3=A9 >>> wrote: >>> > >>> > > Hello, >>> > > >>> > > El 08/12/14 a les 23.45, David P. Discher ha escrit: >>> > > >>> > >> >>> > >> Sent SIGTERM to all processes >>> =E2=94=82 >>> > >> Sent SIGKILL to all >>> processes=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 >>> > >> Requesting system reboot >>> > >> [ 1157.299205] Restarting system. >>> > >> root@borg:/zdata/debian # >>> > >> root@borg:/zdata/debian # >>> > >> root@borg:/zdata/debian # xl create -c debian.cfg >>> > >> root@borg:/zdata/debian # xl destroy debian >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> xc: error: Could not bounce buffer for version hypercall (35 = =3D >>> Resource temporarily unavailabl): Internal error >>> > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain >>> info list: Resource temporarily unavailable >>> > >> debian is an invalid domain identifier (rc=3D-5) >>> > >> root@borg:/zdata/debian # >>> > >> >>> > >> I=E2=80=99m running AHCI with MSI off in the FreeBSD kernel, and s= o far, so >>> good on that front. The great thing is now I got the Xen console worki= ng, >>> so can get the debug output. However the bounce buffer/hypercall issu= e i >>> would think is far more important than MSI interrupts at the monument. >>> > > >>> > > Glad to know you got it working at the end! I've already pointed th= is >>> > > out in my last email, but did you try to increase vm.max_wired even >>> further? >>> > > >>> > > This usually happens when mlock in >>> > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails t= o >>> > > wire down the memory that would be used by the hypercalls. >>> > > >>> > > Roger. >>> > > >>> > >>> >>> From owner-freebsd-xen@FreeBSD.ORG Tue Dec 16 14:38:06 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8728AFAF for ; Tue, 16 Dec 2014 14:38:06 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D50FABA for ; Tue, 16 Dec 2014 14:38:06 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id id10so6257962vcb.5 for ; Tue, 16 Dec 2014 06:38:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xmeQCvh3LH6FwWXn7pgPSUCAF/h6BimAndjUsW0/70c=; b=Csky5tLNG9y2SdwHsszxwJB47A0UhdkkvTGf0/mKWE8oRDs6Z7BlsUHraS4F8q3aWM Zas1xaZr06wcKEQTOq4j7bCYxjmgCE506fLapYaN/1NzkNL/RPJmtqfJ0skFcoU6VUH2 VNIK8UJ+BiFCVvLRWsh8iCGrFl/0ocwyPYTlPMwzN4+TuPdiuVDYqOLH7ky0eBUxyXSK k3yJd76nSiS0K9tGRKaGCCa+nlrU5u0h4xaj+Uc7U3M+SLMnejna5qt7J57KVptDjx2d OWExHx+e4ovnCdBMUv2yCNUKL4yiSvoUpUy51CnQ08KAJ4QqZKwSW5WpIItBQtNP25Qo ZkPQ== MIME-Version: 1.0 X-Received: by 10.220.163.2 with SMTP id y2mr22958245vcx.32.1418740685273; Tue, 16 Dec 2014 06:38:05 -0800 (PST) Received: by 10.52.35.161 with HTTP; Tue, 16 Dec 2014 06:38:05 -0800 (PST) In-Reply-To: References: Date: Tue, 16 Dec 2014 09:38:05 -0500 Message-ID: Subject: Fwd: xn devices not working on rootbsd VPSs From: Shawn Webb To: freebsd-xen@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2014 14:38:06 -0000 Hey All, I've upgraded to a recent 11-CURRENT/amd64 on my RootBSD VPS and the xn devices don't seem to work, I've linked to vnc screenshots below. I'm on r275701. http://imgur.com/7HKtUcs,O273mq7 Thanks, Shawn From owner-freebsd-xen@FreeBSD.ORG Tue Dec 16 15:03:38 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 047EF757 for ; Tue, 16 Dec 2014 15:03:38 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46474DAD for ; Tue, 16 Dec 2014 15:03:36 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.07,587,1413244800"; d="scan'208";a="204923774" Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.210.2; Tue, 16 Dec 2014 10:01:47 -0500 Message-ID: <5490495B.4030705@citrix.com> Date: Tue, 16 Dec 2014 16:01:47 +0100 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Shawn Webb , Subject: Re: Fwd: xn devices not working on rootbsd VPSs References: In-Reply-To: Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2014 15:03:38 -0000 El 16/12/14 a les 15.38, Shawn Webb ha escrit: > Hey All, > > I've upgraded to a recent 11-CURRENT/amd64 on my RootBSD VPS and the xn > devices don't seem to work, I've linked to vnc screenshots below. I'm on > r275701. I'm quite confident this is related to the bug reported by sbruno: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195537 The patch is already under review at: https://reviews.freebsd.org/D1291 And it's also on my queue of patches: http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=patch;h=0a2d3aa8fbaf9f691a190b533b45470d3454c76d Can you try if it solves your issue? Roger. From owner-freebsd-xen@FreeBSD.ORG Tue Dec 16 15:11:26 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E26E832 for ; Tue, 16 Dec 2014 15:11:26 +0000 (UTC) Received: from mail-vc0-x233.google.com (mail-vc0-x233.google.com [IPv6:2607:f8b0:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 12791E94 for ; Tue, 16 Dec 2014 15:11:26 +0000 (UTC) Received: by mail-vc0-f179.google.com with SMTP id le20so6398025vcb.24 for ; Tue, 16 Dec 2014 07:11:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nwILxpjQoXo2jA0EnuLePPQ2zGpZsuWGWhkNpzVLp0o=; b=b4OHljvLdiOeqMKNJUUnTMJjJV+2ggHimFunYGyRdZ0NHQ18LFzjEpHdJFYvB8VKhp dhj0np9vsEwh8NjIr+cR102i/e1HHLFfmwP6ZMYWUGIg1iE8EiCrBmVljWVUsAkF3+4D 9y72bKvxxULVBl3qDznv0Z1ppk57xSYqWd0beGWgvXy9CRBxainguU4Zjss8DKKyppFr S8bgPsPxAA6Mbs+ugx+Qzj5jVXDfOHZb9on5STuDQDkI9o1M274+nJCgFNX8F1U/7Ii0 fTiCLUNtlyKmJFf2OdHsMNOtpXN88cPDd7JXpvr6UyHSBZxH9cXCHQH9BwcSFZzihDOG 2yGQ== MIME-Version: 1.0 X-Received: by 10.52.253.234 with SMTP id ad10mr3116375vdd.54.1418742685180; Tue, 16 Dec 2014 07:11:25 -0800 (PST) Received: by 10.52.35.161 with HTTP; Tue, 16 Dec 2014 07:11:25 -0800 (PST) In-Reply-To: <5490495B.4030705@citrix.com> References: <5490495B.4030705@citrix.com> Date: Tue, 16 Dec 2014 10:11:25 -0500 Message-ID: Subject: Re: Fwd: xn devices not working on rootbsd VPSs From: Shawn Webb To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2014 15:11:26 -0000 On Tue, Dec 16, 2014 at 10:01 AM, Roger Pau Monn=C3=A9 wrote: > > El 16/12/14 a les 15.38, Shawn Webb ha escrit: > > Hey All, > > > > I've upgraded to a recent 11-CURRENT/amd64 on my RootBSD VPS and the xn > > devices don't seem to work, I've linked to vnc screenshots below. I'm o= n > > r275701. > > I'm quite confident this is related to the bug reported by sbruno: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D195537 > > The patch is already under review at: > > https://reviews.freebsd.org/D1291 > > And it's also on my queue of patches: > > > http://xenbits.xen.org/gitweb/?p=3Dpeople/royger/freebsd.git;a=3Dpatch;h= =3D0a2d3aa8fbaf9f691a190b533b45470d3454c76d > > Can you try if it solves your issue? > > Roger. > I will happily do this week. It takes over 24 hours to build a new world+kernel on the VPS since I'm using root-on-zfs. I'll report back soon-ish. Thanks, Shawn From owner-freebsd-xen@FreeBSD.ORG Fri Dec 19 13:11:29 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 809E3F85 for ; Fri, 19 Dec 2014 13:11:29 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A7C1139F for ; Fri, 19 Dec 2014 13:11:28 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.07,606,1413270000"; d="asc'?scan'208";a="175037515" Received: from hioexcmbx07-prd.hq.netapp.com ([10.122.105.40]) by mx11-out.netapp.com with ESMTP; 19 Dec 2014 05:11:23 -0800 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx07-prd.hq.netapp.com (10.122.105.40) with Microsoft SMTP Server (TLS) id 15.0.995.29; Fri, 19 Dec 2014 05:11:21 -0800 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::8cd:f3f3:900f:8a29%21]) with mapi id 15.00.0995.031; Fri, 19 Dec 2014 05:11:21 -0800 From: "Eggert, Lars" To: =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= Subject: Re: xn devices not working on rootbsd VPSs Thread-Topic: xn devices not working on rootbsd VPSs Thread-Index: AQHQG41Izeznqb95YUS+lGyug8Muyw== Date: Fri, 19 Dec 2014 13:11:21 +0000 Message-ID: <945CD7DD-F936-4E85-BECF-5ECDF1B65B40@netapp.com> References: <5490495B.4030705@citrix.com> In-Reply-To: <5490495B.4030705@citrix.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.1993) x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_DBED0F4F-0C27-459B-82B2-D293662497B7"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: "freebsd-xen@freebsd.org" X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Dec 2014 13:11:29 -0000 --Apple-Mail=_DBED0F4F-0C27-459B-82B2-D293662497B7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 2014-12-16, at 16:01, Roger Pau Monn=C3=A9 = wrote: > Can you try if it solves your issue? Ran into the same issue and can confirm this patch makes xn work again = here. Lars --Apple-Mail=_DBED0F4F-0C27-459B-82B2-D293662497B7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBVJQj+NZcnpRveo1xAQKRbwP+MC8r+ruT2XMAHkAnJAMBVBgDelALx7Us w3UwWT7vwSl+sdDsyNSwygX5HP44SsURZR7lcQxeA2rmCm6hrFSkDZteRVFD985J UjY8aP6O0iBKG1d+f9HA8HlqBGq0W+FWEVk5HUkJOMaVKvV4t2oJLr65/4BM3IGv TkAnHhaKQzA= =KcP1 -----END PGP SIGNATURE----- --Apple-Mail=_DBED0F4F-0C27-459B-82B2-D293662497B7--