Date: Mon, 8 Dec 2014 14:45:21 -0800 From: "David P. Discher" <dpd@dpdtech.com> To: =?iso-8859-1?Q?Roger_Pau_Monn=E9?= <roger.pau@citrix.com> Cc: freebsd-xen@freebsd.org Subject: Re: Attempting to Get Xen FreeBSD Dom0 working Message-ID: <7F90454D-16D5-4871-A155-5ADD01014976@dpdtech.com> In-Reply-To: <54819D6A.4050603@citrix.com> References: <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <DCB954B5-0E45-4DE6-AAC4-C5ACF7FD90A7@dpdtech.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <BBBFE7EC-BB42-4514-9A68-437ECAB8E640@dpdtech.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <C56B19DD-B001-4AB6-A296-EC221C58297C@dpdtech.com> <D2157B5C-391D-478B-A9F0-4D34DC21534A@dpdtech.com> <54819D6A.4050603@citrix.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] I tried all the combinations of things … and still couldn’t get the xen console switch to work. I finally googled "xen X8STi console”, and found - http://wiki.xen.org/wiki/Xen_Serial_Console - which calls out the super micro board X8STi - with the options: com1=115200,8n1,0x3e8,5 I don’t know what that “,5” does … but it fixed it. I have also disabled all the ttys in /etc/ttys for the serial ports. (haven’t tried the trailing 5 with the ttyu[012] “on”. ) Ok, back to it … re-started the debian install, and had “xentop” running and “xl info” in a loop, and eventually, within in minutes, the hyper call fails. xc: error: Could not bounce buffer for sysctl hypercall (35 = Resource temporarily unavailabl): Internal error libxl: error: libxl.c:5024:libxl_get_physinfo: getting physinfo: Resource temporarily unavailable libxl_physinfo failed. xc: error: Could not bounce buffer for sysctl hypercall (35 = Resource temporarily unavailabl): Internal error libxl: error: libxl.c:5507:libxl_get_scheduler: getting domain info list: Resource temporarily unavailable The debian install is still going, but makes it impossible to launch the installed system, since no call for xl will work. Sent SIGTERM to all processes │ Sent SIGKILL to all processes───────────────────────────────────────────────┘ Requesting system reboot [ 1157.299205] Restarting system. root@borg:/zdata/debian # root@borg:/zdata/debian # root@borg:/zdata/debian # xl create -c debian.cfg root@borg:/zdata/debian # xl destroy debian xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error libxl: error: libxl.c:658:libxl_list_domain: getting domain info list: Resource temporarily unavailable debian is an invalid domain identifier (rc=-5) root@borg:/zdata/debian # I’m running AHCI with MSI off in the FreeBSD kernel, and so far, so good on that front. The great thing is now I got the Xen console working, so can get the debug output. However the bounce buffer/hypercall issue i would think is far more important than MSI interrupts at the monument. - David P. Discher http://davidpdischer.com/ AIM: DavidDPD | Y!M: daviddpdz On Dec 5, 2014, at 3:56 AM, Roger Pau Monné <roger.pau@citrix.com> wrote: > El 04/12/14 a les 22.28, David P. Discher ha escrit: >> Update: So, I changed a few things, and stuff is working better. >> >> The Xen kernel line is now: >> >> dom0_mem=2048M dom0_max_vcpus=4 dom0pvh=1 sync_console com1=115200,8n1,0x3e8 console=vga,com1 iommu=debug > > I would advise against sync_console, it can easily cause delays in > interrupt delivery which can cause timeouts in FreeBSD. > >> >> Also note, I have these set in FreeBSD: >> >> console="comconsole vidconsole" >> comconsole_speed="115200" >> comconsole_port="0x3e8" >> boot_multicons="yes" >> vm.max_wired=2097152 >> verbose_loading="YES" >> boot_verbose="" # -v: Causes extra debugging information to be printed >> >> hint.ahci.0.msi=0 >> hw.acpi.verbose=1 >> debug.acpi.enable_debug_objects=1 >> >> So far, no AHCI timeouts. I’v gotten completely through an install of Debian … granted it failed, but for a linux reasons - couldn’t find/download a package. But is still going. >> >> The change to the console lines also help … console=vga,com1 & sync_console to xen allowed the IPMI SOL COM3 to fully complete the boot under freebsd. And the tty/login ran and displayed on xc0 : >> >> FreeBSD/amd64 (borg.dpdtech.com) (xc0) > > I will try to find a system similar to yours with an IPMI SOL console > and see if I can figure out what's going on. As a test, could you try to > disable the comconsole from FreeBSD and see if that makes a difference? > > console="vidconsole" > > And remove all the comconsole_* and boot_multicons options. This will > have the side effect of removing serial output from the bootloader, but > it might prevent FreeBSD from screwing the Xen serial console. > >> However, this console will not take any input. I still can’t get switched into the Xen console (Ctrl-A x3) on either the serial of VGA consoles. >> >> Another troubling item, em0 flaps when debian is starting up: >> >> xnb(xnb_probe:1144): Claiming device 0, xnb >> xnb1.0: bpf attached >> xnb(xnb_attach:1292): Attaching to backend/vif/1/0 >> xnb(xnb_frontend_changed:1416): frontend_state=Initialising, xnb_state=InitWait >> em0: Link is Down >> xnb1.0: 2 link states coalesced >> (d1) mapping kernel into physical memory >> (d1) about to get started... >> xnb(xnb_frontend_changed:1416): frontend_state=Connected, xnb_state=InitWait >> xnb(xnb_connect_comms:796): rings connected! >> em0: Link is up 1000 Mbps Full Duplex >> >> em0 is in bridge0, which is what the debian.cfg is using. > > I certainly don't see this kind of flipping on my network card, but I > would focus on fixing the console first, so we can get debug info. > >> >> Also, something really odd … hyper calls aren’t working after launching the debian guest - which also means I can’t launch any more guests. >> >> root@borg:~ # xl list >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error >> libxl: error: libxl.c:658:libxl_list_domain: getting domain info list: Resource temporarily unavailable >> libxl_list_domain failed. > > It seems like you are running out of wired memory, you should increase > vm.max_wired, this can be changed at runtime with sysctl without problems. > >> >> I’m heading out for the afternoon shortly, but it seems the next thing to do is to get the consoles working correctly so I can get debugging info from the hypervisor. Will hopefully bang on this this evening. > > Thanks for the efforts, please keep me posted on how it goes. > > Roger. > [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUhioCAAoJEEmwU6XuhYWO7WsIAJbOnNA2PKi8s7LY/T+1OGs4 vkq4jsETqONfAa/ScvW6s6PqtLeiE2vliAQD2gCp+nQpGIzoOUQgH9owbB1RNQkU UmuyPDXrDbJH75s7eP/Vl35kUJGYcjhPi8gjpEAt+olqiIwqQngvipJt8vG5gqW5 hMVx7S+CaAhQHdZCjk9LPly8+5Ykk9RdBWodp+T02ZsnWjgYRR5C7OcYmPtUyC6f bP08XcZajjt5nHS6rbWwjVb4MLpeFlSipSkdnvjDuNY5DF6v6w5NTLE2S7YSFd97 nBVgzA4aheq8EpPucGmKxZsndpUzMzFDHNjRr+cTwIA+IGgClham4cexQ7N+cLk= =7T5o -----END PGP SIGNATURE-----help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7F90454D-16D5-4871-A155-5ADD01014976>