Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 5 Dec 2014 12:56:26 +0100
From:      =?windows-1252?Q?Roger_Pau_Monn=E9?= <roger.pau@citrix.com>
To:        "David P. Discher" <dpd@dpdtech.com>
Cc:        freebsd-xen@freebsd.org
Subject:   Re: Attempting to Get Xen FreeBSD Dom0 working
Message-ID:  <54819D6A.4050603@citrix.com>
In-Reply-To: <D2157B5C-391D-478B-A9F0-4D34DC21534A@dpdtech.com>
References:  <481F7D02-BFE9-4E35-A475-5A8A05A801CE@dpdtech.com> <547DFCC0.6030003@citrix.com> <DCB954B5-0E45-4DE6-AAC4-C5ACF7FD90A7@dpdtech.com> <547F1476.8080305@citrix.com> <29437DB9-7DC8-47A8-8FC4-2BFDE736B5BC@dpdtech.com> <547F59B4.1010105@citrix.com> <2DDCAA68-3B11-4E3C-AE61-EAD8CEEF1E2D@dpdtech.com> <547F6AC1.9060709@citrix.com> <BBBFE7EC-BB42-4514-9A68-437ECAB8E640@dpdtech.com> <7C356027-01D8-4800-B211-282566BC9871@dpdtech.com> <54801F77.5050700@citrix.com> <C56B19DD-B001-4AB6-A296-EC221C58297C@dpdtech.com> <D2157B5C-391D-478B-A9F0-4D34DC21534A@dpdtech.com>

next in thread | previous in thread | raw e-mail | index | archive | help
El 04/12/14 a les 22.28, David P. Discher ha escrit:
> Update: So, I changed a few things, and stuff is working better.
> 
> The Xen kernel line is now:
> 
>   dom0_mem=2048M dom0_max_vcpus=4 dom0pvh=1 sync_console com1=115200,8n1,0x3e8 console=vga,com1 iommu=debug

I would advise against sync_console, it can easily cause delays in
interrupt delivery which can cause timeouts in FreeBSD.

> 
> Also note, I have these set in FreeBSD:
> 
>   console="comconsole vidconsole"
>   comconsole_speed="115200"
>   comconsole_port="0x3e8"
>   boot_multicons="yes"
>   vm.max_wired=2097152
>   verbose_loading="YES"
>   boot_verbose=""	# -v: Causes extra debugging information to be printed
> 
>   hint.ahci.0.msi=0
>   hw.acpi.verbose=1
>   debug.acpi.enable_debug_objects=1
> 
> So far, no AHCI timeouts. I’v gotten completely through an install of Debian … granted it failed, but for a linux reasons - couldn’t find/download a package.  But is still going.
> 
> The change to the console lines also help … console=vga,com1 & sync_console to xen allowed the IPMI SOL COM3 to fully complete the boot under freebsd.  And the tty/login ran and displayed on xc0 :
> 
>    FreeBSD/amd64 (borg.dpdtech.com) (xc0)

I will try to find a system similar to yours with an IPMI SOL console
and see if I can figure out what's going on. As a test, could you try to
disable the comconsole from FreeBSD and see if that makes a difference?

console="vidconsole"

And remove all the comconsole_* and boot_multicons options. This will
have the side effect of removing serial output from the bootloader, but
it might prevent FreeBSD from screwing the Xen serial console.

> However, this console will not take any input.   I still can’t get switched into the Xen console (Ctrl-A x3) on either the serial of VGA consoles. 
> 
> Another troubling item, em0 flaps when debian is starting up:
> 
> 	xnb(xnb_probe:1144): Claiming device 0, xnb
> 	xnb1.0: bpf attached
> 	xnb(xnb_attach:1292): Attaching to backend/vif/1/0
> 	xnb(xnb_frontend_changed:1416): frontend_state=Initialising, xnb_state=InitWait
> 	em0: Link is Down
> 	xnb1.0: 2 link states coalesced
> 	(d1) mapping kernel into physical memory
> 	(d1) about to get started...
> 	xnb(xnb_frontend_changed:1416): frontend_state=Connected, xnb_state=InitWait
> 	xnb(xnb_connect_comms:796): rings connected!
> 	em0: Link is up 1000 Mbps Full Duplex
> 
> em0 is in bridge0, which is what the debian.cfg is using.

I certainly don't see this kind of flipping on my network card, but I
would focus on fixing the console first, so we can get debug info.

> 
> Also, something really odd … hyper calls aren’t working after launching the debian guest - which also means I can’t launch any more guests. 
> 
> 	root@borg:~ # xl list
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	xc: error: Could not bounce buffer for version hypercall (35 = Resource temporarily unavailabl): Internal error
> 	libxl: error: libxl.c:658:libxl_list_domain: getting domain info list: Resource temporarily unavailable
> 	libxl_list_domain failed.

It seems like you are running out of wired memory, you should increase
vm.max_wired, this can be changed at runtime with sysctl without problems.

> 
> I’m heading out for the afternoon shortly, but it seems the next thing to do is to get the consoles working correctly so I can get debugging info from the hypervisor.  Will hopefully bang on this this evening. 

Thanks for the efforts, please keep me posted on how it goes.

Roger.




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54819D6A.4050603>