From owner-freebsd-xen@FreeBSD.ORG Mon Feb 10 11:06:59 2014 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 870181B5 for ; Mon, 10 Feb 2014 11:06:59 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 701A91FFD for ; Mon, 10 Feb 2014 11:06:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1AB6xeg080280 for ; Mon, 10 Feb 2014 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1AB6wKj080278 for freebsd-xen@FreeBSD.org; Mon, 10 Feb 2014 11:06:58 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Feb 2014 11:06:58 GMT Message-Id: <201402101106.s1AB6wKj080278@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-xen@FreeBSD.org Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2014 11:06:59 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/183139 xen [xen] [patch] ifconfig options on xn0 lost after xen v o kern/180788 xen [xen] [panic] XEN PV kernel 9.2-BETA1 panics on boot o kern/180403 xen [xen] Problems with GENERIC and XENHVM kernels with Xe o kern/180402 xen [xen] XEN kernel does not load in XenClient 4.5.5 o kern/179814 xen [xen] mountroot fails with error=19 under Xen on 9-STA o kern/176471 xen [xen] xn driver crash on detach o kern/176053 xen [xen] [patch] i386: Correct wrong usage of vsnprintf() o kern/175954 xen [xen] XENHVM xn network driver extreme packet loss dur o kern/175822 xen [xen] FreeBSD 9.1 does not work with Xen 4.0 o kern/175757 xen [xen] [patch] xen pvhvm looses keyboard input from VNC o kern/171873 xen [xen] xn network device floods warning in dmesg o kern/171118 xen [xen] FreeBSD XENHVM guest doesn't shutdown cleanly o kern/166174 xen [xen] Problems ROOT MOUNT ERROR o kern/165418 xen [xen] Problems mounting root filesystem from XENHVM o kern/164630 xen [xen] XEN HVM kernel: run_interrupt_driven_hooks: stil o kern/164450 xen [xen] Failed to install FreeeBSD 9.0-RELEASE from CD i o kern/162677 xen [xen] FreeBSD not compatible with "Current Stable Xen" o kern/161318 xen [xen] sysinstall crashes with floating point exception o kern/155468 xen [xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen [xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen [xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen [xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen [xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen [xen] xn0 network interface and PF - Massive performan o kern/153674 xen [xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen [xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen [xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen [xen] XEN pmap code abuses vm page queue lock o kern/153150 xen [xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen [xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen [xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen [xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xen ufs filesystem corruption on XEN DomU system f kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen [xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen [xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 40 problems total. From owner-freebsd-xen@FreeBSD.ORG Mon Feb 10 16:31:15 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6876B9D4 for ; Mon, 10 Feb 2014 16:31:15 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9FBD41D66 for ; Mon, 10 Feb 2014 16:31:14 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,818,1384300800"; d="scan'208";a="99525348" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 10 Feb 2014 16:30:01 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Mon, 10 Feb 2014 11:30:00 -0500 Message-ID: <52F8FE87.1010705@citrix.com> Date: Mon, 10 Feb 2014 17:29:59 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Karl Pielorz , Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2014 16:31:15 -0000 On 08/02/14 16:03, Karl Pielorz wrote: > > Hi, > > I've got an 'issue' (more with Xen than FreeBSD) - the upshot is, I need > to disable the PV NIC (xn) from a VM running XENHVM kernel. > > Is this possible? - In an ideal world I'd like to keep the storage > drivers etc. - I just need to use the HVM 're' interface, not 'xn' > interface... > > For those who want to know what the issue is - we have a number of VM's > on 1 Xen machine (e.g. 'Xen1') all using PV drivers (Windows and FreeBSD > VM's). A FreeBSD VM is the 'default' gateway for the others - and it > doesn't work :( > > Packets are either going missing - or getting mangled. If you 'migrate' > one of those hosts to another Xen machine in the pool (e.g. Xen2) it works. > > If you migrate the default gateway VM over to the other Xen machine - > all the other VM's on the original XenServer suddenly work (i.e. can see > the outside world) - alternatively if you switch from PV drivers to HVM > drivers on the affected guests - everything works, regardless of whether > the VM's are on Xen1, Xen2 - or the same XenServer as the default > gateway VM. > > Only PV <-> PV shows the issue, and only when they're both hosted on the > same XenServer. > > Hence wondering about disabling the PV nic on the default gateway, > instead of having to do it on all the other VM's... That's quite weird... Do you see any messages in the Xen console (xl dmesg)? Can you boot with a hypervisor compiled with debug=y and see if there are any strange messages on the Xen console? Also, does replacing the FreeBSD gateway VM with a Linux PV VM solve the issue? (ie. just to check if Linux also shows this behaviour) Roger. From owner-freebsd-xen@FreeBSD.ORG Tue Feb 11 16:33:36 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A802275 for ; Tue, 11 Feb 2014 16:33:36 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 9CE6A14B9 for ; Tue, 11 Feb 2014 16:33:35 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1BGXN73017446 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 11 Feb 2014 16:33:24 GMT Date: Tue, 11 Feb 2014 16:33:22 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? Message-ID: <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> In-Reply-To: <52F8FE87.1010705@citrix.com> References: <52F8FE87.1010705@citrix.com> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 16:33:36 -0000 --On 10 February 2014 17:29 +0100 Roger Pau Monn=C3=A9 = =20 wrote: > That's quite weird... Do you see any messages in the Xen console (xl > dmesg)? Can you boot with a hypervisor compiled with debug=3Dy and see if > there are any strange messages on the Xen console? Ok, no messages logged anywhere when this goes on - I'm running XenServer=20 6.2 installed from the distribution ISO (w/SP1 applied) - I don't=20 know/think that's compiled with 'debug=3Dy'? dmesg output for XenServer=20 coming up is 'unremarkable' as far as I can see, and nothing is logged with = 'xl dmesg' > Also, does replacing the FreeBSD gateway VM with a Linux PV VM solve the > issue? (ie. just to check if Linux also shows this behaviour) Ok, that I could do - and just did. I used CentOS 6 - which booted in 'Xen' = aware mode (with Xen Virtual ethernet driver) - annoyingly that works=20 *fine* (i.e. no issues). So, Windows 7 PV <-> FreeBSD 9.x / 10.x PV =3D Fail (if on same XenServer = host) Windows 7 HVM <-> FreeBSD PV =3D Works Windows 7 PV <-> Linux (CentOS 6) PV =3D Works So it looks like it might actually be a FreeBSD/Xen issue? I just tried booting FreeBSD 10 without the Xen options in the kernel, and=20 it won't boot (it panics). Is there any way of booting 10.x in HVM mode (i.e. with no Xen PV support)=20 - or at least switching 'xn' for 're' as far as nic's go? Are there any 'offload' type features that can be disabled for 'xn' nics in = FreeBSD? -Karl From owner-freebsd-xen@FreeBSD.ORG Tue Feb 11 17:20:53 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 568754AA for ; Tue, 11 Feb 2014 17:20:53 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 572EF18D5 for ; Tue, 11 Feb 2014 17:20:51 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,826,1384300800"; d="scan'208";a="99863171" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 11 Feb 2014 17:20:44 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Tue, 11 Feb 2014 12:20:43 -0500 Message-ID: <52FA5BEA.6030307@citrix.com> Date: Tue, 11 Feb 2014 18:20:42 +0100 From: =?UTF-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Karl Pielorz , Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? References: <52F8FE87.1010705@citrix.com> <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> In-Reply-To: <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 17:20:53 -0000 On 11/02/14 17:33, Karl Pielorz wrote: > > > --On 10 February 2014 17:29 +0100 Roger Pau Monné > wrote: > >> That's quite weird... Do you see any messages in the Xen console (xl >> dmesg)? Can you boot with a hypervisor compiled with debug=y and see if >> there are any strange messages on the Xen console? > > Ok, no messages logged anywhere when this goes on - I'm running > XenServer 6.2 installed from the distribution ISO (w/SP1 applied) - I > don't know/think that's compiled with 'debug=y'? dmesg output for > XenServer coming up is 'unremarkable' as far as I can see, and nothing > is logged with 'xl dmesg' > >> Also, does replacing the FreeBSD gateway VM with a Linux PV VM solve the >> issue? (ie. just to check if Linux also shows this behaviour) > > Ok, that I could do - and just did. I used CentOS 6 - which booted in > 'Xen' aware mode (with Xen Virtual ethernet driver) - annoyingly that > works *fine* (i.e. no issues). > > So, > > Windows 7 PV <-> FreeBSD 9.x / 10.x PV = Fail (if on same XenServer host) > Windows 7 HVM <-> FreeBSD PV = Works > Windows 7 PV <-> Linux (CentOS 6) PV = Works Just to clarify, are you unable to reach a FreeBSD PV DomU from a Windows PV DomU on the same host, or are you unable to route traffic trough a FreeBSD PV DomU (on the same host)? (ie. can you ssh from the Windows PV DomU into the FreeBSD PV DomU?) > > So it looks like it might actually be a FreeBSD/Xen issue? Yes, it looks that way (dunno if netback might also be involved in this) > I just tried booting FreeBSD 10 without the Xen options in the kernel, > and it won't boot (it panics). > > Is there any way of booting 10.x in HVM mode (i.e. with no Xen PV > support) - or at least switching 'xn' for 're' as far as nic's go? I have to look into this, FreeBSD should not panic when trying to boot without XENHVM on a DomU. AFAIK there's no way to selectively enable/disable the usage of certain PV interfaces, but I certainly don't recommend switching to an emulated nic to do the routing. > Are there any 'offload' type features that can be disabled for 'xn' nics > in FreeBSD? You could try to disable the following: ifconfig xnX -rxcsum -txcsum -tso4 -lro Or a mix of those, I'm afraid I'm not that familiar with netfront/netback to provide any more hints. In the past there have been reports of TSO breaking on Xen PV interfaces, but AFAIK this should be fixed already. Roger. From owner-freebsd-xen@FreeBSD.ORG Tue Feb 11 17:26:52 2014 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 248AF6BA for ; Tue, 11 Feb 2014 17:26:52 +0000 (UTC) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EECA2196F for ; Tue, 11 Feb 2014 17:26:51 +0000 (UTC) Received: from [192.168.6.121] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.7) with ESMTP id s1BHQnYk085382 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 11 Feb 2014 10:26:50 -0700 (MST) (envelope-from gibbs@scsiguy.com) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: FreeBSD 10.0-R as Xen 'guest' - clarification? From: "Justin T. Gibbs" In-Reply-To: <18819F918745D984B618D518@Mail-PC.tdx.co.uk> Date: Tue, 11 Feb 2014 10:26:43 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <7EC86263-B19E-4829-A601-F78DEDCEF7E9@scsiguy.com> References: <18819F918745D984B618D518@Mail-PC.tdx.co.uk> To: Karl Pielorz X-Mailer: Apple Mail (2.1827) Cc: freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 17:26:52 -0000 On Jan 29, 2014, at 8:28 AM, Karl Pielorz = wrote: >=20 > Hi, >=20 > With FreeBSD 10 being out now (Great!) - GENERIC appears now has = everything needed for Xen to run in PVHVM for amd64. >=20 > The man page for xen (man 4 xen) states you should have: >=20 > options NO_ADAPTIVE_MUTEXES > options NO_ADAPTIVE_RWLOCKS > options NO_ADAPTIVE_SX The =93NO_ADAPTIVE=94 settings are an optimization when running in = environments where different guests run on the same physical CPU. = However, many cloud providers seem to statically pin CPUs to VMs, which = means the adaptive lock optimization works as expected. =97 Justin= From owner-freebsd-xen@FreeBSD.ORG Tue Feb 11 18:37:05 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2F8222E for ; Tue, 11 Feb 2014 18:37:05 +0000 (UTC) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 82C55121C for ; Tue, 11 Feb 2014 18:37:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=P9cV5F/bZ6CVXQfc0zfNoZoKnU4=; b=Cszyizk+ka83NJWENO WWYnx1CgdoTgh/0lVQCLb4GZ98OWVAW5usrUz7MBA60T3Xdrw+HK3anM4GNF9ywZ SPyRYE97dWhdMv150z7P7A3lZvvPFovKzczUJvc4uMIAG9cjAYq5h2XM2Kwwec8X cg4m81AqBIrLGOpUmxKmOkgxc= Received: by mf266.sendgrid.net with SMTP id mf266.25600.52FA6DD0C Tue, 11 Feb 2014 18:37:04 +0000 (UTC) Received: from mail.tarsnap.com (unknown [10.100.60.108]) by ismtpd-023 (SG) with ESMTP id 144223cf775.530d.214708 for ; Tue, 11 Feb 2014 18:37:04 +0000 (GMT) Received: (qmail 360 invoked from network); 11 Feb 2014 18:37:03 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 11 Feb 2014 18:37:03 -0000 Received: (qmail 16625 invoked from network); 11 Feb 2014 18:36:24 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 11 Feb 2014 18:36:24 -0000 Message-ID: <52FA6DA8.7030107@freebsd.org> Date: Tue, 11 Feb 2014 10:36:24 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: "Justin T. Gibbs" , Karl Pielorz Subject: Re: FreeBSD 10.0-R as Xen 'guest' - clarification? References: <18819F918745D984B618D518@Mail-PC.tdx.co.uk> <7EC86263-B19E-4829-A601-F78DEDCEF7E9@scsiguy.com> In-Reply-To: <7EC86263-B19E-4829-A601-F78DEDCEF7E9@scsiguy.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=windows-1252 content-transfer-encoding: quoted-printable X-SG-EID: RUbAm5H8PjswBj/QH+sYVehaJogg3iBnZcyVi1bw/Iz+1kfzfmCWd2ev6DFF9NSAeuKrojRPeyagtJclHE8PHOoUTOCuXLwHXvzZTqA/V36oQjJXv9gw9563bxwhgX+oRTrFTk5BAO1X4fgLXKj79pRoutIBKutg2t+1WXjkUQg= Cc: freebsd-xen@FreeBSD.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 18:37:06 -0000 On 02/11/14 09:26, Justin T. Gibbs wrote:=0D > On Jan 29, 2014, at 8:28 AM, Karl Pielorz wrote:= =0D >> The man page for xen (man 4 xen) states you should have:=0D >>=0D >> options NO_ADAPTIVE_MUTEXES=0D >> options NO_ADAPTIVE_RWLOCKS=0D >> options NO_ADAPTIVE_SX=0D > =0D > The =93NO_ADAPTIVE=94 settings are an optimization when running in enviro= nments where different guests run on the same physical CPU. However, many = cloud providers seem to statically pin CPUs to VMs, which means the adaptiv= e lock optimization works as expected.=0D =0D ... and with static pinning, adding the NO_ADAPTIVE_FOO options causes a=0D significant drop in performance, so you really don't want them unless you'r= e=0D going to be sharing CPUs.=0D =0D -- =0D Colin Percival=0D Security Officer Emeritus, FreeBSD | The power to serve=0D Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-xen@FreeBSD.ORG Wed Feb 12 09:13:10 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50D48B04 for ; Wed, 12 Feb 2014 09:13:10 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) by mx1.freebsd.org (Postfix) with ESMTP id E19A9195B for ; Wed, 12 Feb 2014 09:13:09 +0000 (UTC) Received: from [172.16.2.46] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id D397D9DD379 for ; Wed, 12 Feb 2014 10:05:48 +0100 (CET) From: Egoitz Aurrekoetxea Subject: About XenHVM network driver and mbuf tagging Message-Id: <68058FAE-C913-463E-A20C-67239C1220AE@sarenet.es> Date: Wed, 12 Feb 2014 10:05:46 +0100 To: freebsd-xen@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 09:13:10 -0000 Hi all, If you=92re running FreeBSD 9.1 or greater (release versions tested = only) under Xen 4.1 in HVM mode (xen-detect is saying "Running in HVM = context on Xen v4.1.=94) AND if you use some (in combination of both, = and trying to see if with any other network drivers not involving = virtualization and using too the LRO piece of code of the FreeBSD = kernel) kind of software involving mbuf tagging too (Mac Framework, vlan = tagging, ipsec=85.) AND you move an important amount of traffic with = that driver (the net hvm driver in this case) you will see wired memory = starts increasing pretty fast and never being released. This seems to be = due to a problem with the LRO code of the FreeBSD kernel which causes = mbuf_tags not being properly released after the own mbuf chains have = been properly released. So as a temporary workaround (and really as = valid workaround without involving any kind of limitation) you can just = disable LRO flag in you interfaces running this driver (ifconfig = xn0-1-2=85. -lro) and you=92re done. This is a message for just trying to avoid people to waste time in = trying to find some workaround for the issue. For more info, I=92m = moving this topic at freebsd-hackers list with egoitz(at)ramattack.net = email address and with thread subject "mbuf_tag memory freeing issues = with LRO enabled on the XENHVM driver=94. Best regards, Egoitz Aurrekoetxea Departamento de sistemas 944 209 470 Parque Tecnol=F3gico. Edificio 103 48170 Zamudio (Bizkaia) egoitz@sarenet.es www.sarenet.es Antes de imprimir este correo electr=F3nico piense si es necesario = hacerlo. From owner-freebsd-xen@FreeBSD.ORG Wed Feb 12 12:36:23 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB290F94 for ; Wed, 12 Feb 2014 12:36:23 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 9B5101C9C for ; Wed, 12 Feb 2014 12:36:23 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s1CCaI9j042855 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 12 Feb 2014 12:36:18 GMT Date: Wed, 12 Feb 2014 12:36:16 +0000 From: Karl Pielorz To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , freebsd-xen@freebsd.org Subject: Re: Xen PV Networking issue - disable PV NIC in XENHVM FreeBSD? Message-ID: In-Reply-To: <52FA5BEA.6030307@citrix.com> References: <52F8FE87.1010705@citrix.com> <72C34BC121BD5A0FBDAEFA37@Mail-PC.tdx.co.uk> <52FA5BEA.6030307@citrix.com> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 12:36:23 -0000 --On 11 February 2014 18:20 +0100 Roger Pau Monn=C3=A9 = =20 wrote: > Just to clarify, are you unable to reach a FreeBSD PV DomU from a > Windows PV DomU on the same host, or are you unable to route traffic > trough a FreeBSD PV DomU (on the same host)? > > (ie. can you ssh from the Windows PV DomU into the FreeBSD PV DomU?) ssh from the Windows PV host to the FreeBSD PV DomU host appears to work=20 fine. Attempting to 'route' traffic from the Windows PV host 'through' the=20 FreeBSD PV DomU fails - pings go, DNS goes, initial TCP 'setups' go - but=20 stuff dies thereafter (i.e. may be packet size related or something). Move the FreeBSD PV DomU host to the other XenServer - works. Disabled PV=20 NIC in windows, it works. Run up the FreeBSD PV as HVM (i.e. Realtek) -=20 works. Replace FreeBSD with Linux - it works. So FreeBSD PV on same=20 XenServer as Windows, fails. The only combination I've not tested is FreeBSD DomU PV host trying to use=20 the FreeBSD DomU PV other host as a gateway. I've no reason to believe that = won't fail (with the same circumstances as above). The Windows PV Host has 1 NIC which is attached to Xen network 'Office=20 Network' (using e.g. 10.0.0.10). The FreeBSD PV Host has 2 NIC's - Interface 0 is attached to Xen network=20 'Office Network' (using e.g. 10.0.0.1) - Interface 1 is attached to Xen=20 network 'ADSL Network' (using e.g. 192.168.1.10) The FreeBSD DomU PV host gateway can ping / fetch data fine from the ADSL=20 connection itself, fine. It's setup to NAT for the Office Network (standard = rc.conf setup). >> So it looks like it might actually be a FreeBSD/Xen issue? > > Yes, it looks that way (dunno if netback might also be involved in this) Doh! :( That's annoying ;) > I have to look into this, FreeBSD should not panic when trying to boot > without XENHVM on a DomU. Ok the panic wasn't reproducible - I don't know what I did to that kernel=20 build, but re-building GENERIC without the Xen options results in a kernel=20 that boots ok in with HVM drivers (i.e. Realtek NIC's) - and using these=20 fixes the problem (as evidenced above). > You could try to disable the following: > > ifconfig xnX -rxcsum -txcsum -tso4 -lro No combination of those made any difference (which I guess is both a good=20 and a bad thing). Anything else you can think of / worth trying? - As I said I'm running the=20 XenServer distribution ISO (with SP1) - I'm not sure if that has support=20 for much debugging stuff built in? -Karl From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:53 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5AB80F53; Thu, 13 Feb 2014 20:24:53 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2D42A1E52; Thu, 13 Feb 2014 20:24:53 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 11C6EB9D8; Thu, 13 Feb 2014 15:24:52 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 12/13] mca: disable cmc enable on Xen PV Date: Sat, 08 Feb 2014 17:02:47 -0500 Message-ID: <2452208.OksOsMWhSU@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-13-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-13-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:52 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:53 -0000 On Tuesday, December 24, 2013 12:21:01 PM Roger Pau Monne wrote: > Xen PV guests doesn't have a lapic, so disable the lapic call in mca > initialization. I think this is fine, but I wonder if it wouldn't be cleaner to have lapic_enable_cmc() do the check instead. Where else do you check lapic_disabled? > --- > sys/x86/x86/mca.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > index f1369cd..e9d2c1d 100644 > --- a/sys/x86/x86/mca.c > +++ b/sys/x86/x86/mca.c > @@ -897,7 +897,7 @@ _mca_init(int boot) > } > > #ifdef DEV_APIC > - if (PCPU_GET(cmci_mask) != 0 && boot) > + if (PCPU_GET(cmci_mask) != 0 && boot && !lapic_disabled) > lapic_enable_cmc(); > #endif > } -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:59 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3C86FFA; Thu, 13 Feb 2014 20:24:59 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 76A551E5A; Thu, 13 Feb 2014 20:24:59 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 54F03B992; Thu, 13 Feb 2014 15:24:58 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 07/13] xen: implement IO APIC support in Xen mptable parser Date: Sat, 08 Feb 2014 16:36:11 -0500 Message-ID: <11166680.j6yj7PCWSG@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-8-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-8-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:58 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:59 -0000 On Tuesday, December 24, 2013 12:20:56 PM Roger Pau Monne wrote: > Use madt_setup_io (from madt.c) on Xen apic_enumerator, in order to > parse the interrupt sources from the IO APIC. > > I would like to get opinions, but I think we should rename and move > madt_setup_io to io_apic.c. It wouldn't be appropriate for io_apic.c as it isn't generic to I/O APICs but is specific to ACPI. However, mptable.c is really not a great name for this file in sys/x86/xen. I wonder if it should be xen_apic.c instead? Also, if Xen PV has an MADT table, why do you need a custom APIC enumerator at all? That is, what is preventing the code in madt.c from just working? Do you just not have 'device acpi' in the kernel config you are using? -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:57 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 383DFFF2; Thu, 13 Feb 2014 20:24:57 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 052921E57; Thu, 13 Feb 2014 20:24:57 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E17FFB9DD; Thu, 13 Feb 2014 15:24:55 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 09/13] xen: change quality of the MADT ACPI enumerator Date: Sat, 08 Feb 2014 16:42:08 -0500 Message-ID: <1980951.95r2q2cca3@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-10-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-10-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:56 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:57 -0000 On Tuesday, December 24, 2013 12:20:58 PM Roger Pau Monne wrote: > Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can > force the usage of the Xen mptable enumerator even when ACPI is > detected. Hmm, so I think one question is why does the existing MADT parser not work with the MADT table provided by Xen? This may very well be correct, but if it's only a small change to make the existing MADT parser work with Xen's MADT table, that route might be preferable. -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:55 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C837CF59; Thu, 13 Feb 2014 20:24:55 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B8151E56; Thu, 13 Feb 2014 20:24:55 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7A86CB9CA; Thu, 13 Feb 2014 15:24:54 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 10/13] xen: add ACPI bus to xen_nexus when running as Dom0 Date: Sat, 08 Feb 2014 16:50:35 -0500 Message-ID: <2410827.IqfpSAhe3T@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-11-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-11-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:54 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:55 -0000 On Tuesday, December 24, 2013 12:20:59 PM Roger Pau Monne wrote: > Also disable a couple of ACPI devices that are not usable under Dom0. Hmm, setting debug.acpi.disabled in this way is a bit hacky. It might be fine however if there's no way for the user to set it before booting the kernel (as opposed to haing the relevant drivers explicitly disable themselves under Xen which I think would be cleaner, but would also make your patch larger) -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:54 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5C6AF56; Thu, 13 Feb 2014 20:24:54 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5EEAF1E55; Thu, 13 Feb 2014 20:24:54 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 55B71B992; Thu, 13 Feb 2014 15:24:53 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 11/13] pci: introduce a new event on PCI device detection Date: Sat, 08 Feb 2014 16:57:11 -0500 Message-ID: <2534929.ApYhIx3ttm@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-12-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-12-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:53 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:54 -0000 On Tuesday, December 24, 2013 12:21:00 PM Roger Pau Monne wrote: > Add a new event that will fire each time a PCI device is added to the > system, and allows us to register the device with Xen. It's really hackish to make this PCI specific. OTOH, I can't think of a good place to have a more generic new-bus callback. You could make the eventhandler pass the 'device_t' instead of the dinfo. The dinfo isn't really a public structure, and since the device_t's ivars are already set you can use things like 'pci_get_domain()' and 'pci_get_bus()' of the passed in device in your callback function. > --- > sys/dev/pci/pci.c | 1 + > sys/sys/eventhandler.h | 5 +++++ > sys/x86/xen/pv.c | 21 +++++++++++++++++++++ > sys/x86/xen/xen_nexus.c | 6 ++++++ > sys/xen/pv.h | 1 + > 5 files changed, 34 insertions(+), 0 deletions(-) > > diff --git a/sys/dev/pci/pci.c b/sys/dev/pci/pci.c > index 4d8837f..2ee5093 100644 > --- a/sys/dev/pci/pci.c > +++ b/sys/dev/pci/pci.c > @@ -3293,6 +3293,7 @@ pci_add_child(device_t bus, struct pci_devinfo *dinfo) > resource_list_init(&dinfo->resources); > pci_cfg_save(dinfo->cfg.dev, dinfo, 0); > pci_cfg_restore(dinfo->cfg.dev, dinfo); > + EVENTHANDLER_INVOKE(pci_add, dinfo); > pci_print_verbose(dinfo); > pci_add_resources(bus, dinfo->cfg.dev, 0, 0); > } > diff --git a/sys/sys/eventhandler.h b/sys/sys/eventhandler.h > index 111c21b..3201848 100644 > --- a/sys/sys/eventhandler.h > +++ b/sys/sys/eventhandler.h > @@ -269,5 +269,10 @@ typedef void (*unregister_framebuffer_fn)(void *, > struct fb_info *); EVENTHANDLER_DECLARE(register_framebuffer, > register_framebuffer_fn); EVENTHANDLER_DECLARE(unregister_framebuffer, > unregister_framebuffer_fn); > > +/* PCI events */ > +struct pci_devinfo; > +typedef void (*pci_add_fn)(void *, struct pci_devinfo *); > +EVENTHANDLER_DECLARE(pci_add, pci_add_fn); > + > #endif /* _SYS_EVENTHANDLER_H_ */ > > diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c > index e5ad200..a44f8ca 100644 > --- a/sys/x86/xen/pv.c > +++ b/sys/x86/xen/pv.c > @@ -39,6 +39,9 @@ __FBSDID("$FreeBSD$"); > #include > #include > #include > +#include > +#include > +#include > > #include > #include > @@ -63,6 +66,8 @@ __FBSDID("$FreeBSD$"); > > #include > > +#include > + > /* Native initial function */ > extern u_int64_t hammer_time(u_int64_t, u_int64_t); > /* Xen initial function */ > @@ -384,6 +389,22 @@ xen_pv_ioapic_register_intr(struct ioapic_intsrc *pin) > xen_register_pirq(pin->io_irq, pin->io_activehi, pin->io_edgetrigger); > } > > +void > +xen_pv_pci_device_add(void *arg, struct pci_devinfo *dinfo) > +{ > + struct physdev_pci_device_add add_pci; > + int error; > + > + bzero(&add_pci, sizeof(add_pci)); > + add_pci.seg = dinfo->cfg.domain; > + add_pci.bus = dinfo->cfg.bus; > + add_pci.devfn = (dinfo->cfg.slot << 3) | dinfo->cfg.func; > + error = HYPERVISOR_physdev_op(PHYSDEVOP_pci_device_add, &add_pci); > + if (error) > + printf("unable to add device bus %u devfn %u error: %d\n", > + add_pci.bus, add_pci.devfn, error); > +} > + > static void > xen_pv_set_init_ops(void) > { > diff --git a/sys/x86/xen/xen_nexus.c b/sys/x86/xen/xen_nexus.c > index 823b3bc..60c6c5d 100644 > --- a/sys/x86/xen/xen_nexus.c > +++ b/sys/x86/xen/xen_nexus.c > @@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$"); > #include > #include > #include > +#include > > #include > > @@ -42,6 +43,7 @@ __FBSDID("$FreeBSD$"); > #include > > #include > +#include > > static const char *xen_devices[] = > { > @@ -87,6 +89,10 @@ nexus_xen_attach(device_t dev) > /* Disable some ACPI devices that are not usable by Dom0 */ > setenv("debug.acpi.disabled", "cpu hpet timer"); > > + /* Register PCI add hook */ > + EVENTHANDLER_REGISTER(pci_add, xen_pv_pci_device_add, NULL, > + EVENTHANDLER_PRI_FIRST); > + > acpi_dev = BUS_ADD_CHILD(dev, 10, "acpi", 0); > if (acpi_dev == NULL) > panic("Unable to add ACPI bus to Xen Dom0"); > diff --git a/sys/xen/pv.h b/sys/xen/pv.h > index a9d6eb0..ac737a7 100644 > --- a/sys/xen/pv.h > +++ b/sys/xen/pv.h > @@ -24,5 +24,6 @@ > #define __XEN_PV_H__ > > int xen_pv_start_all_aps(void); > +void xen_pv_pci_device_add(void *, struct pci_devinfo *); > > #endif /* __XEN_PV_H__ */ -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Thu Feb 13 20:24:58 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51CF9FF7; Thu, 13 Feb 2014 20:24:58 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 26DB31E58; Thu, 13 Feb 2014 20:24:58 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2AA98B9E6; Thu, 13 Feb 2014 15:24:57 -0500 (EST) From: John Baldwin To: Roger Pau Monne Subject: Re: [PATCH RFC 08/13] xen: change order of Xen intr init and IO APIC registration Date: Sat, 08 Feb 2014 16:37:08 -0500 Message-ID: <1827512.nD8lVLObSo@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <1387884062-41154-9-git-send-email-roger.pau@citrix.com> References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-9-git-send-email-roger.pau@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Feb 2014 15:24:57 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 20:24:58 -0000 On Tuesday, December 24, 2013 12:20:57 PM Roger Pau Monne wrote: > Change order of some of the services in the SI_SUB_INTR stage, so > that it follows the order: > > - System intr initialization > - Xen intr initalization > - IO APIC source registration This is ok. -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Fri Feb 14 01:49:35 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA334FF; Fri, 14 Feb 2014 01:49:35 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 37AC61BAF; Fri, 14 Feb 2014 01:49:33 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,842,1384300800"; d="scan'208";a="100661686" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 14 Feb 2014 01:49:25 +0000 Received: from AMSPEX01CL01.citrite.net (10.69.46.32) by FTLPEX01CL01.citrite.net (10.13.107.78) with Microsoft SMTP Server (TLS) id 14.2.342.4; Thu, 13 Feb 2014 20:49:25 -0500 Received: from [10.68.19.43] (10.68.19.43) by AMSPEX01CL01.citrite.net (10.69.46.32) with Microsoft SMTP Server id 14.2.342.4; Fri, 14 Feb 2014 02:49:23 +0100 Message-ID: <52FD7624.90202@citrix.com> Date: Fri, 14 Feb 2014 01:49:24 +0000 From: Andrew Cooper User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin , Roger Pau Monne Subject: Re: [Xen-devel] [PATCH RFC 09/13] xen: change quality of the MADT ACPI enumerator References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-10-git-send-email-roger.pau@citrix.com> <1980951.95r2q2cca3@ralph.baldwin.cx> In-Reply-To: <1980951.95r2q2cca3@ralph.baldwin.cx> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.68.19.43] X-DLP: MIA1 Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 01:49:35 -0000 On 08/02/2014 21:42, John Baldwin wrote: > On Tuesday, December 24, 2013 12:20:58 PM Roger Pau Monne wrote: >> Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can >> force the usage of the Xen mptable enumerator even when ACPI is >> detected. > Hmm, so I think one question is why does the existing MADT parser > not work with the MADT table provided by Xen? This may very well > be correct, but if it's only a small change to make the existing > MADT parser work with Xen's MADT table, that route might be > preferable. > For dom0, the MADT seen is the system MADT, which does not bear any reality to dom0's topology. For PV domU, no MADT will be found. For HVM domU, the MADT seen ought to represent (virtual) reality. ~Andrew From owner-freebsd-xen@FreeBSD.ORG Fri Feb 14 10:38:21 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F2C9F56; Fri, 14 Feb 2014 10:38:21 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A19261852; Fri, 14 Feb 2014 10:38:19 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,844,1384300800"; d="scan'208";a="100742221" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 14 Feb 2014 10:38:17 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Fri, 14 Feb 2014 05:38:17 -0500 Message-ID: <52FDF217.3040005@citrix.com> Date: Fri, 14 Feb 2014 11:38:15 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: John Baldwin Subject: Re: [PATCH RFC 10/13] xen: add ACPI bus to xen_nexus when running as Dom0 References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1387884062-41154-11-git-send-email-roger.pau@citrix.com> <2410827.IqfpSAhe3T@ralph.baldwin.cx> In-Reply-To: <2410827.IqfpSAhe3T@ralph.baldwin.cx> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA2 Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 10:38:21 -0000 On 08/02/14 22:50, John Baldwin wrote: > On Tuesday, December 24, 2013 12:20:59 PM Roger Pau Monne wrote: >> Also disable a couple of ACPI devices that are not usable under Dom0. > > Hmm, setting debug.acpi.disabled in this way is a bit hacky. It might > be fine however if there's no way for the user to set it before booting > the kernel (as opposed to haing the relevant drivers explicitly disable > themselves under Xen which I think would be cleaner, but would also > make your patch larger) Thanks for the review, the user can pass parameters to FreeBSD when booted as Dom0, I just find it uncomfortable to force the user into always setting something on the command line in order to boot. What do you mean with "haing the relevant drivers explicitly disable themselves under Xen"? Adding a gate on every one of those devices like "if (xen_pv_domain()) return (ENXIO);" in the identify/probe routine seems even worse. Roger. From owner-freebsd-xen@FreeBSD.ORG Fri Feb 14 17:51:38 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CBBE5124; Fri, 14 Feb 2014 17:51:38 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9D63E1458; Fri, 14 Feb 2014 17:51:38 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6C199B9E6; Fri, 14 Feb 2014 12:51:37 -0500 (EST) From: John Baldwin To: Andrew Cooper Subject: Re: [Xen-devel] [PATCH RFC 09/13] xen: change quality of the MADT ACPI enumerator Date: Fri, 14 Feb 2014 12:51:10 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <1980951.95r2q2cca3@ralph.baldwin.cx> <52FD7624.90202@citrix.com> In-Reply-To: <52FD7624.90202@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402141251.10278.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 12:51:37 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 17:51:38 -0000 On Thursday, February 13, 2014 8:49:24 pm Andrew Cooper wrote: > On 08/02/2014 21:42, John Baldwin wrote: > > On Tuesday, December 24, 2013 12:20:58 PM Roger Pau Monne wrote: > >> Lower the quality of the MADT ACPI enumerator, so on Xen Dom0 we can > >> force the usage of the Xen mptable enumerator even when ACPI is > >> detected. > > Hmm, so I think one question is why does the existing MADT parser > > not work with the MADT table provided by Xen? This may very well > > be correct, but if it's only a small change to make the existing > > MADT parser work with Xen's MADT table, that route might be > > preferable. > > > > For dom0, the MADT seen is the system MADT, which does not bear any > reality to dom0's topology. For PV domU, no MADT will be found. For > HVM domU, the MADT seen ought to represent (virtual) reality. Hmm, the other changes suggested that you do want to use the I/O APIC entries and interrupt overrides from the system MADT for dom0? Just not the CPU entries. Is that correct? -- John Baldwin From owner-freebsd-xen@FreeBSD.ORG Fri Feb 14 17:51:37 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AB8C121; Fri, 14 Feb 2014 17:51:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6D4C81457; Fri, 14 Feb 2014 17:51:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 57AA4B9B1; Fri, 14 Feb 2014 12:51:36 -0500 (EST) From: John Baldwin To: Roger Pau =?iso-8859-1?q?Monn=E9?= Subject: Re: [PATCH RFC 10/13] xen: add ACPI bus to xen_nexus when running as Dom0 Date: Fri, 14 Feb 2014 12:50:06 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <1387884062-41154-1-git-send-email-roger.pau@citrix.com> <2410827.IqfpSAhe3T@ralph.baldwin.cx> <52FDF217.3040005@citrix.com> In-Reply-To: <52FDF217.3040005@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201402141250.06829.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 14 Feb 2014 12:51:36 -0500 (EST) Cc: julien.grall@citrix.com, freebsd-xen@freebsd.org, freebsd-current@freebsd.org, kib@freebsd.org, xen-devel@lists.xenproject.org, gibbs@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 17:51:37 -0000 On Friday, February 14, 2014 5:38:15 am Roger Pau Monn=E9 wrote: > On 08/02/14 22:50, John Baldwin wrote: > > On Tuesday, December 24, 2013 12:20:59 PM Roger Pau Monne wrote: > >> Also disable a couple of ACPI devices that are not usable under Dom0. > >=20 > > Hmm, setting debug.acpi.disabled in this way is a bit hacky. It might > > be fine however if there's no way for the user to set it before booting > > the kernel (as opposed to haing the relevant drivers explicitly disable > > themselves under Xen which I think would be cleaner, but would also > > make your patch larger) >=20 > Thanks for the review, the user can pass parameters to FreeBSD when > booted as Dom0, I just find it uncomfortable to force the user into > always setting something on the command line in order to boot. Can the user set debug.acpi.disabled? If so, you are overriding their setting which would be bad. > What do you mean with "haing the relevant drivers explicitly disable > themselves under Xen"? Adding a gate on every one of those devices like > "if (xen_pv_domain()) return (ENXIO);" in the identify/probe routine > seems even worse. A check like this in probe() is what I had in mind, though I agree it's not perfect. =2D-=20 John Baldwin