From owner-svn-src-head@FreeBSD.ORG Tue Oct 28 03:04:46 2014
Return-Path: <owner-svn-src-head@FreeBSD.ORG>
Delivered-To: svn-src-head@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id BFB3DFA1
for <svn-src-head@freebsd.org>; Tue, 28 Oct 2014 03:04:46 +0000 (UTC)
Received: from mail-yh0-f48.google.com (mail-yh0-f48.google.com
[209.85.213.48])
(using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G2" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 782BFF16
for <svn-src-head@freebsd.org>; Tue, 28 Oct 2014 03:04:46 +0000 (UTC)
Received: by mail-yh0-f48.google.com with SMTP id v1so5246918yhn.21
for <svn-src-head@freebsd.org>; Mon, 27 Oct 2014 20:04:39 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:sender:content-type:mime-version:subject:from
:in-reply-to:date:cc:message-id:references:to;
bh=dOUmjB9f1DlvSnHMy/wJx8ZDyWnBg0+wkOS4Cxv1RK4=;
b=I3w4ECsyQFQlSBVb0KiyX1Hx66qaNf/d6WUnn4/7zzTFCX+TJwwxr4CGMzyKqKjmdb
G1fMg2KLjuJzWDZE0o4wklB2AvLMnr6wGQAm6j4iAnJZw6oI1zlTrhDLcKMzDxunPErh
dPHCsNTKfAhQTYH+kgqS0rqIvWhFS7C6QQ4a1bojz3QSFJWczwR71quK1vO9lWUvepJW
zTsORrsfNCjxKW5gq/uovR0kGfsh3NtJ2jsFAE9gG4Dc4fakSYr38U3uN+wd6bmQZ3pF
7UnXlsD/5dNgAr6kft1tdh3TbCIi4HZhfwyuuwuXGiZUT8bxKa09vDGpKeNQqDeZonoi
+QgA==
X-Gm-Message-State: ALoCoQkmHhXbL9mApP0yaQAZsTdv4bW0AfrTDkarX8fximh+Xf2wM31YRonhwrjwi+WUjZkeEJ9I
X-Received: by 10.236.216.82 with SMTP id f78mr106060yhp.33.1414461528574;
Mon, 27 Oct 2014 18:58:48 -0700 (PDT)
Received: from [192.168.0.14] (173-18-133-79.client.mchsi.com. [173.18.133.79])
by mx.google.com with ESMTPSA id v42sm56218yhn.31.2014.10.27.18.58.47
for <multiple recipients>
(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
Mon, 27 Oct 2014 18:58:47 -0700 (PDT)
Sender: Warner Losh <wlosh@bsdimp.com>
Content-Type: multipart/signed;
boundary="Apple-Mail=_E2B88923-0130-4DF4-A045-532A780EADC9";
protocol="application/pgp-signature"; micalg=pgp-sha512
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\))
Subject: Re: svn commit: r273214 - in head/sys: amd64/vmm/intel modules/vmm
From: Warner Losh <imp@bsdimp.com>
In-Reply-To: <5487187.5oKgfZ1XGT@ralph.baldwin.cx>
Date: Mon, 27 Oct 2014 20:58:36 -0500
Message-Id: <19D1B515-74B3-43A0-A017-B170FFAEBA6F@bsdimp.com>
References: <201410171320.s9HDKo53045297@svn.freebsd.org>
<1725598.2CCKLon8F3@ralph.baldwin.cx>
<2B7C3745-04E4-4FA9-A849-AECE54EA83A3@bsdimp.com>
<5487187.5oKgfZ1XGT@ralph.baldwin.cx>
To: John Baldwin <jhb@FreeBSD.org>
X-Mailer: Apple Mail (2.1878.6)
Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org,
src-committers <src-committers@freebsd.org>, Warner Losh <imp@freebsd.org>
X-BeenThere: svn-src-head@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: SVN commit messages for the src tree for head/-current
<svn-src-head.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-head>,
<mailto:svn-src-head-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head/>
List-Post: <mailto:svn-src-head@freebsd.org>
List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-head>,
<mailto:svn-src-head-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 28 Oct 2014 03:04:46 -0000
--Apple-Mail=_E2B88923-0130-4DF4-A045-532A780EADC9
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=windows-1252
On Oct 27, 2014, at 1:18 PM, John Baldwin <jhb@FreeBSD.org> wrote:
> On Monday, October 27, 2014 11:36:41 AM Warner Losh wrote:
>> On Oct 27, 2014, at 10:54 AM, John Baldwin <jhb@freebsd.org> wrote:
>>> On Friday, October 17, 2014 01:20:50 PM Warner Losh wrote:
>>>> Author: imp
>>>> Date: Fri Oct 17 13:20:49 2014
>>>> New Revision: 273214
>>>> URL: https://svnweb.freebsd.org/changeset/base/273214
>>>>=20
>>>> Log:
>>>> Fix build to not bogusly always rebuild vmm.ko.
>>>>=20
>>>> Rename vmx_assym.s to vmx_assym.h to reflect that file's actual use
>>>> and update vmx_support.S's include to match. Add vmx_assym.h to the
>>>> SRCS to that it gets properly added to the dependency list. Add
>>>> vmx_support.S to SRCS as well, so it gets built and needs fewer
>>>> special-case goo. Remove now-redundant special-case goo. Finally,
>>>> vmx_genassym.o doesn't need to depend on a hand expanded ${_ILINKS}
>>>> explicitly, that's all taken care of by beforedepend.
>>>>=20
>>>> With these items fixed, we no longer build vmm.ko every single time
>>>> through the modules on a KERNFAST build.
>>>=20
>>> So I cheered for this before, but it appears to be broken. :(
>>>=20
>>> Namely, I rebuilt world + kernel on my laptop this weekend (it was =
about a
>>> month old). My normal setup builds kernels with NO_KERNELCLEAN=3Dyes.=
On my
>>> next reboot when I started a bhyve VM I promptly got a panic due to =
a page
>>>=20
>>> fault in this assembly code:
>>> /*
>>> =09
>>> * If 'vmx->eptgen[curcpu]' is not identical to =
'pmap->pm_eptgen'
>>> * then we must invalidate all mappings associated with this =
EPTP.
>>> */
>>> =09
>>> movq PM_EPTGEN(%r11), %r10
>>> cmpq %r10, VMX_EPTGEN(%rsi, %rax, 8)
>>> je guest_restore
>>>=20
>>> (The 'cmpq' instruction)
>>>=20
>>> This change came to mind, so I blew away the 'vmm' module directory =
and
>>> rebuilt my kernel. Comparing the assembly of this instruction =
before and
>>> after used different values for VMX_EPTGEN. In other words, the
>>> NO_KERNELCLEAN=3Dyes build failed to regenerate vmx_assym.h and the =
build
>>> used stale values.
>>=20
>> Is there a way to force this condition for testing?
>=20
> You could checkout an older tree (probably before the recent merge of =
AMD SVM
> support) and build vmm.ko, then svn update and see if vmx_assym and
> vmx_support.o are updated.
>=20
> Actually, this was simpler:
>=20
> % cd sys/modules/vmm
> % make depend
> % make vmx_assym.h # reports nothing to do
> % touch machine/vmm.h # vmx_genassym.c includes this
> % make vmx_assym.h # should rebuild, but doesn=92t
Thanks!
>>> In particular, if you examine the generated .depend file, you will =
find
>>> that there are no dependencies recorded for vmx_genassym.o, so it is
>>> never rebuilt if any of the headers it includes are changed. In my =
case
>>> the panic happened to be one that was easily diagnosed, but I could
>>> imagine stale assym headers causing very odd crashes that would be =
quite
>>> hard to track down. I think these changes should be reverted if we =
can't
>>> fix the dependencies of the associated object files they are =
generated
>>> from. :(
>>=20
>> Give me a bit and I=92ll fix it. There=92s a number of implicit =
dependencies
>> that don=92t get recorded in the .depend file, iirc, so that=92s not =
completely
>> conclusive. Not building, though is kinda a big hint that something=92s=
>> amiss.
>=20
> I think the thing here is that for the assym files we don't record any
> dependency info at all. The main kernel build does record =
dependencies
> for genassym.o in .depend, so it must be doable.
True.
> In kern.pre.mk:
>=20
> GEN_CFILES=3D $S/$M/$M/genassym.c ${MFILES:T:S/.m$/.c/}
>=20
> and those are then explicitly passed to mkdep in kern.post.mk.
>=20
> So this fixes it:
>=20
> Index: Makefile
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- Makefile (revision 273555)
> +++ Makefile (working copy)
> @@ -4,6 +4,7 @@ KMOD=3D vmm
>=20
> SRCS=3D opt_acpi.h opt_ddb.h device_if.h bus_if.h pci_if.h
> SRCS+=3D vmx_assym.h svm_assym.h
> +DPSRCS=3D vmx_genassym.c svm_genassym.c
That=92s the magic I was looking for...
> CFLAGS+=3D -DVMM_KEEP_STATS -DSMP
> CFLAGS+=3D -I${.CURDIR}/../../amd64/vmm
>=20
> I'll try to track down all the other assym files and fix them as well.
Cool. There=92s three more (but I had only fixed two of them, since I =
didn=92t have a i386 build handy).
>> However, -DNO_CLEAN has always been a very-sharp edged tool that will =
cut
>> you in a number of ways, so there=92s no rush to back this out.
>=20
> This is the first time in many years that NO_KERNELCLEAN=3Dyes has =
been a
> problem for me. (worlds sometimes have issues, but kernels rarely =
do).
> Also, usually when it breaks it fails to compile, it doesn't compile =
and
> then panic. :(
But it is current=85 :)
Warner
--Apple-Mail=_E2B88923-0130-4DF4-A045-532A780EADC9
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org
iQIcBAEBCgAGBQJUTvhMAAoJEGwc0Sh9sBEAjAcP/3pgovT4SIGF175htGXM6qRp
ntMa5kE6M9ccJVZLBqCsMLZPYYP1cY10J6jLTxdVup+hfvT3VbD8UMeqmkCoK8Qf
iWD+IjzNEs8VQY2HBHD1mlPVHxgh8mcKPVu56WPRRsdfmfrk5SeKpBxQZCNfdAME
HpxYZXQfUoTD4evHqCCz5yL0UZpoCeLMhsN+Tf5iY+Ldm69Pq3259JNVbS32cBuw
zHSstSoRi/5QpbU6amzluUKXJrgr6ao63caR4zFITkU3YjGPzyHoBR77L2XxkT/q
WRSvh4VdkrqGeKmAO0zPMUcM4jRVEfuE6ayeJpAoGaEZ/7W9JR7sjgs9una4NEB9
77dRKL1KOrbam+tA4g7O37N34ubpOvTiMhZCbhiA0WE0cwGyUGvKXxKWuFXB7fkZ
usscxI7QXq9kI7bTyqk4MHsp1ZUcaP8vdohfRp1Ex6v2L6Zn6f7Kg/BOEi1ydWoQ
3mAHi5jKNNj+wXAwhQgu3ps9BDtNL5RlPIG++JZ3FB9WS//Tg64r8mSX7Rbfk/z3
9pLvwQ9evRv3xTWp0verqaxyBiWXkKu82TVd0MCBrwxp1x7uZHPDgaJnqF2xsTRj
UOHb3bsu5N1MQ/ZaarnL0iPsUoOV4CD+Y59SlQ+ZR5J3TAsfj68YsAtl7f5WyWgg
GptlW9WOkAmUN3CRYIjX
=gG2/
-----END PGP SIGNATURE-----
--Apple-Mail=_E2B88923-0130-4DF4-A045-532A780EADC9--