From owner-svn-src-all@FreeBSD.ORG Mon Oct 27 18:29:49 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEC2EFD9; Mon, 27 Oct 2014 18:29:49 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A54AE99; Mon, 27 Oct 2014 18:29:49 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6B7BCB98A; Mon, 27 Oct 2014 14:29:48 -0400 (EDT) From: John Baldwin To: Warner Losh Subject: Re: svn commit: r273214 - in head/sys: amd64/vmm/intel modules/vmm Date: Mon, 27 Oct 2014 14:18:17 -0400 Message-ID: <5487187.5oKgfZ1XGT@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-PRERELEASE; KDE/4.14.2; amd64; ; ) In-Reply-To: <2B7C3745-04E4-4FA9-A849-AECE54EA83A3@bsdimp.com> References: <201410171320.s9HDKo53045297@svn.freebsd.org> <1725598.2CCKLon8F3@ralph.baldwin.cx> <2B7C3745-04E4-4FA9-A849-AECE54EA83A3@bsdimp.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 27 Oct 2014 14:29:48 -0400 (EDT) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers , Warner Losh X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 18:29:49 -0000 On Monday, October 27, 2014 11:36:41 AM Warner Losh wrote: > On Oct 27, 2014, at 10:54 AM, John Baldwin wrote: > > On Friday, October 17, 2014 01:20:50 PM Warner Losh wrote: > >> Author: imp > >> Date: Fri Oct 17 13:20:49 2014 > >> New Revision: 273214 > >> URL: https://svnweb.freebsd.org/changeset/base/273214 > >>=20 > >> Log: > >> Fix build to not bogusly always rebuild vmm.ko. > >> =20 > >> Rename vmx_assym.s to vmx_assym.h to reflect that file's actual u= se > >> and update vmx_support.S's include to match. Add vmx_assym.h to t= he > >> SRCS to that it gets properly added to the dependency list. Add > >> vmx_support.S to SRCS as well, so it gets built and needs fewer > >> special-case goo. Remove now-redundant special-case goo. Finally,= > >> vmx_genassym.o doesn't need to depend on a hand expanded ${_ILINK= S} > >> explicitly, that's all taken care of by beforedepend. > >> =20 > >> With these items fixed, we no longer build vmm.ko every single ti= me > >> through the modules on a KERNFAST build. > >=20 > > So I cheered for this before, but it appears to be broken. :( > >=20 > > Namely, I rebuilt world + kernel on my laptop this weekend (it was = about a > > month old). My normal setup builds kernels with NO_KERNELCLEAN=3Dy= es. On my > > next reboot when I started a bhyve VM I promptly got a panic due to= a page > >=20 > > fault in this assembly code: > > =09/* > > =09 > > =09 * If 'vmx->eptgen[curcpu]' is not identical to 'pmap->pm_eptgen= ' > > =09 * then we must invalidate all mappings associated with this EPT= P. > > =09 */ > > =09 > > =09movq=09PM_EPTGEN(%r11), %r10 > > =09cmpq=09%r10, VMX_EPTGEN(%rsi, %rax, 8) > > =09je=09guest_restore > >=20 > > (The 'cmpq' instruction) > >=20 > > This change came to mind, so I blew away the 'vmm' module directory= and > > rebuilt my kernel. Comparing the assembly of this instruction befo= re and > > after used different values for VMX_EPTGEN. In other words, the > > NO_KERNELCLEAN=3Dyes build failed to regenerate vmx_assym.h and the= build > > used stale values. >=20 > Is there a way to force this condition for testing? You could checkout an older tree (probably before the recent merge of A= MD SVM support) and build vmm.ko, then svn update and see if vmx_assym and vmx_support.o are updated. Actually, this was simpler: % cd sys/modules/vmm % make depend % make vmx_assym.h # reports nothing to do % touch machine/vmm.h # vmx_genassym.c includes this % make vmx_assym.h # should rebuild, but doesn't > > In particular, if you examine the generated .depend file, you will = find > > that there are no dependencies recorded for vmx_genassym.o, so it i= s > > never rebuilt if any of the headers it includes are changed. In my= case > > the panic happened to be one that was easily diagnosed, but I could= > > imagine stale assym headers causing very odd crashes that would be = quite > > hard to track down. I think these changes should be reverted if we= can't > > fix the dependencies of the associated object files they are genera= ted > > from. :( >=20 > Give me a bit and I=E2=80=99ll fix it. There=E2=80=99s a number of im= plicit dependencies > that don=E2=80=99t get recorded in the .depend file, iirc, so that=E2= =80=99s not completely > conclusive. Not building, though is kinda a big hint that something=E2= =80=99s > amiss. I think the thing here is that for the assym files we don't record any dependency info at all. The main kernel build does record dependencies= for genassym.o in .depend, so it must be doable. In kern.pre.mk: GEN_CFILES=3D $S/$M/$M/genassym.c ${MFILES:T:S/.m$/.c/} and those are then explicitly passed to mkdep in kern.post.mk. So this fixes it: Index: Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Makefile=09(revision 273555) +++ Makefile=09(working copy) @@ -4,6 +4,7 @@ KMOD=3D=09vmm =20 SRCS=3D=09opt_acpi.h opt_ddb.h device_if.h bus_if.h pci_if.h SRCS+=3D=09vmx_assym.h svm_assym.h +DPSRCS=3D=09vmx_genassym.c svm_genassym.c =20 CFLAGS+=3D -DVMM_KEEP_STATS -DSMP CFLAGS+=3D -I${.CURDIR}/../../amd64/vmm I'll try to track down all the other assym files and fix them as well. > However, -DNO_CLEAN has always been a very-sharp edged tool that will= cut > you in a number of ways, so there=E2=80=99s no rush to back this out.= This is the first time in many years that NO_KERNELCLEAN=3Dyes has been= a problem for me. (worlds sometimes have issues, but kernels rarely do).= Also, usually when it breaks it fails to compile, it doesn't compile an= d then panic. :( --=20 John Baldwin