Date: Mon, 27 Oct 2014 14:18:17 -0400 From: John Baldwin <jhb@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers <src-committers@freebsd.org>, Warner Losh <imp@freebsd.org> Subject: Re: svn commit: r273214 - in head/sys: amd64/vmm/intel modules/vmm Message-ID: <5487187.5oKgfZ1XGT@ralph.baldwin.cx> In-Reply-To: <2B7C3745-04E4-4FA9-A849-AECE54EA83A3@bsdimp.com> References: <201410171320.s9HDKo53045297@svn.freebsd.org> <1725598.2CCKLon8F3@ralph.baldwin.cx> <2B7C3745-04E4-4FA9-A849-AECE54EA83A3@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday, October 27, 2014 11:36:41 AM Warner Losh wrote: > On Oct 27, 2014, at 10:54 AM, John Baldwin <jhb@freebsd.org> wrote: > > On Friday, October 17, 2014 01:20:50 PM Warner Losh wrote: > >> Author: imp > >> Date: Fri Oct 17 13:20:49 2014 > >> New Revision: 273214 > >> URL: https://svnweb.freebsd.org/changeset/base/273214 > >>=20 > >> Log: > >> Fix build to not bogusly always rebuild vmm.ko. > >> =20 > >> Rename vmx_assym.s to vmx_assym.h to reflect that file's actual u= se > >> and update vmx_support.S's include to match. Add vmx_assym.h to t= he > >> SRCS to that it gets properly added to the dependency list. Add > >> vmx_support.S to SRCS as well, so it gets built and needs fewer > >> special-case goo. Remove now-redundant special-case goo. Finally,= > >> vmx_genassym.o doesn't need to depend on a hand expanded ${_ILINK= S} > >> explicitly, that's all taken care of by beforedepend. > >> =20 > >> With these items fixed, we no longer build vmm.ko every single ti= me > >> through the modules on a KERNFAST build. > >=20 > > So I cheered for this before, but it appears to be broken. :( > >=20 > > Namely, I rebuilt world + kernel on my laptop this weekend (it was = about a > > month old). My normal setup builds kernels with NO_KERNELCLEAN=3Dy= es. On my > > next reboot when I started a bhyve VM I promptly got a panic due to= a page > >=20 > > fault in this assembly code: > > =09/* > > =09 > > =09 * If 'vmx->eptgen[curcpu]' is not identical to 'pmap->pm_eptgen= ' > > =09 * then we must invalidate all mappings associated with this EPT= P. > > =09 */ > > =09 > > =09movq=09PM_EPTGEN(%r11), %r10 > > =09cmpq=09%r10, VMX_EPTGEN(%rsi, %rax, 8) > > =09je=09guest_restore > >=20 > > (The 'cmpq' instruction) > >=20 > > This change came to mind, so I blew away the 'vmm' module directory= and > > rebuilt my kernel. Comparing the assembly of this instruction befo= re and > > after used different values for VMX_EPTGEN. In other words, the > > NO_KERNELCLEAN=3Dyes build failed to regenerate vmx_assym.h and the= build > > used stale values. >=20 > Is there a way to force this condition for testing? You could checkout an older tree (probably before the recent merge of A= MD SVM support) and build vmm.ko, then svn update and see if vmx_assym and vmx_support.o are updated. Actually, this was simpler: % cd sys/modules/vmm % make depend % make vmx_assym.h # reports nothing to do % touch machine/vmm.h # vmx_genassym.c includes this % make vmx_assym.h # should rebuild, but doesn't > > In particular, if you examine the generated .depend file, you will = find > > that there are no dependencies recorded for vmx_genassym.o, so it i= s > > never rebuilt if any of the headers it includes are changed. In my= case > > the panic happened to be one that was easily diagnosed, but I could= > > imagine stale assym headers causing very odd crashes that would be = quite > > hard to track down. I think these changes should be reverted if we= can't > > fix the dependencies of the associated object files they are genera= ted > > from. :( >=20 > Give me a bit and I=E2=80=99ll fix it. There=E2=80=99s a number of im= plicit dependencies > that don=E2=80=99t get recorded in the .depend file, iirc, so that=E2= =80=99s not completely > conclusive. Not building, though is kinda a big hint that something=E2= =80=99s > amiss. I think the thing here is that for the assym files we don't record any dependency info at all. The main kernel build does record dependencies= for genassym.o in .depend, so it must be doable. In kern.pre.mk: GEN_CFILES=3D $S/$M/$M/genassym.c ${MFILES:T:S/.m$/.c/} and those are then explicitly passed to mkdep in kern.post.mk. So this fixes it: Index: Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Makefile=09(revision 273555) +++ Makefile=09(working copy) @@ -4,6 +4,7 @@ KMOD=3D=09vmm =20 SRCS=3D=09opt_acpi.h opt_ddb.h device_if.h bus_if.h pci_if.h SRCS+=3D=09vmx_assym.h svm_assym.h +DPSRCS=3D=09vmx_genassym.c svm_genassym.c =20 CFLAGS+=3D -DVMM_KEEP_STATS -DSMP CFLAGS+=3D -I${.CURDIR}/../../amd64/vmm I'll try to track down all the other assym files and fix them as well. > However, -DNO_CLEAN has always been a very-sharp edged tool that will= cut > you in a number of ways, so there=E2=80=99s no rush to back this out.= This is the first time in many years that NO_KERNELCLEAN=3Dyes has been= a problem for me. (worlds sometimes have issues, but kernels rarely do).= Also, usually when it breaks it fails to compile, it doesn't compile an= d then panic. :( --=20 John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5487187.5oKgfZ1XGT>