From owner-freebsd-stable@FreeBSD.ORG Wed Feb 10 00:22:53 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FA61106566C; Wed, 10 Feb 2010 00:22:53 +0000 (UTC) (envelope-from tmclaugh@sdf.lonestar.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 37F148FC52; Wed, 10 Feb 2010 00:22:53 +0000 (UTC) Received: from straycat.dhs.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1A0MqGd054383; Wed, 10 Feb 2010 00:22:52 GMT (envelope-from tmclaugh@sdf.lonestar.org) Received: from tomcat.straycat.dhs.org (tomcat.straycat.dhs.org [192.168.2.213]) by straycat.dhs.org (8.14.1/8.14.1) with ESMTP id o1A0MooJ031162; Tue, 9 Feb 2010 19:22:51 -0500 (EST) Message-ID: <4B71FC5A.9080601@sdf.lonestar.org> Date: Tue, 09 Feb 2010 19:22:50 -0500 From: Tom McLaughlin User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.7) Gecko/20100120 Fedora/3.0.1-1.fc12 Lightning/1.0b2pre Thunderbird/3.0.1 MIME-Version: 1.0 To: John Baldwin References: <4B6B89E7.8030002@sdf.lonestar.org> <201002091352.24131.jhb@freebsd.org> <4B71DEFC.2000200@sdf.lonestar.org> <201002091723.44625.jhb@freebsd.org> In-Reply-To: <201002091723.44625.jhb@freebsd.org> X-Enigmail-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Cc: kib@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Recent MFC to 7 causes crash on VMware ESXi X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2010 00:22:53 -0000 John Baldwin wrote, On 02/09/2010 05:23 PM: > On Tuesday 09 February 2010 5:17:32 pm Tom McLaughlin wrote: >> John Baldwin wrote, On 02/09/2010 01:52 PM: >>> On Monday 08 February 2010 9:49:00 am John Baldwin wrote: >>>> On Saturday 06 February 2010 4:47:16 pm Tom McLaughlin wrote: >>>>> John Baldwin wrote, On 02/05/2010 08:27 AM: >>>>>> On Thursday 04 February 2010 10:00:55 pm Tom McLaughlin wrote: >>>>>>> Hi all, a recent MFC to 7-STABLE has started to cause issues for my > VMs >>>>>>> on VMware ESXi 3.5u4. After loading the mpt driver for the LSI disk >>>>>>> controller the VM just shuts off. The workaround is to change the > disk >>>>>>> controller to the BusLogic type. Still, it used to work up until last >>>>>>> week. The change was made around January 26th and based on the > commits >>>>>>> that day I'm guessing it's either r203047 or r203073 >>>>>>> >>>>>>> I have the same issue with both amd64 and i386 VMs. This affects HEAD >>>>>>> and 8-STABLE as well and first affected HEAD over the summer. (I just >>>>>>> worked around it and went about my business at the time. :-/) I've >>>>>>> attached a dmesg from a kernel before the problem and one from after > it >>>>>>> started. >>>>>> >>>>>> What if you set 'hw.clfush_disable=1' from the loader? >>>>>> >>>>> >>>>> Yes, that corrected it on all my VMs. I've talked to people on ESXi 4 >>>>> and they do not see the problem. I have yet to try 3.5u5 to see if this >>>>> is a non-issue. 3.5 will be supported for awhile longer from VMware. >>>>> I'm going to try upgrading the box during the week. >>>> >>>> I believe folks had to do this on HEAD/8.x as well. Perhaps we can >>>> automatically disable clflush if we are executing under VMware or Xen: >>> >>> Tom, were you able to verify that this patch fixes the problem for you >>> without requiring you to set the hw.clflush_disable tunable? >>> >> >> John, I'm getting the following build error on all branches: >> >> /usr/src/sys/amd64/amd64/initcpu.c: In function 'initializecpucache': >> /usr/src/sys/amd64/amd64/initcpu.c:184: error: 'vm_guest' undeclared >> (first use in this function) >> /usr/src/sys/amd64/amd64/initcpu.c:184: error: (Each undeclared >> identifier is reported only once >> /usr/src/sys/amd64/amd64/initcpu.c:184: error: for each function it >> appears in.) > > Oh foo. Can you add 'extern int vm_guest;' to that file near the top? Adding that to the top fixed the build on 8.x and HEAD. Both of those are also now working just fine. However, 7.x still does not build. cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mcmodel=kernel -mno-red-zone -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror vers.c linking kernel.debug initcpu.o(.text+0x272): In function `initializecpucache': /usr/src/sys/amd64/amd64/initcpu.c:186: undefined reference to `vm_guest' *** Error code 1 > >> *** Error code 1 >> >> >> tom >> >>>> Index: amd64/amd64/initcpu.c >>>> =================================================================== >>>> --- amd64/amd64/initcpu.c (revision 203430) >>>> +++ amd64/amd64/initcpu.c (working copy) >>>> @@ -177,17 +177,16 @@ >>>> if ((cpu_feature & CPUID_CLFSH) != 0) >>>> cpu_clflush_line_size = ((cpu_procinfo >> 8) & 0xff) * 8; >>>> /* >>>> - * XXXKIB: (temporary) hack to work around traps generated when >>>> - * CLFLUSHing APIC registers window. >>>> + * XXXKIB: (temporary) hack to work around traps generated >>>> + * when CLFLUSHing APIC registers window under virtualization >>>> + * environments. >>>> */ >>>> TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); >>>> - if (cpu_vendor_id == CPU_VENDOR_INTEL && !(cpu_feature & CPUID_SS) && >>>> - hw_clflush_disable == -1) >>>> + if (vm_guest != 0 /* VM_GUEST_NO */ && hw_clflush_disable == -1) >>>> cpu_feature &= ~CPUID_CLFSH; >>>> /* >>>> * Allow to disable CLFLUSH feature manually by >>>> - * hw.clflush_disable tunable. This may help Xen guest on some AMD >>>> - * CPUs. >>>> + * hw.clflush_disable tunable. >>>> */ >>>> if (hw_clflush_disable == 1) >>>> cpu_feature &= ~CPUID_CLFSH; >>>> Index: i386/i386/initcpu.c >>>> =================================================================== >>>> --- i386/i386/initcpu.c (revision 203430) >>>> +++ i386/i386/initcpu.c (working copy) >>>> @@ -724,17 +724,16 @@ >>>> if ((cpu_feature & CPUID_CLFSH) != 0) >>>> cpu_clflush_line_size = ((cpu_procinfo >> 8) & 0xff) * 8; >>>> /* >>>> - * XXXKIB: (temporary) hack to work around traps generated when >>>> - * CLFLUSHing APIC registers window. >>>> + * XXXKIB: (temporary) hack to work around traps generated >>>> + * when CLFLUSHing APIC registers window under virtualization >>>> + * environments. >>>> */ >>>> TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); >>>> - if (cpu_vendor_id == CPU_VENDOR_INTEL && !(cpu_feature & CPUID_SS) && >>>> - hw_clflush_disable == -1) >>>> + if (vm_guest != 0 /* VM_GUEST_NO */ && hw_clflush_disable == -1) >>>> cpu_feature &= ~CPUID_CLFSH; >>>> /* >>>> * Allow to disable CLFLUSH feature manually by >>>> - * hw.clflush_disable tunable. This may help Xen guest on some AMD >>>> - * CPUs. >>>> + * hw.clflush_disable tunable. >>>> */ >>>> if (hw_clflush_disable == 1) >>>> cpu_feature &= ~CPUID_CLFSH; >>>> >>>> -- >>>> John Baldwin >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >>>> >>> >> >> >> -- >> | tmclaugh at sdf.lonestar.org tmclaugh at FreeBSD.org | >> | FreeBSD http://www.FreeBSD.org | >> >> > -- | tmclaugh at sdf.lonestar.org tmclaugh at FreeBSD.org | | FreeBSD http://www.FreeBSD.org |