From owner-freebsd-stable@FreeBSD.ORG Mon Feb 8 16:06:07 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A4B5106568B; Mon, 8 Feb 2010 16:06:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 58E378FC12; Mon, 8 Feb 2010 16:06:05 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o18G61jB019807 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 8 Feb 2010 18:06:01 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o18G611X073489; Mon, 8 Feb 2010 18:06:01 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o18G60KF073488; Mon, 8 Feb 2010 18:06:00 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 8 Feb 2010 18:06:00 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20100208160600.GN9991@deviant.kiev.zoral.com.ua> References: <4B6B89E7.8030002@sdf.lonestar.org> <201002080949.00877.jhb@freebsd.org> <20100208145636.GK9991@deviant.kiev.zoral.com.ua> <201002081032.37841.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wmhq21yAGFMoSpeN" Content-Disposition: inline In-Reply-To: <201002081032.37841.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Tom McLaughlin , freebsd-stable@freebsd.org Subject: Re: Recent MFC to 7 causes crash on VMware ESXi X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Feb 2010 16:06:07 -0000 --wmhq21yAGFMoSpeN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 08, 2010 at 10:32:37AM -0500, John Baldwin wrote: > On Monday 08 February 2010 9:56:36 am Kostik Belousov wrote: > > On Mon, Feb 08, 2010 at 09:49:00AM -0500, John Baldwin wrote: > > > On Saturday 06 February 2010 4:47:16 pm Tom McLaughlin wrote: > > > > John Baldwin wrote, On 02/05/2010 08:27 AM: > > > > > On Thursday 04 February 2010 10:00:55 pm Tom McLaughlin wrote: > > > > >> Hi all, a recent MFC to 7-STABLE has started to cause issues for= my VMs > > > > >> on VMware ESXi 3.5u4. After loading the mpt driver for the LSI = disk > > > > >> controller the VM just shuts off. The workaround is to change t= he disk > > > > >> controller to the BusLogic type. Still, it used to work up unti= l last > > > > >> week. The change was made around January 26th and based on the = commits > > > > >> that day I'm guessing it's either r203047 or r203073 > > > > >> > > > > >> I have the same issue with both amd64 and i386 VMs. This affect= s HEAD > > > > >> and 8-STABLE as well and first affected HEAD over the summer. (= I just > > > > >> worked around it and went about my business at the time. :-/) I= 've > > > > >> attached a dmesg from a kernel before the problem and one from a= fter it > > > > >> started. > > > > >=20 > > > > > What if you set 'hw.clfush_disable=3D1' from the loader? > > > > >=20 > > > >=20 > > > > Yes, that corrected it on all my VMs. I've talked to people on ESX= i 4 > > > > and they do not see the problem. I have yet to try 3.5u5 to see if= this > > > > is a non-issue. 3.5 will be supported for awhile longer from VMwar= e. > > > > I'm going to try upgrading the box during the week. > > >=20 > > > I believe folks had to do this on HEAD/8.x as well. Perhaps we can= =20 > > > automatically disable clflush if we are executing under VMware or Xen: > > >=20 > > > Index: amd64/amd64/initcpu.c > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > --- amd64/amd64/initcpu.c (revision 203430) > > > +++ amd64/amd64/initcpu.c (working copy) > > > @@ -177,17 +177,16 @@ > > > if ((cpu_feature & CPUID_CLFSH) !=3D 0) > > > cpu_clflush_line_size =3D ((cpu_procinfo >> 8) & 0xff) * 8; > > > /* > > > - * XXXKIB: (temporary) hack to work around traps generated when > > > - * CLFLUSHing APIC registers window. > > > + * XXXKIB: (temporary) hack to work around traps generated > > > + * when CLFLUSHing APIC registers window under virtualization > > > + * environments. > > > */ > > > TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); > > > - if (cpu_vendor_id =3D=3D CPU_VENDOR_INTEL && !(cpu_feature & CPUID_= SS) && > > > - hw_clflush_disable =3D=3D -1) > > > + if (vm_guest !=3D 0 /* VM_GUEST_NO */ && hw_clflush_disable =3D=3D = -1) > > > cpu_feature &=3D ~CPUID_CLFSH; > > > /* > > > * Allow to disable CLFLUSH feature manually by > > > - * hw.clflush_disable tunable. This may help Xen guest on some AMD > > > - * CPUs. > > > + * hw.clflush_disable tunable. > > > */ > > > if (hw_clflush_disable =3D=3D 1) > > > cpu_feature &=3D ~CPUID_CLFSH; > > > Index: i386/i386/initcpu.c > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > --- i386/i386/initcpu.c (revision 203430) > > > +++ i386/i386/initcpu.c (working copy) > > > @@ -724,17 +724,16 @@ > > > if ((cpu_feature & CPUID_CLFSH) !=3D 0) > > > cpu_clflush_line_size =3D ((cpu_procinfo >> 8) & 0xff) * 8; > > > /* > > > - * XXXKIB: (temporary) hack to work around traps generated when > > > - * CLFLUSHing APIC registers window. > > > + * XXXKIB: (temporary) hack to work around traps generated > > > + * when CLFLUSHing APIC registers window under virtualization > > > + * environments. > > > */ > > > TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); > > > - if (cpu_vendor_id =3D=3D CPU_VENDOR_INTEL && !(cpu_feature & CPUID_= SS) && > > > - hw_clflush_disable =3D=3D -1) > > > + if (vm_guest !=3D 0 /* VM_GUEST_NO */ && hw_clflush_disable =3D=3D = -1) > > > cpu_feature &=3D ~CPUID_CLFSH; > > > /* > > > * Allow to disable CLFLUSH feature manually by > > > - * hw.clflush_disable tunable. This may help Xen guest on some AMD > > > - * CPUs. > > > + * hw.clflush_disable tunable. > > > */ > > > if (hw_clflush_disable =3D=3D 1) > > > cpu_feature &=3D ~CPUID_CLFSH; > >=20 > > It might be better to "or" old condition, i.e. Intel without SS, and > > new one, vm_guest !=3D 0, instead of replacing the old ? >=20 > I thought the old condition only happened under VMware? Reports I got where from XEN. --wmhq21yAGFMoSpeN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAktwNmgACgkQC3+MBN1Mb4jHegCeLLTpOtXKNdE5reQ8HEkDqnI9 9LMAn2jggrg3DW/nlP8Wvc5ux9L8BcJv =Ixp5 -----END PGP SIGNATURE----- --wmhq21yAGFMoSpeN--