From owner-freebsd-stable@FreeBSD.ORG Mon Feb 8 18:51:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EC771065672; Mon, 8 Feb 2010 18:51:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id D5CE28FC08; Mon, 8 Feb 2010 18:51:51 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o18IplY2034992 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 8 Feb 2010 20:51:47 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o18IpkcQ074553; Mon, 8 Feb 2010 20:51:46 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o18IpkNN074552; Mon, 8 Feb 2010 20:51:46 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 8 Feb 2010 20:51:46 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20100208185146.GO9991@deviant.kiev.zoral.com.ua> References: <4B6B89E7.8030002@sdf.lonestar.org> <201002081032.37841.jhb@freebsd.org> <20100208160600.GN9991@deviant.kiev.zoral.com.ua> <201002081315.06445.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4ZQ/M1iA+qg8otEW" Content-Disposition: inline In-Reply-To: <201002081315.06445.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Tom McLaughlin , freebsd-stable@freebsd.org Subject: Re: Recent MFC to 7 causes crash on VMware ESXi X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Feb 2010 18:51:52 -0000 --4ZQ/M1iA+qg8otEW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 08, 2010 at 01:15:06PM -0500, John Baldwin wrote: > On Monday 08 February 2010 11:06:00 am Kostik Belousov wrote: > > On Mon, Feb 08, 2010 at 10:32:37AM -0500, John Baldwin wrote: > > > On Monday 08 February 2010 9:56:36 am Kostik Belousov wrote: > > > > On Mon, Feb 08, 2010 at 09:49:00AM -0500, John Baldwin wrote: > > > > > On Saturday 06 February 2010 4:47:16 pm Tom McLaughlin wrote: > > > > > > John Baldwin wrote, On 02/05/2010 08:27 AM: > > > > > > > On Thursday 04 February 2010 10:00:55 pm Tom McLaughlin wrote: > > > > > > >> Hi all, a recent MFC to 7-STABLE has started to cause issues= for=20 > my VMs > > > > > > >> on VMware ESXi 3.5u4. After loading the mpt driver for the = LSI=20 > disk > > > > > > >> controller the VM just shuts off. The workaround is to chan= ge=20 > the disk > > > > > > >> controller to the BusLogic type. Still, it used to work up = until=20 > last > > > > > > >> week. The change was made around January 26th and based on = the=20 > commits > > > > > > >> that day I'm guessing it's either r203047 or r203073 > > > > > > >> > > > > > > >> I have the same issue with both amd64 and i386 VMs. This af= fects=20 > HEAD > > > > > > >> and 8-STABLE as well and first affected HEAD over the summer= . (I=20 > just > > > > > > >> worked around it and went about my business at the time. :-/= ) =20 > I've > > > > > > >> attached a dmesg from a kernel before the problem and one fr= om=20 > after it > > > > > > >> started. > > > > > > >=20 > > > > > > > What if you set 'hw.clfush_disable=3D1' from the loader? > > > > > > >=20 > > > > > >=20 > > > > > > Yes, that corrected it on all my VMs. I've talked to people on= ESXi=20 > 4 > > > > > > and they do not see the problem. I have yet to try 3.5u5 to se= e if=20 > this > > > > > > is a non-issue. 3.5 will be supported for awhile longer from= =20 > VMware. > > > > > > I'm going to try upgrading the box during the week. > > > > >=20 > > > > > I believe folks had to do this on HEAD/8.x as well. Perhaps we c= an=20 > > > > > automatically disable clflush if we are executing under VMware or= Xen: > > > > >=20 > > > > > Index: amd64/amd64/initcpu.c > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > --- amd64/amd64/initcpu.c (revision 203430) > > > > > +++ amd64/amd64/initcpu.c (working copy) > > > > > @@ -177,17 +177,16 @@ > > > > > if ((cpu_feature & CPUID_CLFSH) !=3D 0) > > > > > cpu_clflush_line_size =3D ((cpu_procinfo >> 8) & 0xff) * 8; > > > > > /* > > > > > - * XXXKIB: (temporary) hack to work around traps generated when > > > > > - * CLFLUSHing APIC registers window. > > > > > + * XXXKIB: (temporary) hack to work around traps generated > > > > > + * when CLFLUSHing APIC registers window under virtualization > > > > > + * environments. > > > > > */ > > > > > TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); > > > > > - if (cpu_vendor_id =3D=3D CPU_VENDOR_INTEL && !(cpu_feature & CP= UID_SS)=20 > && > > > > > - hw_clflush_disable =3D=3D -1) > > > > > + if (vm_guest !=3D 0 /* VM_GUEST_NO */ && hw_clflush_disable =3D= =3D -1) > > > > > cpu_feature &=3D ~CPUID_CLFSH; > > > > > /* > > > > > * Allow to disable CLFLUSH feature manually by > > > > > - * hw.clflush_disable tunable. This may help Xen guest on some= AMD > > > > > - * CPUs. > > > > > + * hw.clflush_disable tunable. > > > > > */ > > > > > if (hw_clflush_disable =3D=3D 1) > > > > > cpu_feature &=3D ~CPUID_CLFSH; > > > > > Index: i386/i386/initcpu.c > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > --- i386/i386/initcpu.c (revision 203430) > > > > > +++ i386/i386/initcpu.c (working copy) > > > > > @@ -724,17 +724,16 @@ > > > > > if ((cpu_feature & CPUID_CLFSH) !=3D 0) > > > > > cpu_clflush_line_size =3D ((cpu_procinfo >> 8) & 0xff) * 8; > > > > > /* > > > > > - * XXXKIB: (temporary) hack to work around traps generated when > > > > > - * CLFLUSHing APIC registers window. > > > > > + * XXXKIB: (temporary) hack to work around traps generated > > > > > + * when CLFLUSHing APIC registers window under virtualization > > > > > + * environments. > > > > > */ > > > > > TUNABLE_INT_FETCH("hw.clflush_disable", &hw_clflush_disable); > > > > > - if (cpu_vendor_id =3D=3D CPU_VENDOR_INTEL && !(cpu_feature & CP= UID_SS)=20 > && > > > > > - hw_clflush_disable =3D=3D -1) > > > > > + if (vm_guest !=3D 0 /* VM_GUEST_NO */ && hw_clflush_disable =3D= =3D -1) > > > > > cpu_feature &=3D ~CPUID_CLFSH; > > > > > /* > > > > > * Allow to disable CLFLUSH feature manually by > > > > > - * hw.clflush_disable tunable. This may help Xen guest on some= AMD > > > > > - * CPUs. > > > > > + * hw.clflush_disable tunable. > > > > > */ > > > > > if (hw_clflush_disable =3D=3D 1) > > > > > cpu_feature &=3D ~CPUID_CLFSH; > > > >=20 > > > > It might be better to "or" old condition, i.e. Intel without SS, and > > > > new one, vm_guest !=3D 0, instead of replacing the old ? > > >=20 > > > I thought the old condition only happened under VMware? > >=20 > > Reports I got where from XEN. >=20 > Ok. Those would also be covered under the vm_guest test as it is > non-zero for Xen, VMware, Parallels, etc. What I said was suggestion and not objection. Ignore me. --4ZQ/M1iA+qg8otEW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAktwXUIACgkQC3+MBN1Mb4jJiwCbBk2nq4+qqLeYoyRfviecLe1L sbUAoJ/t8OrLyHBWnRBX1AtJz36wGwoJ =sClO -----END PGP SIGNATURE----- --4ZQ/M1iA+qg8otEW--