From owner-freebsd-amd64@FreeBSD.ORG Wed Mar 15 18:46:13 2006 Return-Path: X-Original-To: freebsd-amd64@freebsd.org Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31C8216A400 for ; Wed, 15 Mar 2006 18:46:13 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4467A43D70 for ; Wed, 15 Mar 2006 18:46:08 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 14B161A4E46; Wed, 15 Mar 2006 10:46:08 -0800 (PST) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 1212A51954; Wed, 15 Mar 2006 13:46:07 -0500 (EST) Date: Wed, 15 Mar 2006 13:46:06 -0500 From: Kris Kennaway To: JoaoBR Message-ID: <20060315184606.GA85629@xor.obsecurity.org> References: <200603140740.38388.joao@matik.com.br> <200603141914.54442.joao@matik.com.br> <20060315022800.GA47353@xor.obsecurity.org> <200603150601.26135.joao@matik.com.br> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <200603150601.26135.joao@matik.com.br> User-Agent: Mutt/1.4.2.1i Cc: freebsd-amd64@freebsd.org, Kris Kennaway Subject: Re: amd64 slower than i386 on identical AMD 64 system? X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Mar 2006 18:46:13 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 15, 2006 at 06:01:25AM -0300, JoaoBR wrote: > On Tuesday 14 March 2006 23:28, Kris Kennaway wrote: > > On Tue, Mar 14, 2006 at 07:14:54PM -0300, JoaoBR wrote: > > > I can confirm this too > > > SMP amd64s are having constant crashes when running >2GB and <4GB of = RAM. > > > In order not getting anything wrong I am talking about X2-SMP > > > mono-chip-MBs this is not happening on dual-chip-MB with two separate > > > processors. I run the same hardware as UP-amd64 and it never crashes > > > Since this crashes are more frequent with IPI_PREEMPTION I have now s= ome > > > servers under test running without PREEMPTION at all and appearently = the > > > crashes are gone > > > > Right, IPI_PREEMPTION is not stable (nor is it enabled by default). > > Why did you decide to use it? > > >=20 > for the obvious ... the never-ending-search for better performance > BTW this option is very bad documented and no hint in NOTES that it may n= ot=20 > work OK. Just as a general point of common sense: when you find things that aren't enabled by default, there's almost always a good reason for this. Sometimes the reason is clear (driver conflicts with another driver, etc), but for secret poorly-documented options that change kernel behaviour it should ring warning bells that it might not be a good idea to just set and forget. > > > Overall the amd64-SMP kernels running on X2 processors are extermly > > > sensitive to non polling NICs and are crashing often. The overall > > > performance also is bad. > > > Soon I change this cards into polling ones, seems XL is best, I do not > > > have crashes anymore. > > > Funny that single 64bit AMDs are running fine with non polling NICs e= ven > > > when running a SMP enabled kernel. Soon I put back the X2 ... boom. > > > > Crashing with or without the use of broken kernel options? >=20 > without, SMP kernel but POLLING enabled OK, please file a bug report including panic traces, etc. This is necessary for someone to analyze and (if appropriate) fix your problem. > > > My servers are cache servers in first place and I have some web and m= ail > > > server running amd64 and the cpu scheduling seems to work well. Overa= ll I > > > have the impression that the ULE scheduler is giving better performan= ce > > > on a machine with more than 2MB/s going through > > > > You need to be very careful when claiming bad performance: ULE is > > well-known to perform badly on many workloads. > > >=20 > well, read again > I said here that ULE is giving me BETTER performance And above in the quoted text you said bad performance, so who knows what settings you had then :-) Maybe you're blaming something on polling that is really the fault of ULE. > > In summary, you need to rule out whether your issues are resulting > > from a poor choice of non-standard kernel options, or are actually > > bugs in FreeBSD. > > >=20 > obvious, but we often do not know all for sure so it's good talking about= =20 >=20 > resuming my experience: >=20 > SMP with single dual-core processors on standard MBs are sensitive (cras= hing=20 > easily or time-outs) with non polling NICs >=20 > SMP with single dual-core processors are randomly crashing when >2GB and = <4GB=20 > on standard MBs >=20 > Both issues are not appearing at all when changing the X2 for a standard= =20 > athlon 64 processor, means same hardware, same OS version and kernel >=20 > Both issues are also not appearing when using the dual-core running same= =20 > hardware, same os version but UP kernel (only cutting options SMP). >=20 > I understand here that amd64 is still not dealing well with dual-cores an= d=20 > more than 2GB of RAM It really sounds like it could be a broken BIOS on your system (check for upgrades). AFAIK, dual-core systems are not known to have these problems in general. Kris --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (FreeBSD) iD8DBQFEGGDuWry0BWjoQKURAvQ0AKDmeIUxRQOIYrQvCPq5sWd+sjZ6xgCg3jBN ylcqmedSgCo0uzY1K2WEW4o= =Cedt -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM--