From owner-freebsd-stable@FreeBSD.ORG Tue Nov 20 16:27:55 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC17116A41B; Tue, 20 Nov 2007 16:27:55 +0000 (UTC) (envelope-from lol@chistydom.ru) Received: from hermes.hw.ru (hermes.hw.ru [80.68.240.91]) by mx1.freebsd.org (Postfix) with ESMTP id D75BD13C45D; Tue, 20 Nov 2007 16:27:54 +0000 (UTC) (envelope-from lol@chistydom.ru) Received: from [80.68.244.40] (account a_popov@rbc.ru [80.68.244.40] verified) by hermes.hw.ru (CommuniGate Pro SMTP 5.0.13) with ESMTPA id 201584434; Tue, 20 Nov 2007 19:27:28 +0300 Message-ID: <47430AE8.7050408@chistydom.ru> Date: Tue, 20 Nov 2007 19:27:20 +0300 From: Alexey Popov User-Agent: Thunderbird 2.0.0.6 (X11/20070924) MIME-Version: 1.0 To: Ivan Voras References: <4741905E.8050300@chistydom.ru> <47419AB3.5030008@chistydom.ru> <4741B3DE.2000009@chistydom.ru> In-Reply-To: Content-Type: multipart/mixed; boundary="------------090001010301070405070406" Cc: freebsd-stable@freebsd.org Subject: Re: 2 x quad-core system is slower that 2 x dual core on FreeBSD X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Nov 2007 16:27:55 -0000 This is a multi-part message in MIME format. --------------090001010301070405070406 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi. Ivan Voras wrote: > Many people (including me) have run FreeBSD on machines like yours > without such problems, so let's dig further. > > You don't have WITNESS, INVARIANTS, DIAGNOSTICS or something similar > enabled? Can you try a generic SMP kernel (called "SMP" in 6.x; the > "GENERIC" in 7.x has SMP by default) and see how it works? > > Can you disable SMP and try with only one CPU (on the 2xquad machine)? > You can do it in loader.conf by setting kern.smp.disabled=1, or perhaps > in BIOS. If there's a problem in some hardware or a driver, you'd still > get a big load on sys time. You might also want to halt certain logical > CPUs in the OS itself (see smp(4) man page) and see if there's a certain > relationship between how many CPUs are running and what the sys load is. Now I'm running yesterday's FreeBSD 7.0-BETA3 amd64 with GENERIC kernel. I rebuilt kernel and world with clean make.conf. Also I rebuilt Apache, PHP and eAccelerator from scratch. I tried APC as well. No success. I tried 7-STABLE with UP kernel (GENERIC built without SMP config option). It works fine and can handle around 5-10 requests per second. It consumes %sys time is much less than %user time (see top output in attach). I.e. it seems to work good as a simple server with not so powerfull CPU. After that I rebuilt with SMP GENERIC kernel and put on that server 2 times more requests that UP could handle. For the first time it worked good. Then I increased load to 2.5 times more than UP. Immediately Apache child count increased to MaxClients (24), most of them in RUN state, and %sys became greater than %user (see attach). I think after some threshold of load FreeBSD is paying more CPU time to the management of running processes than to run them. Also I tried to halt CPUs by machdep.hlt_cpus sysctl, but in that case %sys in top was still much greater than %user. With best regards, Alexey Popov --------------090001010301070405070406 Content-Type: text/plain; name="top-SMP.txt" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="top-SMP.txt" bGFzdCBwaWQ6ICAxMTAwOyAgbG9hZCBhdmVyYWdlczogIDguNTUsICA1LjIwLCAgMi4zNSAg ICAgICAgICAgICAgICAgICAgdXAgMCswMDowNTozOSAgMTg6NTk6NTIKNDggcHJvY2Vzc2Vz OiAgMjIgcnVubmluZywgMjYgc2xlZXBpbmcKQ1BVIHN0YXRlczogIDUuOSUgdXNlciwgIDAu MCUgbmljZSwgODEuMyUgc3lzdGVtLCAgMC4wJSBpbnRlcnJ1cHQsIDEyLjglIGlkbGUKTWVt OiAyNDVNIEFjdGl2ZSwgMTRNIEluYWN0LCAxMDJNIFdpcmVkLCAxMDhLIENhY2hlLCA0OE0g QnVmLCAzNTQzTSBGcmVlClN3YXA6IDIwNDhNIFRvdGFsLCAyMDQ4TSBGcmVlCgogIFBJRCBV U0VSTkFNRSAgIFRIUiBQUkkgTklDRSAgIFNJWkUgICAgUkVTIFNUQVRFICBDICAgVElNRSAg IFdDUFUgQ09NTUFORAogMTA5MyB3d3cgICAgICAgICAgMSAxMDUgICAgMCA5NDUyNEsgMzk3 MTZLIENQVTEgICA3ICAgMDowOSAzNC40MSUgaHR0cGQKIDEwOTQgd3d3ICAgICAgICAgIDEg MTA1ICAgIDAgOTE0NTJLIDM5NzMySyBzZWxlY3QgNCAgIDA6MDkgMzQuMDElIGh0dHBkCiAx MDk3IHd3dyAgICAgICAgICAxICAtNCAgICAwICAgIDk4TSA0ODM5MksgUlVOICAgIDcgICAw OjEwIDMzLjQxJSBodHRwZAogMTA5OCB3d3cgICAgICAgICAgMSAxMDUgICAgMCA5MjQ3Nksg NDMxNzZLIENQVTQgICA3ICAgMDowOSAzMy4yNyUgaHR0cGQKIDEwOTkgd3d3ICAgICAgICAg IDEgIC00ICAgIDAgOTI0NzZLIDQwNzg0SyBSVU4gICAgNyAgIDA6MDkgMzMuMjElIGh0dHBk CiAxMTAwIHd3dyAgICAgICAgICAxICAtNCAgICAwIDkyNDc2SyA0MTA4MEsgUlVOICAgIDQg ICAwOjA5IDMyLjg3JSBodHRwZAogMTA5NSB3d3cgICAgICAgICAgMSAgLTQgICAgMCA5MjQ3 NksgNDA4MjRLIFJVTiAgICA2ICAgMDowOSAzMi43NCUgaHR0cGQKIDEwOTAgd3d3ICAgICAg ICAgIDEgIC00ICAgIDAgOTY1NzJLIDQyNzAwSyBSVU4gICAgNSAgIDA6MDkgMzIuNTQlIGh0 dHBkCiAxMDg5IHd3dyAgICAgICAgICAxICAtNCAgICAwIDkzNTA0SyA0MjAzMksgUlVOICAg IDcgICAwOjA5IDMyLjQxJSBodHRwZAogMTA5MSB3d3cgICAgICAgICAgMSAgLTQgICAgMCA5 NTU0OEsgNDQ5MDBLIFJVTiAgICA0ICAgMDowOSAzMS45NSUgaHR0cGQKIDEwOTYgd3d3ICAg ICAgICAgIDEgIC00ICAgIDAgOTg2MjBLIDQ3MTYwSyBSVU4gICAgNiAgIDA6MDkgMzEuODYl IGh0dHBkCiAxMDg2IHd3dyAgICAgICAgICAxICAtNCAgICAwIDk2NTcySyA0NTc1MksgUlVO ICAgIDYgICAwOjEwIDMwLjkyJSBodHRwZAogMTA4NyB3d3cgICAgICAgICAgMSAxMDQgICAg MCA5MjQ3NksgNDEwMTZLIENQVTcgICA2ICAgMDowOSAzMC43MCUgaHR0cGQKIDEwODggd3d3 ICAgICAgICAgIDEgMTA0ICAgIDAgOTI0NzZLIDM4MzMySyBDUFUyICAgNiAgIDA6MTAgMzAu NTElIGh0dHBkCiAxMDg1IHd3dyAgICAgICAgICAxIDEwNSAgICAwIDk3NTk2SyA0NDQxNksg Q1BVNSAgIDUgICAwOjA5IDMwLjIzJSBodHRwZAogMTA5MiB3d3cgICAgICAgICAgMSAgLTQg ICAgMCA5MjQ3NksgNDAxNzJLIFJVTiAgICA2ICAgMDowOCAyOS40NSUgaHR0cGQKCg== --------------090001010301070405070406 Content-Type: text/plain; name="top-UP.txt" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="top-UP.txt" bGFzdCBwaWQ6ICAyMjAzOyAgbG9hZCBhdmVyYWdlczogIDkuNzEsIDEwLjA4LCAgNy4zOCAg ICAgICAgICAgICAgICAgICAgdXAgMCswMDoxNzo0OCAgMTg6NTA6MzkKMzUgcHJvY2Vzc2Vz OiAgNSBydW5uaW5nLCAzMCBzbGVlcGluZwpDUFUgc3RhdGVzOiA4Mi4yJSB1c2VyLCAgMC4w JSBuaWNlLCAxMy44JSBzeXN0ZW0sICAwLjAlIGludGVycnVwdCwgIDQuMCUgaWRsZQpNZW06 IDEyOE0gQWN0aXZlLCAxNU0gSW5hY3QsIDEwOU0gV2lyZWQsIDEzMksgQ2FjaGUsIDg4TSBC dWYsIDM2NTdNIEZyZWUKU3dhcDogMjA0OE0gVG90YWwsIDIwNDhNIEZyZWUKCiAgUElEIFVT RVJOQU1FICAgVEhSIFBSSSBOSUNFICAgU0laRSAgICBSRVMgU1RBVEUgICAgVElNRSAgIFdD UFUgQ09NTUFORAogMjIwMSB3d3cgICAgICAgICAgMSAxMTcgICAgMCA5MjQ3NksgNDAzNTJL IFJVTiAgICAgIDA6MDIgMTAuNDglIGh0dHBkCiAyMjAzIHd3dyAgICAgICAgICAxIDExNiAg ICAwIDk2NTcySyA0MDkxMksgUlVOICAgICAgMDowMiAxMC4wNSUgaHR0cGQKIDIxOTUgd3d3 ICAgICAgICAgIDEgMTE3ICAgIDAgOTM1MDBLIDQzMDEySyBzZWxlY3QgICAwOjAyICA5LjYx JSBodHRwZAogMjIwMiB3d3cgICAgICAgICAgMSAgMjAgICAgMCA5MTQ1MksgNDEwNTZLIGxv Y2tmICAgIDA6MDIgIDguOTUlIGh0dHBkCiAyMTk0IHd3dyAgICAgICAgICAxIDExNyAgICAw ICAgMTAyTSA0OTQ0MEsgUlVOICAgICAgMDowMyAgOC4xMCUgaHR0cGQKIDIxOTIgd3d3ICAg ICAgICAgIDEgMTE0ICAgIDAgOTk2NDhLIDQ2MTY4SyBzZWxlY3QgICAwOjAzICA2Ljc2JSBo dHRwZAogMjE3OSB3d3cgICAgICAgICAgMSAgMjAgICAgMCA5MjQ3NksgNDE2NzJLIGxvY2tm ICAgIDA6MDQgIDYuNjklIGh0dHBkCiAyMTczIHd3dyAgICAgICAgICAxIDExOCAgICAwICAg MTAwTSA0ODkyMEsgUlVOICAgICAgMDowNCAgNS45MiUgaHR0cGQKIDIxNzQgd3d3ICAgICAg ICAgIDEgIDIwICAgIDAgOTI0NzZLIDQyOTY0SyBsb2NrZiAgICAwOjA0ICA1LjY3JSBodHRw ZAogIDg3OCBsbHAgICAgICAgICAgMSAgOTYgICAgMCAzMjkyOEsgIDQ1NzZLIHNlbGVjdCAg IDA6MDAgIDAuMDAlIHNzaGQKICA4OTEgcm9vdCAgICAgICAgIDEgIDIwICAgIDAgIDk2MTZL ICAyOTI0SyBwYXVzZSAgICAwOjAwICAwLjAwJSBjc2gKICA2OTEgcm9vdCAgICAgICAgIDEg IDk2ICAgIDAgIDg5NTJLICAyNTI4SyBzZWxlY3QgICAwOjAwICAwLjAwJSBudHBkCiAyMTYx IHJvb3QgICAgICAgICAxIDEzMSAgICAwIDg2MzMySyAxMzA4MEsgc2VsZWN0ICAgMDowMCAg MC4wMCUgaHR0cGQKIDIxNzggcm9vdCAgICAgICAgIDEgIDk2ICAgIDAgIDc2NTZLICAyMTY4 SyBSVU4gICAgICAwOjAwICAwLjAwJSB0b3AKICA4NzUgcm9vdCAgICAgICAgIDEgICA0ICAg IDAgMzI5MjhLICA0NTEySyBzYndhaXQgICAwOjAwICAwLjAwJSBzc2hkCiAgNzc0IHJvb3Qg ICAgICAgICAxICAgNCAgICAwICA0ODUySyAgMTY1Mksga3FyZWFkICAgMDowMCAgMC4wMCUg bWFzdGVyCgo= --------------090001010301070405070406--