From owner-freebsd-performance@FreeBSD.ORG Mon Apr 9 11:36:07 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 90AF416A40B for ; Mon, 9 Apr 2007 11:36:07 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.233]) by mx1.freebsd.org (Postfix) with ESMTP id 4DFC213C44B for ; Mon, 9 Apr 2007 11:36:07 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: by wx-out-0506.google.com with SMTP id s18so1330068wxc for ; Mon, 09 Apr 2007 04:36:06 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=EdOU525LJmHlannyK9BDCvJaF8DWnCDKME45Gkwe7tfNkJqlwDdNGFb36gxIdkzACujt4Lw0BIEgCldsJHTPj24xi8eHdlhLVvE4GdbPmCJn0Pf1K51BgWhSANdJRJgFlsBK18KA/Gn9aAu3QGamlFLeAsxZnCcBcLNckSiXeOY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=SI6YMmmAOclGjTpjax/AbsG7HUCdF3BEw3YuqOVGhDcbqNhFc0dC2gPSKbkhNzw2gAxHGijT/XmXFpuSEnO75iA9uqQZl2s0tKzPU8oDyH3dXz8ZRPT+M0QItyw2K8CAwH7CJNLs4W1xhwHzw0rO7c8eaa75BIm4J12OZlHL034= Received: by 10.78.118.5 with SMTP id q5mr847040huc.1176117026770; Mon, 09 Apr 2007 04:10:26 -0700 (PDT) Received: by 10.78.188.9 with HTTP; Mon, 9 Apr 2007 04:10:26 -0700 (PDT) Message-ID: <70e8236f0704090410r5457e32ct3928edea654868b6@mail.gmail.com> Date: Mon, 9 Apr 2007 12:10:26 +0100 From: "Joao Barros" To: "Danny Knaggs" In-Reply-To: <46160016.8080504@googlemail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <46160016.8080504@googlemail.com> Cc: freebsd-performance@freebsd.org, =?ISO-8859-1?Q?S=F8ren_Schmidt?= , freebsd-sparc64@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: FreeBSD 6.2 on SPARC64/x86 with Promise IDE Controller X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2007 11:36:07 -0000 I think nullpt sent you off track when he told you to try freebsd-performan= ce. I believe this is an issue for drivers and/or sparc (or maybe current) I think I have a TX2 too. I can try it on my Ultra 5 if more testing is required. PS: I'm cc'ing S=F8ren Schmidt, he's the maintainer of most of the ata driv= ers On 4/6/07, Danny Knaggs wrote: > Hello all! > > This is first time I've used a mailing list, so bear with me! > > > I've been asked to submit my findings of the ata driver in FreeBSD 6.2 > on my sparc64 and x86 box from bsdforums.org. > > Link to my thread: http://www.bsdforums.org/forums/showthread.php?t=3D486= 82 > > > I've just installed a Promise IDE Controller card (Ultra 133 TX2 - > PDC20269) in my Sun Ultra 10 and have come across a slight snag. > > If I don't put in "hw.ata.ata_dma=3D0" in the loader options I get DMA > timeout errors after it has queried the HDD on the Promise controller. > > I have found a link which someone else has a similar problem (NetBSD on > Alpha) which maybe useful: > http://archive.netbsd.se/?ml=3Dfreebsd-alpha&a=3D2007-02&t=3D3177803 > > Now, after BSD has loaded I can successfully change the DMA mode to > UDMA66 on the HDD without any problems (get ~30MB/s transfer rate, > compared to ~15MB/s when using the on-board controller using "dd"). Any > higher and I get DMA timeout messages. > > The HDD works fine when it's attached to the on-board controller. > > > Now, I thought I try the same Promise IDE card in x86 box with FreeBSD > 6.2 and found something interesting... > > The HDD will not operate correctly at UDMA133 - it performs very slowly > (<15Mb/s). > > Forcing the HDD to run at UDMA100 gives me 64Mb/s transfer. Which is > roughly what I expect. > > > So, it seems something is broken with the ATA driver - Sparc/Alpha > getting the worse of it! > > If anyone has ideas/brainwaves/etc - I'm willing to give it a whirl! > > > Thanks in advance. > > > Dan. > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd= .org" > --=20 Joao Barros From owner-freebsd-performance@FreeBSD.ORG Mon Apr 9 16:10:37 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4639816A402 for ; Mon, 9 Apr 2007 16:10:37 +0000 (UTC) (envelope-from knaggsy2000@googlemail.com) Received: from mail35-kcom.uk.cleanport.com (mail35-kcom.uk.cleanport.com [212.79.248.233]) by mx1.freebsd.org (Postfix) with ESMTP id 9712613C468 for ; Mon, 9 Apr 2007 16:10:36 +0000 (UTC) (envelope-from knaggsy2000@googlemail.com) X-VirusChecked: NOT checked Received: from (unresolved) ([212.50.160.34] HELO=smtpout.karoo.kcom.com) by mail35-kcom.uk.cleanport.com (CleanSMTPd 1.5.5) with ESMTP id 46213877-0; Mon, 09 Apr 2007 18:10:29 +0200 Received: from adsl-83-100-231-141.karoo.kcom.com ([83.100.231.141] helo=sedna.local) by smtpout.karoo.kcom.comwith esmtp (Exim 4.30) id 1HawRo-0001s5-8p server-id smtp-in2; Mon, 09 Apr 2007 17:10:28 +0100 Received: from [10.1.1.1] (asteroid.local [10.1.1.1]) by sedna.local (Postfix) with ESMTP id 1FFFF33C19; Mon, 9 Apr 2007 16:13:41 +0100 (BST) Message-ID: <461A5815.3080304@googlemail.com> Date: Mon, 09 Apr 2007 16:13:25 +0100 From: Danny Knaggs User-Agent: Thunderbird 1.5.0.10 (X11/20070407) MIME-Version: 1.0 To: Joao Barros References: <46160016.8080504@googlemail.com> <70e8236f0704090410r5457e32ct3928edea654868b6@mail.gmail.com> In-Reply-To: <70e8236f0704090410r5457e32ct3928edea654868b6@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: FreeBSD 6.2 on SPARC64/x86 with Promise IDE Controller X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2007 16:10:37 -0000 Maybe you're right that nullpt sent me off course. However, thanks for pointing this e-mail to correct person who needs to=20 investigate the issue. Joao Barros wrote: > I think nullpt sent you off track when he told you to try=20 > freebsd-performance. > I believe this is an issue for drivers and/or sparc (or maybe current) >=20 > I think I have a TX2 too. I can try it on my Ultra 5 if more testing > is required. >=20 > PS: I'm cc'ing S=F8ren Schmidt, he's the maintainer of most of the ata=20 > drivers From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 04:48:02 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 45F0616A404 for ; Tue, 10 Apr 2007 04:48:02 +0000 (UTC) (envelope-from kevin@insidesystems.net) Received: from imap.insidesystems.net (imap.insidesystems.net [206.216.149.56]) by mx1.freebsd.org (Postfix) with ESMTP id 22C3413C4B7 for ; Tue, 10 Apr 2007 04:48:02 +0000 (UTC) (envelope-from kevin@insidesystems.net) Received: from [68.32.227.193] (helo=[127.0.0.1]) by imap.insidesystems.net with esmtpa (Exim 4.66 (FreeBSD)) (envelope-from ) id 1Hb7bA-0006iS-F6 for freebsd-performance@freebsd.org; Tue, 10 Apr 2007 04:04:52 +0000 Message-ID: <461B0CD0.8090404@insidesystems.net> Date: Tue, 10 Apr 2007 00:04:32 -0400 From: Kevin Way Organization: Inside Systems, Inc. User-Agent: Thunderbird 2.0b2 (Windows/20070116) MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms090905020204060708080506" Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 04:48:02 -0000 This is a cryptographically signed message in MIME format. --------------ms090905020204060708080506 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Kris Kennaway wrote: > If so, then your task is the following: > > Make SYSV semaphores less dumb about process wakeups. Currently > whenever the semaphore state changes, all processes sleeping on the > semaphore are woken, even if we only have released enough resources > for one waiting process to claim. i.e. there is a thundering herd > wakeup situation which destroys performance at high loads. Fixing > this will involve replacing the wakeup() calls with appropriate > amounts of wakeup_one(). Could this cause problem cause a situation where an 8-Core system was 50-75% slower than an otherwise equivalent 2-Core system? I have a graph of my sysbench/pgsql results here: http://blog.insidesystems.net/files/sysctl-pgsq-amd64-wtf.png As the graph shows, the 8-core system is about half the speed of the 2-core system at 2 simultaneous threads, and it decays down to approximately 1/4 the speed of the 2-core system as the # of threads hits 5. All other (non-pgsql, non-sysv) tests came back approximately as expected, but I'm left wondering if I did something wrong, or if 8 cpus are slower than 2, when it comes to Postgres on currently available FreeBSD. Kevin Way Inside Systems, Inc. (Full detail of what I did is available at: http://blog.insidesystems.net/articles/2007/04/09/what-did-i-do-wrong ) --------------ms090905020204060708080506 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMljCC A2IwggLLoAMCAQICEAvaCxfBP4mOqwl0erTOLjMwDQYJKoZIhvcNAQECBQAwXzELMAkGA1UE BhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAxIFB1Ymxp YyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk4MDUxMjAwMDAwMFoXDTA4 MDUxMjIzNTk1OVowgcwxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJp U2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRv cnkvUlBBIEluY29ycC4gQnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2ln biBDbGFzcyAxIENBIEluZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0 ZWQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBALtaRIoEFrtV/QN6ii2UTxV4NrgNSrJv nFS/vOh3Kp258Gi7ldkxQXB6gUu5SBNWLccI4YRCq8CikqtEXKpC8IIOAukv+8I7u77JJwpd trA2QjO1blSIT4dKvxna+RXoD4e2HOPMxpqOf2okkuP84GW6p7F+78nbN2rISsgJBuSZAgMB AAGjgbAwga0wDwYDVR0TBAgwBgEB/wIBADBHBgNVHSAEQDA+MDwGC2CGSAGG+EUBBwEBMC0w KwYIKwYBBQUHAgEWH3d3dy52ZXJpc2lnbi5jb20vcmVwb3NpdG9yeS9SUEEwMQYDVR0fBCow KDAmoCSgIoYgaHR0cDovL2NybC52ZXJpc2lnbi5jb20vcGNhMS5jcmwwCwYDVR0PBAQDAgEG MBEGCWCGSAGG+EIBAQQEAwIBBjANBgkqhkiG9w0BAQIFAAOBgQACfZ5vRUs4oLje6VNkIbzk TCuPHv6SQKzYCjlqoTIhLAebq1n+0mIafVU4sDdz3PQHZmNiveFTcFKH56jYUulbLarh3s+s MVTUixnI2COo7wQrMn0sGBzIfImoLnfyRNFlCk10te7TG5JzdC6JOzUTcudAMZrTssSr51a+ i+P7FTCCBJQwggP9oAMCAQICEH//rl4NoatSoVbKWNMj7LQwDQYJKoZIhvcNAQEFBQAwgcwx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3 b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29ycC4g QnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENBIElu ZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQwHhcNMDcwMzA4MDAw MDAwWhcNMDcwNTA3MjM1OTU5WjCCAQYxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29t L3JlcG9zaXRvcnkvUlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4wHAYDVQQL ExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxJjAkBgNVBAsTHURpZ2l0YWwgSUQgQ2xhc3MgMSAt IE5ldHNjYXBlMRIwEAYDVQQDFAlLZXZpbiBXYXkxJjAkBgkqhkiG9w0BCQEWF2tldmluQGlu c2lkZXN5c3RlbXMubmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArwYwJ9u1 sjZXgnLX1jzAsrIw13m/UOl3OyFkNdnZEo3CH2IiequADpO3FaLLJe7AAIlnP/VDGxv+QHQ1 +eOEu+8jI2xSdTjTMBIxF5fugzEhy1fdQ+QBta85x76+bO4e5XAJUNr8Ao5H8FpCvYlM26hO 5NkUTXdfI7JPszS6AP/e0E6Xrj3zsbZA8HmhOhidH9fG6TNwTPmOG/ji9MvX55OdJcJCIazq cA1kGsgHyX9t62Wn+3NoZBGpFOx+3NVDQglain6VnAHtKwJqcCnRZHtLOO8ofUlZeOR/9beg inkyyKf3F0tyfT8yeprdZ0NreJti9MmhuCj6HmrmO0uGWQIDAQABo4G1MIGyMAkGA1UdEwQC MAAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXAzAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3 dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDBAYI KwYBBQUHAwIwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC52ZXJpc2lnbi5jb20vY2xh c3MxLmNybDANBgkqhkiG9w0BAQUFAAOBgQCJJJAQdU1zobfScAJaaJ76DvJrbJmloOItqhri dCE2396gPTIY9NQjRI3kvbuh1Qzz3voPFeKOlnNXiAJsrwvFCgwFXCxzXZF0J0Wn/Ci6O3YZ BoGIBBgfrt3d5gPkCMTcTDoKqFFLS7fu6eOab8oiGUvKYcXlMxqU8WK4Y3JS2DCCBJQwggP9 oAMCAQICEH//rl4NoatSoVbKWNMj7LQwDQYJKoZIhvcNAQEFBQAwgcwxFzAVBgNVBAoTDlZl cmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQL Ez13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29ycC4gQnkgUmVmLixMSUFC LkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENBIEluZGl2aWR1YWwgU3Vi c2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQwHhcNMDcwMzA4MDAwMDAwWhcNMDcwNTA3 MjM1OTU5WjCCAQYxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2ln biBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkv UlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4wHAYDVQQLExVQZXJzb25hIE5v dCBWYWxpZGF0ZWQxJjAkBgNVBAsTHURpZ2l0YWwgSUQgQ2xhc3MgMSAtIE5ldHNjYXBlMRIw EAYDVQQDFAlLZXZpbiBXYXkxJjAkBgkqhkiG9w0BCQEWF2tldmluQGluc2lkZXN5c3RlbXMu bmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArwYwJ9u1sjZXgnLX1jzAsrIw 13m/UOl3OyFkNdnZEo3CH2IiequADpO3FaLLJe7AAIlnP/VDGxv+QHQ1+eOEu+8jI2xSdTjT MBIxF5fugzEhy1fdQ+QBta85x76+bO4e5XAJUNr8Ao5H8FpCvYlM26hO5NkUTXdfI7JPszS6 AP/e0E6Xrj3zsbZA8HmhOhidH9fG6TNwTPmOG/ji9MvX55OdJcJCIazqcA1kGsgHyX9t62Wn +3NoZBGpFOx+3NVDQglain6VnAHtKwJqcCnRZHtLOO8ofUlZeOR/9beginkyyKf3F0tyfT8y eprdZ0NreJti9MmhuCj6HmrmO0uGWQIDAQABo4G1MIGyMAkGA1UdEwQCMAAwRAYDVR0gBD0w OzA5BgtghkgBhvhFAQcXAzAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5j b20vcnBhMAsGA1UdDwQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwMwYD VR0fBCwwKjAooCagJIYiaHR0cDovL2NybC52ZXJpc2lnbi5jb20vY2xhc3MxLmNybDANBgkq hkiG9w0BAQUFAAOBgQCJJJAQdU1zobfScAJaaJ76DvJrbJmloOItqhridCE2396gPTIY9NQj RI3kvbuh1Qzz3voPFeKOlnNXiAJsrwvFCgwFXCxzXZF0J0Wn/Ci6O3YZBoGIBBgfrt3d5gPk CMTcTDoKqFFLS7fu6eOab8oiGUvKYcXlMxqU8WK4Y3JS2DGCBKowggSmAgEBMIHhMIHMMRcw FQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azFGMEQGA1UECxM9d3d3LnZlcmlzaWduLmNvbS9yZXBvc2l0b3J5L1JQQSBJbmNvcnAuIEJ5 IFJlZi4sTElBQi5MVEQoYyk5ODFIMEYGA1UEAxM/VmVyaVNpZ24gQ2xhc3MgMSBDQSBJbmRp dmlkdWFsIFN1YnNjcmliZXItUGVyc29uYSBOb3QgVmFsaWRhdGVkAhB//65eDaGrUqFWyljT I+y0MAkGBSsOAwIaBQCgggKdMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcN AQkFMQ8XDTA3MDQxMDA0MDQzMlowIwYJKoZIhvcNAQkEMRYEFOAloinKgVA68TJAdhkohI9+ HR1qMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqG SIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIHyBgkrBgEEAYI3EAQxgeQwgeEw gcwxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBO ZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29y cC4gQnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENB IEluZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQCEH//rl4NoatS oVbKWNMj7LQwgfQGCyqGSIb3DQEJEAILMYHkoIHhMIHMMRcwFQYDVQQKEw5WZXJpU2lnbiwg SW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazFGMEQGA1UECxM9d3d3LnZl cmlzaWduLmNvbS9yZXBvc2l0b3J5L1JQQSBJbmNvcnAuIEJ5IFJlZi4sTElBQi5MVEQoYyk5 ODFIMEYGA1UEAxM/VmVyaVNpZ24gQ2xhc3MgMSBDQSBJbmRpdmlkdWFsIFN1YnNjcmliZXIt UGVyc29uYSBOb3QgVmFsaWRhdGVkAhB//65eDaGrUqFWyljTI+y0MA0GCSqGSIb3DQEBAQUA BIIBAIB2yroqPoplvl518PoJqlAbEJxbAYrmN/DksTOldrwipRWfJdXrkUC+KiY1tZeL2ctB d4YX16Y7Iz0b9waLx/QvpoIOJOapjIMT4ARITgqMpnjkyWJM60oTVs9INpsSu9cPr2Wh/a06 nChFcd+xiUt4v6ZMIkrml0BkT9yObH0DM6VW/cus0PJ8/pXfan4TH9O6hm4ebsLJ7wBQ/vgc UA5gOThwLpiMfUgqUxbRzQ6ocsldaMKGuKkr6euQMUXTQ8aG/9n1Kfb5zrlP/lc+uuIe4fb+ dI/kKWM1ndicNEWk0iJOqBB/+tWP96Ncam2NxvRcHLqAK6O/k/C56Of39TIAAAAAAAA= --------------ms090905020204060708080506-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 04:57:55 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AA05F16A403 for ; Tue, 10 Apr 2007 04:57:55 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 9878213C4CC for ; Tue, 10 Apr 2007 04:57:55 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 8C0261A4D87; Mon, 9 Apr 2007 21:58:00 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id B96F8515AC; Tue, 10 Apr 2007 00:57:54 -0400 (EDT) Date: Tue, 10 Apr 2007 00:57:54 -0400 From: Kris Kennaway To: Kevin Way Message-ID: <20070410045754.GA41769@xor.obsecurity.org> References: <461B0CD0.8090404@insidesystems.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline In-Reply-To: <461B0CD0.8090404@insidesystems.net> User-Agent: Mutt/1.4.2.2i Cc: freebsd-performance@freebsd.org Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 04:57:55 -0000 --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 12:04:32AM -0400, Kevin Way wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if we only have released enough resources > >for one waiting process to claim. i.e. there is a thundering herd > >wakeup situation which destroys performance at high loads. Fixing > >this will involve replacing the wakeup() calls with appropriate > >amounts of wakeup_one(). > Could this cause problem cause a situation where an 8-Core system was=20 > 50-75% slower than an otherwise equivalent 2-Core system? >=20 > I have a graph of my sysbench/pgsql results here: >=20 > http://blog.insidesystems.net/files/sysctl-pgsq-amd64-wtf.png >=20 > As the graph shows, the 8-core system is about half the speed of the=20 > 2-core system at 2 simultaneous threads, and it decays down to=20 > approximately 1/4 the speed of the 2-core system as the # of threads hits= 5. >=20 > All other (non-pgsql, non-sysv) tests came back approximately as=20 > expected, but I'm left wondering if I did something wrong, or if 8 cpus= =20 > are slower than 2, when it comes to Postgres on currently available FreeB= SD. I wouldn't expect 6.2 to have good scaling on 8 cpus for this kind of benchmark. Fixing that was what we have been working on for 7.0 (and basically succeeded). The most important fixes are already in 7.0 CVS, but a possible merge to 6.x will only happen after a suitably conservative delay. Kris --YiEDa0DAkWCtVeE4 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGGxlSWry0BWjoQKURAlzLAJ9Q0F2UTkn64W9LQ8GLkd+ogDySIwCfUS3F ZIagTRCYOWo6F12ypxTh7BY= =ey3P -----END PGP SIGNATURE----- --YiEDa0DAkWCtVeE4-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 10:56:38 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0C0BA16A403; Tue, 10 Apr 2007 10:56:38 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from smtp4.clear.net.nz (smtp4.clear.net.nz [203.97.37.64]) by mx1.freebsd.org (Postfix) with ESMTP id CCE9813C469; Tue, 10 Apr 2007 10:56:37 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from [192.168.1.11] (121-72-64-123.dsl.telstraclear.net [121.72.64.123]) by smtp4.clear.net.nz (CLEAR Net Mail) with ESMTP id <0JGA00HAI30OUE20@smtp4.clear.net.nz>; Tue, 10 Apr 2007 22:41:12 +1200 (NZST) Date: Tue, 10 Apr 2007 22:41:04 +1200 From: Mark Kirkwood In-reply-to: <20070226002234.GA80974@xor.obsecurity.org> To: Kris Kennaway Message-id: <461B69C0.4060707@paradise.net.nz> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit References: <20070226002234.GA80974@xor.obsecurity.org> User-Agent: Thunderbird 1.5.0.10 (X11/20070313) Cc: performance@FreeBSD.org, current@FreeBSD.org, pgsql-hackers Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 10:56:38 -0000 Kris Kennaway wrote: > If so, then your task is the following: > > Make SYSV semaphores less dumb about process wakeups. Currently > whenever the semaphore state changes, all processes sleeping on the > semaphore are woken, even if we only have released enough resources > for one waiting process to claim. i.e. there is a thundering herd > wakeup situation which destroys performance at high loads. Fixing > this will involve replacing the wakeup() calls with appropriate > amounts of wakeup_one(). I'm forwarding this to the pgsql-hackers list so that folks more qualified than I can comment, but as I understand the way postgres implements locking each process has it *own* semaphore it waits on - and who is waiting for what is controlled by an in (shared) memory hash of lock structs (access to these is controlled via platform Dependant spinlock code). So a given semaphore state change should only involve one process wakeup. Cheers Mark From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 11:49:42 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 65AB016A405; Tue, 10 Apr 2007 11:49:42 +0000 (UTC) (envelope-from mux@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5233613C44B; Tue, 10 Apr 2007 11:49:42 +0000 (UTC) (envelope-from mux@freebsd.org) Received: by elvis.mu.org (Postfix, from userid 1920) id D38D01A4D81; Tue, 10 Apr 2007 04:21:50 -0700 (PDT) Date: Tue, 10 Apr 2007 13:21:50 +0200 From: Maxime Henrion To: Mark Kirkwood Message-ID: <20070410112150.GC39474@elvis.mu.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <461B69C0.4060707@paradise.net.nz> User-Agent: Mutt/1.4.2.2i X-Mailman-Approved-At: Tue, 10 Apr 2007 16:09:29 +0000 Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Kris Kennaway Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 11:49:42 -0000 Mark Kirkwood wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if we only have released enough resources > >for one waiting process to claim. i.e. there is a thundering herd > >wakeup situation which destroys performance at high loads. Fixing > >this will involve replacing the wakeup() calls with appropriate > >amounts of wakeup_one(). > > I'm forwarding this to the pgsql-hackers list so that folks more > qualified than I can comment, but as I understand the way postgres > implements locking each process has it *own* semaphore it waits on - > and who is waiting for what is controlled by an in (shared) memory hash > of lock structs (access to these is controlled via platform Dependant > spinlock code). So a given semaphore state change should only involve > one process wakeup. Yes but there are still a lot of wakeups to be avoided in the current System V semaphore code. More specifically, not only do we wakeup all the processes waiting on a single semaphore everytime something changes, but we also wakeup all processes waiting on *any* of the semaphore in the semaphore *set*, whatever the reason we're sleeping. I came up with a quick patch so that Kris could do some testing with it, and it appears to have helped, but only very slightly; apparently some contention within the netisr code caused problems, so that in some cases the patch helped slightly, and in others it didn't. The semaphore code needs a clean rewrite and I hope to take care of this soon, as time permits, since we are heavy consumers of PostgreSQL under FreeBSD at my company. Cheers, Maxime From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 14:39:18 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E833316A402; Tue, 10 Apr 2007 14:39:18 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by mx1.freebsd.org (Postfix) with ESMTP id AA5AE13C4C2; Tue, 10 Apr 2007 14:39:18 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l3AENgaG025574; Tue, 10 Apr 2007 10:23:43 -0400 (EDT) To: Mark Kirkwood In-reply-to: <461B69C0.4060707@paradise.net.nz> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> Comments: In-reply-to Mark Kirkwood message dated "Tue, 10 Apr 2007 22:41:04 +1200" Date: Tue, 10 Apr 2007 10:23:42 -0400 Message-ID: <25573.1176215022@sss.pgh.pa.us> From: Tom Lane X-Mailman-Approved-At: Tue, 10 Apr 2007 16:42:15 +0000 Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 14:39:19 -0000 Mark Kirkwood writes: > Kris Kennaway wrote: >> If so, then your task is the following: >> >> Make SYSV semaphores less dumb about process wakeups. Currently >> whenever the semaphore state changes, all processes sleeping on the >> semaphore are woken, even if we only have released enough resources >> for one waiting process to claim. i.e. there is a thundering herd >> wakeup situation which destroys performance at high loads. Fixing >> this will involve replacing the wakeup() calls with appropriate >> amounts of wakeup_one(). > I'm forwarding this to the pgsql-hackers list so that folks more > qualified than I can comment, but as I understand the way postgres > implements locking each process has it *own* semaphore it waits on - > and who is waiting for what is controlled by an in (shared) memory hash > of lock structs (access to these is controlled via platform Dependant > spinlock code). So a given semaphore state change should only involve > one process wakeup. Correct. The behavior Kris describes is surely bad, but it's not relevant to Postgres' usage of SysV semaphores. regards, tom lane From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 18:43:32 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 49ABA16A406; Tue, 10 Apr 2007 18:43:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 1A73813C4E8; Tue, 10 Apr 2007 18:43:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 3BA471A4D86; Tue, 10 Apr 2007 11:43:12 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id BD2E8513AE; Tue, 10 Apr 2007 14:43:04 -0400 (EDT) Date: Tue, 10 Apr 2007 14:43:04 -0400 From: Kris Kennaway To: Mark Kirkwood Message-ID: <20070410184304.GB44123@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="huq684BweRXVnRxX" Content-Disposition: inline In-Reply-To: <461B69C0.4060707@paradise.net.nz> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Kris Kennaway Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 18:43:32 -0000 --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 10:41:04PM +1200, Mark Kirkwood wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if we only have released enough resources > >for one waiting process to claim. i.e. there is a thundering herd > >wakeup situation which destroys performance at high loads. Fixing > >this will involve replacing the wakeup() calls with appropriate > >amounts of wakeup_one(). >=20 > I'm forwarding this to the pgsql-hackers list so that folks more=20 > qualified than I can comment, but as I understand the way postgres=20 > implements locking each process has it *own* semaphore it waits on -=20 > and who is waiting for what is controlled by an in (shared) memory hash= =20 > of lock structs (access to these is controlled via platform Dependant=20 > spinlock code). So a given semaphore state change should only involve=20 > one process wakeup. I have not studied the exact code path, but there are indeed multiple wakeups happening from the semaphore code (as many as the number of active postgresql processes). It is easy to instrument sleepq_broadcast() and log them when they happen. Anyway mux@ fixed this some time ago, which indeed helped scaling for traffic over a local domain socket (particularly at higher loads), but I saw some anomalous results when using loopback TCP traffic. I think this is unrelated (in this situation TCP is highly contended, and it is often the case that fixing one bottleneck can make a highly contended situation perform worse, because you were effectively serializing a bit before, and reducing the non-linear behaviour) but am still investigating, so the patch has not yet been committed. Kris --huq684BweRXVnRxX Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGG9q4Wry0BWjoQKURAgj5AKD8GphymMDpkMqiJsyxu77xXZN5RACbBlbV OxZZdXcUrbW7nwz2Ac/srxo= =UDMf -----END PGP SIGNATURE----- --huq684BweRXVnRxX-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 18:43:37 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C2EEC16A407; Tue, 10 Apr 2007 18:43:37 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id AD7CC13C4C3; Tue, 10 Apr 2007 18:43:37 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id B76241A4D81; Tue, 10 Apr 2007 11:43:39 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 2FB3651B82; Tue, 10 Apr 2007 14:43:33 -0400 (EDT) Date: Tue, 10 Apr 2007 14:43:33 -0400 From: Kris Kennaway To: Tom Lane Message-ID: <20070410184332.GC44123@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <25573.1176215022@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XOIedfhf+7KOe/yw" Content-Disposition: inline In-Reply-To: <25573.1176215022@sss.pgh.pa.us> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 18:43:37 -0000 --XOIedfhf+7KOe/yw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 10:23:42AM -0400, Tom Lane wrote: > Mark Kirkwood writes: > > Kris Kennaway wrote: > >> If so, then your task is the following: > >>=20 > >> Make SYSV semaphores less dumb about process wakeups. Currently > >> whenever the semaphore state changes, all processes sleeping on the > >> semaphore are woken, even if we only have released enough resources > >> for one waiting process to claim. i.e. there is a thundering herd > >> wakeup situation which destroys performance at high loads. Fixing > >> this will involve replacing the wakeup() calls with appropriate > >> amounts of wakeup_one(). >=20 > > I'm forwarding this to the pgsql-hackers list so that folks more=20 > > qualified than I can comment, but as I understand the way postgres=20 > > implements locking each process has it *own* semaphore it waits on -= =20 > > and who is waiting for what is controlled by an in (shared) memory hash= =20 > > of lock structs (access to these is controlled via platform Dependant= =20 > > spinlock code). So a given semaphore state change should only involve= =20 > > one process wakeup. >=20 > Correct. The behavior Kris describes is surely bad, but it's not > relevant to Postgres' usage of SysV semaphores. Sorry, but the behaviour is real. Kris --XOIedfhf+7KOe/yw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGG9rUWry0BWjoQKURAmsHAKDnlEDzWjsnN6DSpJZwD9rJ4uhxkwCfT5Wk KPvE0REv7jdpbMea6D0KuYU= =l8NM -----END PGP SIGNATURE----- --XOIedfhf+7KOe/yw-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 18:46:58 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E865A16A401; Tue, 10 Apr 2007 18:46:57 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by mx1.freebsd.org (Postfix) with ESMTP id ACE5913C44C; Tue, 10 Apr 2007 18:46:57 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l3AIkurG028538; Tue, 10 Apr 2007 14:46:56 -0400 (EDT) To: Kris Kennaway In-reply-to: <20070410184332.GC44123@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <25573.1176215022@sss.pgh.pa.us> <20070410184332.GC44123@xor.obsecurity.org> Comments: In-reply-to Kris Kennaway message dated "Tue, 10 Apr 2007 14:43:33 -0400" Date: Tue, 10 Apr 2007 14:46:56 -0400 Message-ID: <28537.1176230816@sss.pgh.pa.us> From: Tom Lane X-Mailman-Approved-At: Tue, 10 Apr 2007 18:51:24 +0000 Cc: performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , pgsql-hackers Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 18:46:58 -0000 Kris Kennaway writes: >>> Make SYSV semaphores less dumb about process wakeups. Currently >>> whenever the semaphore state changes, all processes sleeping on the >>> semaphore are woken, even if we only have released enough resources >>> for one waiting process to claim. >> Correct. The behavior Kris describes is surely bad, but it's not >> relevant to Postgres' usage of SysV semaphores. > Sorry, but the behaviour is real. Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is that Postgres never has more than one process waiting on any particular SysV semaphore, and so the problem doesn't really affect us. Or do you mean that the kernel wakes all processes sleeping on *any* SysV semaphore? That would be nasty :-( regards, tom lane From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 19:45:13 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 04B5B16A47F; Tue, 10 Apr 2007 19:45:13 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5491513C44B; Tue, 10 Apr 2007 19:45:12 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 592261A4D81; Tue, 10 Apr 2007 12:45:15 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id B32155152C; Tue, 10 Apr 2007 15:45:08 -0400 (EDT) Date: Tue, 10 Apr 2007 15:45:08 -0400 From: Kris Kennaway To: Tom Lane Message-ID: <20070410194508.GA73072@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <25573.1176215022@sss.pgh.pa.us> <20070410184332.GC44123@xor.obsecurity.org> <28537.1176230816@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="FL5UXtIhxfXey3p5" Content-Disposition: inline In-Reply-To: <28537.1176230816@sss.pgh.pa.us> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 19:45:13 -0000 --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: > Kris Kennaway writes: > >>> Make SYSV semaphores less dumb about process wakeups. Currently > >>> whenever the semaphore state changes, all processes sleeping on the > >>> semaphore are woken, even if we only have released enough resources > >>> for one waiting process to claim. >=20 > >> Correct. The behavior Kris describes is surely bad, but it's not > >> relevant to Postgres' usage of SysV semaphores. >=20 > > Sorry, but the behaviour is real. >=20 > Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is > that Postgres never has more than one process waiting on any particular > SysV semaphore, and so the problem doesn't really affect us. >=20 > Or do you mean that the kernel wakes all processes sleeping on *any* > SysV semaphore? That would be nasty :-( To be clear, some behaviour that postgresql does with sysv semaphores causes wakeups of many processes at once. i.e. if you have 20 clients, you will get up to 20 wakeups. I haven't studied the precise cause of this, but it is empirically true. This is the scaling problem I described, and it's what mux's patch addresses. Kris --FL5UXtIhxfXey3p5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGG+lEWry0BWjoQKURAoFLAJ43YXRWteFzSq6h203IAD91EMuKCgCfTOFV tacaoES8XR0lq1tH7qtwUN4= =bTF6 -----END PGP SIGNATURE----- --FL5UXtIhxfXey3p5-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 20:09:45 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B10216A400; Tue, 10 Apr 2007 20:09:45 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2547B13C468; Tue, 10 Apr 2007 20:09:45 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id DA2261A4D8B; Tue, 10 Apr 2007 13:09:50 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 3DC735152C; Tue, 10 Apr 2007 16:09:44 -0400 (EDT) Date: Tue, 10 Apr 2007 16:09:44 -0400 From: Kris Kennaway To: Tom Lane Message-ID: <20070410200944.GA73534@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <25573.1176215022@sss.pgh.pa.us> <20070410184332.GC44123@xor.obsecurity.org> <28537.1176230816@sss.pgh.pa.us> <20070410194508.GA73072@xor.obsecurity.org> <555.1176234720@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HlL+5n6rz5pIUxbD" Content-Disposition: inline In-Reply-To: <555.1176234720@sss.pgh.pa.us> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 20:09:45 -0000 --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 03:52:00PM -0400, Tom Lane wrote: > Kris Kennaway writes: > > On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: > >> Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is > >> that Postgres never has more than one process waiting on any particular > >> SysV semaphore, and so the problem doesn't really affect us. >=20 > > To be clear, some behaviour that postgresql does with sysv semaphores > > causes wakeups of many processes at once. i.e. if you have 20 > > clients, you will get up to 20 wakeups. I haven't studied the precise > > cause of this, but it is empirically true. This is the scaling > > problem I described, and it's what mux's patch addresses. >=20 > [ shrug... ] To the extent that that happens, it's Postgres' own issue, > and no amount of kernel rejiggering will change it. But I certainly > have no objection to a patch that fixes the kernel behavior ... As we've discussed before, by far the bigger issue with postgresql performance on FreeBSD is the default setting of update_process_titles=3Don. Kris --HlL+5n6rz5pIUxbD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGG+8HWry0BWjoQKURAkleAKDmbOCNbFOMGymoOWUAahlAectF9wCdEiRV QF1Y1ncUaTBcw2+UCNoRYfM= =ZPuL -----END PGP SIGNATURE----- --HlL+5n6rz5pIUxbD-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 22:09:25 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4ACFA16A401; Tue, 10 Apr 2007 22:09:25 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 36E5013C465; Tue, 10 Apr 2007 22:09:25 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 1026F1A4D81; Tue, 10 Apr 2007 15:09:31 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 2BB5651566; Tue, 10 Apr 2007 18:09:24 -0400 (EDT) Date: Tue, 10 Apr 2007 18:09:24 -0400 From: Kris Kennaway To: Tom Lane Message-ID: <20070410220923.GA74088@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <20070410184304.GB44123@xor.obsecurity.org> <3721.1176240977@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+QahgC5+KEYLbs62" Content-Disposition: inline In-Reply-To: <3721.1176240977@sss.pgh.pa.us> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 22:09:25 -0000 --+QahgC5+KEYLbs62 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: > Kris Kennaway writes: > > I have not studied the exact code path, but there are indeed multiple > > wakeups happening from the semaphore code (as many as the number of > > active postgresql processes). It is easy to instrument > > sleepq_broadcast() and log them when they happen. >=20 > There are certainly cases where Postgres will wake up a number of > processes in quick succession, but that should happen from a separate > semop() kernel call, on a different semaphore, for each such process. > If there's really multiple processes being released by the same semop() > then there's a bug we need to look into (or maybe it's a kernel bug?). > Anyway I'd be interested to know what the test case is, and which PG > version you were testing. I used 8.2 (and some older version when I first noticed it a year ago) and either sysbench or supersmack will show it - presumably anything that makes simultaneous queries. Just instrument sleepq_broadcast() to e.g. log a KTR event when it wakes more than 1 process and you'll see it happening. Kris --+QahgC5+KEYLbs62 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGHAsTWry0BWjoQKURAuEBAKCWokB1FVD2u9Ldy2OqutQGbU1pOQCgsRE8 NsPcoUax8YkEbnXR7mY9SBE= =6la/ -----END PGP SIGNATURE----- --+QahgC5+KEYLbs62-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 22:28:32 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B021E16A401; Tue, 10 Apr 2007 22:28:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 9A18A13C4BA; Tue, 10 Apr 2007 22:28:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 797EB1A4D86; Tue, 10 Apr 2007 15:28:38 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id C921451933; Tue, 10 Apr 2007 18:28:31 -0400 (EDT) Date: Tue, 10 Apr 2007 18:28:31 -0400 From: Kris Kennaway To: Tom Lane Message-ID: <20070410222831.GA75767@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <20070410184304.GB44123@xor.obsecurity.org> <3721.1176240977@sss.pgh.pa.us> <20070410220923.GA74088@xor.obsecurity.org> <4307.1176243997@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <4307.1176243997@sss.pgh.pa.us> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , Kris Kennaway Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 22:28:32 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 06:26:37PM -0400, Tom Lane wrote: > Kris Kennaway writes: > > On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: > >> Anyway I'd be interested to know what the test case is, and which PG > >> version you were testing. >=20 > > I used 8.2 (and some older version when I first noticed it a year ago) > > and either sysbench or supersmack will show it - presumably anything > > that makes simultaneous queries. Just instrument sleepq_broadcast() > > to e.g. log a KTR event when it wakes more than 1 process and you'll > > see it happening. >=20 > Sorry, I'm not much of a BSD kernel hacker ... but sleepq_broadcast > seems a rather generic name. Is that called *only* from semop? It's part of how wakeup() is implemented. > I'm wondering if you are seeing simultaneous wakeup from some other > cause --- sleep timeout being the obvious possibility. We are aware > of behaviors (search the PG lists for "context swap storm") where a > number of backends will all fail to get a spinlock and do short usleep > or select-timeout waits. In this situation they'd all wake up at the > next scheduler clock tick ... Nope, it's not this. Kris --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGHA+PWry0BWjoQKURAo2KAKCc7vY5gceTDKOIVm7/jjLD6PrWVwCg6XrM fEXzN+sfe/MtkOx61CjEG9g= =RBAk -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 21:36:20 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C07E316A404; Tue, 10 Apr 2007 21:36:20 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by mx1.freebsd.org (Postfix) with ESMTP id 85A6B13C489; Tue, 10 Apr 2007 21:36:18 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l3ALaHW8003722; Tue, 10 Apr 2007 17:36:17 -0400 (EDT) To: Kris Kennaway In-reply-to: <20070410184304.GB44123@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <20070410184304.GB44123@xor.obsecurity.org> Comments: In-reply-to Kris Kennaway message dated "Tue, 10 Apr 2007 14:43:04 -0400" Date: Tue, 10 Apr 2007 17:36:17 -0400 Message-ID: <3721.1176240977@sss.pgh.pa.us> From: Tom Lane X-Mailman-Approved-At: Tue, 10 Apr 2007 22:58:10 +0000 Cc: performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , pgsql-hackers Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 21:36:20 -0000 Kris Kennaway writes: > I have not studied the exact code path, but there are indeed multiple > wakeups happening from the semaphore code (as many as the number of > active postgresql processes). It is easy to instrument > sleepq_broadcast() and log them when they happen. There are certainly cases where Postgres will wake up a number of processes in quick succession, but that should happen from a separate semop() kernel call, on a different semaphore, for each such process. If there's really multiple processes being released by the same semop() then there's a bug we need to look into (or maybe it's a kernel bug?). Anyway I'd be interested to know what the test case is, and which PG version you were testing. regards, tom lane From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 22:26:38 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E5D6316A404; Tue, 10 Apr 2007 22:26:38 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by mx1.freebsd.org (Postfix) with ESMTP id AB13113C469; Tue, 10 Apr 2007 22:26:38 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l3AMQb8B004308; Tue, 10 Apr 2007 18:26:37 -0400 (EDT) To: Kris Kennaway In-reply-to: <20070410220923.GA74088@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <20070410184304.GB44123@xor.obsecurity.org> <3721.1176240977@sss.pgh.pa.us> <20070410220923.GA74088@xor.obsecurity.org> Comments: In-reply-to Kris Kennaway message dated "Tue, 10 Apr 2007 18:09:24 -0400" Date: Tue, 10 Apr 2007 18:26:37 -0400 Message-ID: <4307.1176243997@sss.pgh.pa.us> From: Tom Lane X-Mailman-Approved-At: Tue, 10 Apr 2007 22:58:15 +0000 Cc: performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , pgsql-hackers Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 22:26:39 -0000 Kris Kennaway writes: > On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: >> Anyway I'd be interested to know what the test case is, and which PG >> version you were testing. > I used 8.2 (and some older version when I first noticed it a year ago) > and either sysbench or supersmack will show it - presumably anything > that makes simultaneous queries. Just instrument sleepq_broadcast() > to e.g. log a KTR event when it wakes more than 1 process and you'll > see it happening. Sorry, I'm not much of a BSD kernel hacker ... but sleepq_broadcast seems a rather generic name. Is that called *only* from semop? I'm wondering if you are seeing simultaneous wakeup from some other cause --- sleep timeout being the obvious possibility. We are aware of behaviors (search the PG lists for "context swap storm") where a number of backends will all fail to get a spinlock and do short usleep or select-timeout waits. In this situation they'd all wake up at the next scheduler clock tick ... regards, tom lane From owner-freebsd-performance@FreeBSD.ORG Tue Apr 10 19:52:01 2007 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 269ED16A409; Tue, 10 Apr 2007 19:52:01 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by mx1.freebsd.org (Postfix) with ESMTP id DAB5C13C484; Tue, 10 Apr 2007 19:52:00 +0000 (UTC) (envelope-from tgl@sss.pgh.pa.us) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id l3AJq02Y000556; Tue, 10 Apr 2007 15:52:00 -0400 (EDT) To: Kris Kennaway In-reply-to: <20070410194508.GA73072@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> <25573.1176215022@sss.pgh.pa.us> <20070410184332.GC44123@xor.obsecurity.org> <28537.1176230816@sss.pgh.pa.us> <20070410194508.GA73072@xor.obsecurity.org> Comments: In-reply-to Kris Kennaway message dated "Tue, 10 Apr 2007 15:45:08 -0400" Date: Tue, 10 Apr 2007 15:52:00 -0400 Message-ID: <555.1176234720@sss.pgh.pa.us> From: Tom Lane X-Mailman-Approved-At: Tue, 10 Apr 2007 22:59:53 +0000 Cc: performance@FreeBSD.org, current@FreeBSD.org, Mark Kirkwood , pgsql-hackers Subject: Re: [HACKERS] Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 19:52:01 -0000 Kris Kennaway writes: > On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: >> Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is >> that Postgres never has more than one process waiting on any particular >> SysV semaphore, and so the problem doesn't really affect us. > To be clear, some behaviour that postgresql does with sysv semaphores > causes wakeups of many processes at once. i.e. if you have 20 > clients, you will get up to 20 wakeups. I haven't studied the precise > cause of this, but it is empirically true. This is the scaling > problem I described, and it's what mux's patch addresses. [ shrug... ] To the extent that that happens, it's Postgres' own issue, and no amount of kernel rejiggering will change it. But I certainly have no objection to a patch that fixes the kernel behavior ... regards, tom lane