Date: Tue, 17 Apr 2007 19:17:21 -0400 From: Kevin Way <kevin@insidesystems.net> To: Kris Kennaway <kris@obsecurity.org> Cc: freebsd-performance@freebsd.org, Robert Watson <rwatson@FreeBSD.org> Subject: Re: Anyone interested in improving postgresql scaling? Message-ID: <46255581.8000902@insidesystems.net> In-Reply-To: <20070417192120.GA5191@xor.obsecurity.org> References: <461B0CD0.8090404@insidesystems.net> <20070417174825.X42234@fledge.watson.org> <4625174B.5090208@insidesystems.net> <20070417192120.GA5191@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms090308060806080007010801 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Kris Kennaway wrote: > On Tue, Apr 17, 2007 at 02:51:55PM -0400, Kevin Way wrote: > > >> I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE >> and 4BSD. The results are available at: >> >> http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 >> >> Unfortunately, I can't run the additional tests, as the machines in >> question have now been deployed to production. I should have similar >> equipment available in a few weeks, and will try the other tests at that >> point. >> > > That's better but still not comparable to my results. One important > change you may have missed was to enable MALLOC_PRODUCTION in > /usr/src/lib/libc/stdlib/malloc.c. This disables malloc debugging > which may have significant overhead. > > My results from about a month ago: > > http://people.freebsd.org/~kris/scaling/nickel.png > > The big dip between 5-7 clients is believed to be due to a ULE bug > that has known cause but is not yet fixed (on dual core CPUs processes > are bouncing around between the two cores too much). The main point > is that apart from this anomaly in the middle there is approximately > linear scaling from 1 to 8 CPUs. On your system it is not scaling > linearly beyond 4. > > I do have some other changes in my kernel but I don't think any of > them are enough to explain this difference. > > Anyway, get back to us when you can retest with the above change. > > Confirmed, I did miss MALLOC_PRODUCTION. I incorrectly believed that I just needed an empty malloc.conf. This is somewhat exciting news from my vantage point. I'll post a corrected re-test as soon as I have another available 8-core machine. --------------ms090308060806080007010801 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMljCC A2IwggLLoAMCAQICEAvaCxfBP4mOqwl0erTOLjMwDQYJKoZIhvcNAQECBQAwXzELMAkGA1UE BhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAxIFB1Ymxp YyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk4MDUxMjAwMDAwMFoXDTA4 MDUxMjIzNTk1OVowgcwxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJp U2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRv cnkvUlBBIEluY29ycC4gQnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2ln biBDbGFzcyAxIENBIEluZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0 ZWQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBALtaRIoEFrtV/QN6ii2UTxV4NrgNSrJv nFS/vOh3Kp258Gi7ldkxQXB6gUu5SBNWLccI4YRCq8CikqtEXKpC8IIOAukv+8I7u77JJwpd trA2QjO1blSIT4dKvxna+RXoD4e2HOPMxpqOf2okkuP84GW6p7F+78nbN2rISsgJBuSZAgMB AAGjgbAwga0wDwYDVR0TBAgwBgEB/wIBADBHBgNVHSAEQDA+MDwGC2CGSAGG+EUBBwEBMC0w KwYIKwYBBQUHAgEWH3d3dy52ZXJpc2lnbi5jb20vcmVwb3NpdG9yeS9SUEEwMQYDVR0fBCow KDAmoCSgIoYgaHR0cDovL2NybC52ZXJpc2lnbi5jb20vcGNhMS5jcmwwCwYDVR0PBAQDAgEG MBEGCWCGSAGG+EIBAQQEAwIBBjANBgkqhkiG9w0BAQIFAAOBgQACfZ5vRUs4oLje6VNkIbzk TCuPHv6SQKzYCjlqoTIhLAebq1n+0mIafVU4sDdz3PQHZmNiveFTcFKH56jYUulbLarh3s+s MVTUixnI2COo7wQrMn0sGBzIfImoLnfyRNFlCk10te7TG5JzdC6JOzUTcudAMZrTssSr51a+ i+P7FTCCBJQwggP9oAMCAQICEH//rl4NoatSoVbKWNMj7LQwDQYJKoZIhvcNAQEFBQAwgcwx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3 b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29ycC4g QnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENBIElu ZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQwHhcNMDcwMzA4MDAw MDAwWhcNMDcwNTA3MjM1OTU5WjCCAQYxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29t L3JlcG9zaXRvcnkvUlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4wHAYDVQQL ExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxJjAkBgNVBAsTHURpZ2l0YWwgSUQgQ2xhc3MgMSAt IE5ldHNjYXBlMRIwEAYDVQQDFAlLZXZpbiBXYXkxJjAkBgkqhkiG9w0BCQEWF2tldmluQGlu c2lkZXN5c3RlbXMubmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArwYwJ9u1 sjZXgnLX1jzAsrIw13m/UOl3OyFkNdnZEo3CH2IiequADpO3FaLLJe7AAIlnP/VDGxv+QHQ1 +eOEu+8jI2xSdTjTMBIxF5fugzEhy1fdQ+QBta85x76+bO4e5XAJUNr8Ao5H8FpCvYlM26hO 5NkUTXdfI7JPszS6AP/e0E6Xrj3zsbZA8HmhOhidH9fG6TNwTPmOG/ji9MvX55OdJcJCIazq cA1kGsgHyX9t62Wn+3NoZBGpFOx+3NVDQglain6VnAHtKwJqcCnRZHtLOO8ofUlZeOR/9beg inkyyKf3F0tyfT8yeprdZ0NreJti9MmhuCj6HmrmO0uGWQIDAQABo4G1MIGyMAkGA1UdEwQC MAAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXAzAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3 dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDBAYI KwYBBQUHAwIwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC52ZXJpc2lnbi5jb20vY2xh c3MxLmNybDANBgkqhkiG9w0BAQUFAAOBgQCJJJAQdU1zobfScAJaaJ76DvJrbJmloOItqhri dCE2396gPTIY9NQjRI3kvbuh1Qzz3voPFeKOlnNXiAJsrwvFCgwFXCxzXZF0J0Wn/Ci6O3YZ BoGIBBgfrt3d5gPkCMTcTDoKqFFLS7fu6eOab8oiGUvKYcXlMxqU8WK4Y3JS2DCCBJQwggP9 oAMCAQICEH//rl4NoatSoVbKWNMj7LQwDQYJKoZIhvcNAQEFBQAwgcwxFzAVBgNVBAoTDlZl cmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQL Ez13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29ycC4gQnkgUmVmLixMSUFC LkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENBIEluZGl2aWR1YWwgU3Vi c2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQwHhcNMDcwMzA4MDAwMDAwWhcNMDcwNTA3 MjM1OTU5WjCCAQYxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2ln biBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkv UlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4wHAYDVQQLExVQZXJzb25hIE5v dCBWYWxpZGF0ZWQxJjAkBgNVBAsTHURpZ2l0YWwgSUQgQ2xhc3MgMSAtIE5ldHNjYXBlMRIw EAYDVQQDFAlLZXZpbiBXYXkxJjAkBgkqhkiG9w0BCQEWF2tldmluQGluc2lkZXN5c3RlbXMu bmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArwYwJ9u1sjZXgnLX1jzAsrIw 13m/UOl3OyFkNdnZEo3CH2IiequADpO3FaLLJe7AAIlnP/VDGxv+QHQ1+eOEu+8jI2xSdTjT MBIxF5fugzEhy1fdQ+QBta85x76+bO4e5XAJUNr8Ao5H8FpCvYlM26hO5NkUTXdfI7JPszS6 AP/e0E6Xrj3zsbZA8HmhOhidH9fG6TNwTPmOG/ji9MvX55OdJcJCIazqcA1kGsgHyX9t62Wn +3NoZBGpFOx+3NVDQglain6VnAHtKwJqcCnRZHtLOO8ofUlZeOR/9beginkyyKf3F0tyfT8y eprdZ0NreJti9MmhuCj6HmrmO0uGWQIDAQABo4G1MIGyMAkGA1UdEwQCMAAwRAYDVR0gBD0w OzA5BgtghkgBhvhFAQcXAzAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5j b20vcnBhMAsGA1UdDwQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwMwYD VR0fBCwwKjAooCagJIYiaHR0cDovL2NybC52ZXJpc2lnbi5jb20vY2xhc3MxLmNybDANBgkq hkiG9w0BAQUFAAOBgQCJJJAQdU1zobfScAJaaJ76DvJrbJmloOItqhridCE2396gPTIY9NQj RI3kvbuh1Qzz3voPFeKOlnNXiAJsrwvFCgwFXCxzXZF0J0Wn/Ci6O3YZBoGIBBgfrt3d5gPk CMTcTDoKqFFLS7fu6eOab8oiGUvKYcXlMxqU8WK4Y3JS2DGCBKowggSmAgEBMIHhMIHMMRcw FQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azFGMEQGA1UECxM9d3d3LnZlcmlzaWduLmNvbS9yZXBvc2l0b3J5L1JQQSBJbmNvcnAuIEJ5 IFJlZi4sTElBQi5MVEQoYyk5ODFIMEYGA1UEAxM/VmVyaVNpZ24gQ2xhc3MgMSBDQSBJbmRp dmlkdWFsIFN1YnNjcmliZXItUGVyc29uYSBOb3QgVmFsaWRhdGVkAhB//65eDaGrUqFWyljT I+y0MAkGBSsOAwIaBQCgggKdMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcN AQkFMQ8XDTA3MDQxNzIzMTcyMVowIwYJKoZIhvcNAQkEMRYEFPhvjMHAXYJwwXzYFx4Ig+5I wtU5MFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqG SIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIHyBgkrBgEEAYI3EAQxgeQwgeEw gcwxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBO ZXR3b3JrMUYwRAYDVQQLEz13d3cudmVyaXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29y cC4gQnkgUmVmLixMSUFCLkxURChjKTk4MUgwRgYDVQQDEz9WZXJpU2lnbiBDbGFzcyAxIENB IEluZGl2aWR1YWwgU3Vic2NyaWJlci1QZXJzb25hIE5vdCBWYWxpZGF0ZWQCEH//rl4NoatS oVbKWNMj7LQwgfQGCyqGSIb3DQEJEAILMYHkoIHhMIHMMRcwFQYDVQQKEw5WZXJpU2lnbiwg SW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29yazFGMEQGA1UECxM9d3d3LnZl cmlzaWduLmNvbS9yZXBvc2l0b3J5L1JQQSBJbmNvcnAuIEJ5IFJlZi4sTElBQi5MVEQoYyk5 ODFIMEYGA1UEAxM/VmVyaVNpZ24gQ2xhc3MgMSBDQSBJbmRpdmlkdWFsIFN1YnNjcmliZXIt UGVyc29uYSBOb3QgVmFsaWRhdGVkAhB//65eDaGrUqFWyljTI+y0MA0GCSqGSIb3DQEBAQUA BIIBAChHryWsSWWr++Zjz8hDeCM88oyI6XXuSoka3XKSj+78QJOIWg5ecc/jum+ZZIMYs6Q+ Ap0OknKOONVEi5Jb8QimJe+TKtQgqGKejT1YhI7bG4HFo5UWFZJnb/w5pzp1rN7lkOQeL8AQ t/iXQJWcqgbpGeXbeptRWA2SsSiT0LBM59WVTmeNWqz6A9HnxcMsqUwTdEmoAW1MOtxEJJFG YtXqXzhi6vbnf9RKR4+IZ9un1NEELBJvIH2QpyI7quyLV3gpC5cRtVbjXb0W5ZX+LkZseXVI V2Icqj1GeWVuo8DBBz2efZrsl8nj7ci0Y0hvoCKAjhOaPBIRwC7ixMnIJKMAAAAAAAA= --------------ms090308060806080007010801--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46255581.8000902>