From owner-freebsd-amd64@FreeBSD.ORG Tue Mar 15 14:02:37 2005 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 30BB816A4CE for ; Tue, 15 Mar 2005 14:02:37 +0000 (GMT) Received: from smtp-gw-cl-d.dmv.com (smtp-gw-cl-d.dmv.com [216.240.97.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 94F5043D4C for ; Tue, 15 Mar 2005 14:02:36 +0000 (GMT) (envelope-from sven@dmv.com) Received: from lanshark.dmv.com (lanshark.dmv.com [216.240.97.46]) j2FEIxWa028308; Tue, 15 Mar 2005 09:19:01 -0500 (EST) (envelope-from sven@dmv.com) From: Sven Willenberger To: Scott Long In-Reply-To: <42366B99.4040904@samsco.org> References: <1110847561.3412.38.camel@lanshark.dmv.com> <42366B99.4040904@samsco.org> Content-Type: text/plain Date: Tue, 15 Mar 2005 09:02:33 -0500 Message-Id: <1110895353.4291.16.camel@lanshark.dmv.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.48 on 216.240.97.42 cc: freebsd-amd64@freebsd.org Subject: Re: amr performance woes and a bright side X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 14:02:37 -0000 On Mon, 2005-03-14 at 21:59 -0700, Scott Long wrote: > Sven Willenberger wrote: > > I have been testing a new box for utlimate use as a postgresql server: > > dual opteron (2.2GHz), 8G RAM, LSI 320-2x megaraid (battery-backed > > memory) with 2 single 73 GB drives and an 8x146GB RAID 0+1 array > > (hitachi U320 10k RPM). In doing so I have also tested the amd64 > > 5.3-stable release againt gentoo x86_64 and Fedora FC3/x86_64. > > > > First the bad news: > > > > The linux boxen were configured with the postgres data drives on the > > raid 0+1 using XFS with a separate pg_xlog on a different drive. Both > > gentoo and FC3 were using 2.6.x kernels using the x86_64 distro. > > pgbench was initialized using no scaling factor (1 mill rows), scaling > > 10 (10 million) and 100. > > With no scaling the linux boxen hit about 160 tps using 10 connections > > and 1000 -2000 transactions. > > The BSD system hit 100-120 tps. This is a difference I could potentially > > live with. Now enter the scaled tables: > > Linux systems hit marks of 450+ tps when pgbenching againt millions of > > rows while the BSD box stayed at 100 tps or worse .. dipping as low as > > 90 tps. > > > > Bonnie benchmarks: > > Linux: > > Sequential output: Per Char = 65000 K/sec, Block = 658782 K/sec, Rewrite > > = 639654 K/sec > > Sequential input: Per Char = 66240 K/sec, Block = 1278993 K/sec > > Sequential create: create 641/sec , read n/a, delete 205/sec > > Random create: create 735/sec, read n/a, delete 126/sec > > > > BSD: > > Sequential output: Per Char = 370K/sec (!!), block = 132281 K/sec, > > Rewrite = 124070 K/sec > > Sequential input: Per Char = 756 K/sec, block = 700402 K/sec > > Sequential create: create 139/sec, read 6308/seec, delete n/a > > Random create: create 137/sec, read 5877/sec, delete n/a > > > > the bonnie tests were run several times with similar results. > > > > It would seem to me that the pgbench marks and tests are being hampered > > by comparatively poor I/O to the raid array and disks under the amr > > driver control. I am hoping there are some tweaks that I could do or > > perhaps some patches to the driver in -CURRENT that could be > > applied/backported/MFC'ed to try and improve this performance. > > > > Oh, the "bright" side? FreeBSD is the only OS here that didn't kernel > > Oops due to memory allocation issues, or whatever caused them (the > > backtrace showed kmalloc). That may be because of the XFS file system (I > > didn't try EXT3 or its kin) or because of issues with LSI and the linux > > kernel or who knows what. I am hoping to get the stability and OS > > performance of FreeBSD and the raw disk performance witnessed in the > > Linux systems all rolled up into one. Help? > > > > Sven > > > > First of all, are you using the same hardware and just switching the OS? > Are you sure that the RAID and disk cache settings are identical? > Second, some of the Linux numbers are very hard to believe; PCI-X has a > theoretical bandwidth of 1066MB/sec, so it's highly unlikely that you're > going to get 1249MB/sec out of it in the block read test. bonnie is an > excellent tool for testing the randomness of cache effects and memory > bandwidth, it's not so good at testing actual I/O performance =-) > > So setting aside the bonnie tests, the PQSQL stats do indeed show a > problem. Is PGSQL threaded? If so, you might be running into some of > the threading performance problems that are well known and are being > worked on. I don't know a whole lot about PGSQL or the tests that you > are talking about, but if you had an easy recipe for duplicating your > test environment, I'd like to experiment some myself. > > Scott Yes, these tests were done on the same hardware, with the same hardware raid configuration with fresh OS installs for each battery of tests. The bonnie numbers do seem a bit out of whack upon closer scrutiny. As far as setting up PGSQL, in each case it was set up from packages (FreeBSD ports, Gentoo emerge, FC3 yum) and the postgresql.conf file was adjusted to use the same set of values for memory, etc. pgbench (postgresql-contrib) was run as follows for testing: pgbench -i -U postgres/pgsql pgtest (where the User is either postgres or pgsql depending on platform and pgtest is the test db set up using createdb) pgbench -c 10 -t 1000 -U pgsql pgtest pgbench -c 10 -t 4000 -U pgsql pgtest pgbench -c 10 -t 10000 -U pgsql pgtest pgbench -i -s 10 -U pgsql pgtest (scaling factor of 10 to increase the table sizes for benchmarking) pgbench -c 10 -t 1000 etc ...... pgbench -i -s 100 -U pgsql pgtest Sven