From owner-freebsd-performance@FreeBSD.ORG Mon Oct 2 14:28:08 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DBE2716A416 for ; Mon, 2 Oct 2006 14:28:08 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4219B43D5D for ; Mon, 2 Oct 2006 14:28:07 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 2 Oct 2006 10:28:07 -0400 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 2 Oct 06 14:28:07 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id TYVCAW95; Mon, 2 Oct 2006 10:28:06 -0400 Date: Mon, 2 Oct 2006 10:28:06 -0400 From: Nick Evans To: Mike Jakubik Message-ID: <20061002102806.7addbcdf@pleiades.nextvenue.com> In-Reply-To: <451D4F07.7020108@rogers.com> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2006 14:28:07.0143 (UTC) FILETIME=[FA740770:01C6E62E] Cc: performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2006 14:28:08 -0000 On Fri, 29 Sep 2006 12:51:19 -0400 Mike Jakubik wrote: > Jan Zacharias wrote: > > Btw. the same problem exists on MAC OSX, as OSX also uses a BSD Kernel > > the problem might be, that samba was optimized for the linux kernel > > tcp stack. > > Quite possibly, just as MySQL has been written with Linux primarily in mind. It should be possible to profile smbd and see where the bottleneck is, no? Has anyone tried it? From owner-freebsd-performance@FreeBSD.ORG Mon Oct 2 14:30:28 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C720F16A40F for ; Mon, 2 Oct 2006 14:30:28 +0000 (UTC) (envelope-from mikej@rogers.com) Received: from smtp106.rog.mail.re2.yahoo.com (smtp106.rog.mail.re2.yahoo.com [68.142.225.204]) by mx1.FreeBSD.org (Postfix) with SMTP id 2ADB143D5D for ; Mon, 2 Oct 2006 14:30:27 +0000 (GMT) (envelope-from mikej@rogers.com) Received: (qmail 41319 invoked from network); 2 Oct 2006 14:30:23 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Gq0knFV3B8TNiOf+4nbRa7qawx6sutk18twm7Ycn4U2/fisSTWi/DgvbOVQPrThw3GHHroe3UxzWv3T8CgQWnIrxjyzQyej7pYOn8/7UHcH4jFUSQWJKyAMbKgyoehgS1NCUs00rh5y6BTayR6cHXaxXY8v99zAazSqdfPieefA= ; Received: from unknown (HELO ?127.0.0.1?) (mikej@rogers.com@74.111.253.239 with plain) by smtp106.rog.mail.re2.yahoo.com with SMTP; 2 Oct 2006 14:30:23 -0000 Message-ID: <4521227F.6040801@rogers.com> Date: Mon, 02 Oct 2006 10:30:23 -0400 From: Mike Jakubik User-Agent: Thunderbird 1.5.0.7 (Windows/20060909) MIME-Version: 1.0 To: Nick Evans References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> <20061002102806.7addbcdf@pleiades.nextvenue.com> In-Reply-To: <20061002102806.7addbcdf@pleiades.nextvenue.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2006 14:30:28 -0000 Nick Evans wrote: > Mike Jakubik wrote: > >> Quite possibly, just as MySQL has been written with Linux primarily in mind. >> > > It should be possible to profile smbd and see where the bottleneck is, no? > Has anyone tried it? > It should be, but i don't believe anyone has tried yet. From owner-freebsd-performance@FreeBSD.ORG Mon Oct 2 16:42:43 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28DF416A407 for ; Mon, 2 Oct 2006 16:42:43 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A9ED43D46 for ; Mon, 2 Oct 2006 16:42:42 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 2 Oct 2006 12:42:41 -0400 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 2 Oct 06 16:42:41 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id TYVCAXR1; Mon, 2 Oct 2006 12:42:41 -0400 Date: Mon, 2 Oct 2006 12:42:41 -0400 From: Nick Evans To: Mike Jakubik Message-ID: <20061002124241.64891173@pleiades.nextvenue.com> In-Reply-To: <4521227F.6040801@rogers.com> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> <20061002102806.7addbcdf@pleiades.nextvenue.com> <4521227F.6040801@rogers.com> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2006 16:42:41.0565 (UTC) FILETIME=[C72F0CD0:01C6E641] Cc: performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2006 16:42:43 -0000 On Mon, 02 Oct 2006 10:30:23 -0400 Mike Jakubik wrote: > Nick Evans wrote: > > Mike Jakubik wrote: > > > >> Quite possibly, just as MySQL has been written with Linux primarily in > >> mind. > > > > It should be possible to profile smbd and see where the bottleneck is, no? > > Has anyone tried it? > > > > > It should be, but i don't believe anyone has tried yet. > Does anyone know how to get gprof to set the gmon.out file with a pid number in the filename? It's kind of hard to get output of the pid that controls a file transfer without it. Everything I've found via Google is not working so far. From owner-freebsd-performance@FreeBSD.ORG Tue Oct 3 06:19:56 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F32E216A403 for ; Tue, 3 Oct 2006 06:19:55 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.176.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 510D643D45 for ; Tue, 3 Oct 2006 06:19:54 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.8/8.13.7) with ESMTP id k936Jp2o065310 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 3 Oct 2006 08:19:51 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.8/8.13.3/Submit) id k936JnQ8065308; Tue, 3 Oct 2006 08:19:49 +0200 (CEST) Date: Tue, 3 Oct 2006 08:19:49 +0200 From: Divacky Roman To: Nick Evans Message-ID: <20061003061949.GA65231@stud.fit.vutbr.cz> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> <20061002102806.7addbcdf@pleiades.nextvenue.com> <4521227F.6040801@rogers.com> <20061002124241.64891173@pleiades.nextvenue.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061002124241.64891173@pleiades.nextvenue.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.57 on 147.229.176.14 Cc: Mike Jakubik , performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Oct 2006 06:19:56 -0000 On Mon, Oct 02, 2006 at 12:42:41PM -0400, Nick Evans wrote: > On Mon, 02 Oct 2006 10:30:23 -0400 > Mike Jakubik wrote: > > > Nick Evans wrote: > > > Mike Jakubik wrote: > > > > > >> Quite possibly, just as MySQL has been written with Linux primarily in > > >> mind. > > > > > > It should be possible to profile smbd and see where the bottleneck is, no? > > > Has anyone tried it? > > > > > > > > > It should be, but i don't believe anyone has tried yet. > > > > Does anyone know how to get gprof to set the gmon.out file with a pid number > in the filename? It's kind of hard to get output of the pid that controls a > file transfer without it. Everything I've found via Google is not working so > far. I'd start with plain ktrace and look in the trace to check for 'suspicious things'. It might be something trivial like calling gettimeofday() way too often as it was with mysql.... From owner-freebsd-performance@FreeBSD.ORG Tue Oct 3 15:27:54 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB95916A416 for ; Tue, 3 Oct 2006 15:27:54 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 212FE43D6E for ; Tue, 3 Oct 2006 15:27:47 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 3 Oct 2006 11:27:46 -0400 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 3 Oct 06 15:27:46 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id TYVCA6NB; Tue, 3 Oct 2006 11:27:46 -0400 Date: Tue, 3 Oct 2006 11:27:46 -0400 From: Nick Evans To: Divacky Roman Message-ID: <20061003112746.58ce03b3@pleiades.nextvenue.com> In-Reply-To: <20061003061949.GA65231@stud.fit.vutbr.cz> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> <20061002102806.7addbcdf@pleiades.nextvenue.com> <4521227F.6040801@rogers.com> <20061002124241.64891173@pleiades.nextvenue.com> <20061003061949.GA65231@stud.fit.vutbr.cz> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 03 Oct 2006 15:27:46.0667 (UTC) FILETIME=[7A6DCBB0:01C6E700] Cc: Mike Jakubik , performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Oct 2006 15:27:55 -0000 On Tue, 3 Oct 2006 08:19:49 +0200 Divacky Roman wrote: > On Mon, Oct 02, 2006 at 12:42:41PM -0400, Nick Evans wrote: > > On Mon, 02 Oct 2006 10:30:23 -0400 > > Mike Jakubik wrote: > > > > > Nick Evans wrote: > > > > Mike Jakubik wrote: > > > > > > > >> Quite possibly, just as MySQL has been written with Linux primarily > > > >> in mind. > > > > > > > > It should be possible to profile smbd and see where the bottleneck > > > > is, no? Has anyone tried it? > > > > > > > > > > > > > It should be, but i don't believe anyone has tried yet. > > > > > > > Does anyone know how to get gprof to set the gmon.out file with a pid > > number in the filename? It's kind of hard to get output of the pid that > > controls a file transfer without it. Everything I've found via Google is > > not working so far. > > I'd start with plain ktrace and look in the trace to check for 'suspicious > things'. It might be something trivial like calling gettimeofday() way too > often as it was with mysql.... Done. I can provide the tracefile if anyone is interested. From owner-freebsd-performance@FreeBSD.ORG Tue Oct 3 15:36:37 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BB4016A40F for ; Tue, 3 Oct 2006 15:36:37 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id 501D543D45 for ; Tue, 3 Oct 2006 15:36:36 +0000 (GMT) (envelope-from wmoran@collaborativefusion.com) Received: from collaborativefusion.com (mx01.pub.collaborativefusion.com [206.210.89.201]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Tue, 03 Oct 2006 11:36:35 -0400 id 0005642B.45228383.0001739B Received: from Internal Mail-Server (206.210.89.202) by mx01 (envelope-from wmoran@collaborativefusion.com) with AES256-SHA encrypted SMTP; 3 Oct 2006 11:33:43 -0400 Date: Tue, 3 Oct 2006 11:36:35 -0400 From: Bill Moran To: Nick Evans Message-Id: <20061003113635.a0c76714.wmoran@collaborativefusion.com> In-Reply-To: <20061003112746.58ce03b3@pleiades.nextvenue.com> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <451D4630.7040902@rogers.com> <451D4F07.7020108@rogers.com> <20061002102806.7addbcdf@pleiades.nextvenue.com> <4521227F.6040801@rogers.com> <20061002124241.64891173@pleiades.nextvenue.com> <20061003061949.GA65231@stud.fit.vutbr.cz> <20061003112746.58ce03b3@pleiades.nextvenue.com> Organization: Collaborative Fusion X-Mailer: Sylpheed version 2.2.7 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Mike Jakubik , Divacky Roman , performance@freebsd.org, Jan Zacharias Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Oct 2006 15:36:37 -0000 In response to Nick Evans : > > I'd start with plain ktrace and look in the trace to check for 'suspicious > > things'. It might be something trivial like calling gettimeofday() way too > > often as it was with mysql.... > > Done. I can provide the tracefile if anyone is interested. Please post this somewhere and provide a URL. I'm no kernel expert, but I've had success speeding stuff up through ktraces before ... -- Bill Moran Collaborative Fusion Inc. From owner-freebsd-performance@FreeBSD.ORG Wed Oct 4 02:14:13 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 87EC316A40F for ; Wed, 4 Oct 2006 02:14:13 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1878B43D83 for ; Wed, 4 Oct 2006 02:14:03 +0000 (GMT) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by koef.zs64.net (8.13.8/8.13.8) with ESMTP id k942E2BM070192; Wed, 4 Oct 2006 04:14:02 +0200 (CEST) (envelope-from cracauer@koef.zs64.net) Received: (from cracauer@localhost) by koef.zs64.net (8.13.8/8.13.8/Submit) id k942E1oE070191; Tue, 3 Oct 2006 22:14:01 -0400 (EDT) (envelope-from cracauer) Date: Tue, 3 Oct 2006 22:14:01 -0400 From: Martin Cracauer To: Jan Zacharias Message-ID: <20061004021401.GA69704@cons.org> References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.2i Cc: Performance/tuning Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Oct 2006 02:14:13 -0000 Jan Zacharias wrote on Fri, Sep 29, 2006 at 05:28:27PM +0200: > Hi Olivier, > > generally the Samba performance on *BSD just sucks, > I wrote to this list with the same problem some weeks ago, > no one was able to tune smbd so that it would outperform > as well as under linux. I haven't seen previous discussions about this, but in general you have a problem on FreeBSD when processes takes up some good amount of CPU but there is currently performance monitoring being does for network traffic, outbound traffic in particular. When you compare FreeBSD's and Linux' behaviour while CPU eating processes and network traffic compete for the machine, you see that FreeBSD has the network traffic slow down a whole lot, while Linux cuts resources off the CPU consuming processes (this is contrary to what people normally assume, which is that FreeBSD is the ninja macho networker's tool and Linux is more for "desktop" performance, it's the other way round). While there are different situations where one or the other can be better, the situation of serving out SMB via Samba maximizes a situation where FreeBSD looks bad. The Samba processes take up quite a bit of CPU and apparently that goes directly to the latency or throughput of the output. If suspect that packets are written into the network buffers and then new CPU demands from the Samba process clobber the timely delivery. Linux does the opposite, once the packets are available for delivery it rushes out the packages as soon as it can and the CPU demanding process is left starving while doing so (in this Samba case that is what you want, in other cases it might ruin your computation performance). If my assumption is correct, then ktracing or profiling won't do much to understand the situation. A complex analysis on the client side about latency and throughput after specific SMB calls from the server would have to be done both on FreeBSD and Linux to find out where the differences are, exactly. It's probably more promising to directly mess with the scheduler and randomly mess with it while observing what happens to your client's performance. ==> However, you should try the ULE scheduler, instead of the 4BSD scheduler. If you do, please report your findings back. Also make sure that you don't use a giant-locked driver somewhere in the chain to the filessystem and that WITNESS and friends are off. Is this a SMP machine? Martin BTW, the situation is entirely different from the mysql situation where we damage performance by using "the wrong" thread library by default. Samba is not threaded and mysql performance is only to a minor part network-bound. -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/ From owner-freebsd-performance@FreeBSD.ORG Wed Oct 4 03:18:27 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E288E16A412 for ; Wed, 4 Oct 2006 03:18:27 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: from qb-out-0506.google.com (qb-out-0506.google.com [72.14.204.231]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6DE0A43D58 for ; Wed, 4 Oct 2006 03:18:27 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: by qb-out-0506.google.com with SMTP id a10so833027qbd for ; Tue, 03 Oct 2006 20:18:26 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=RqJWAf2qYZJUm1KxvaD8/+RBCuyXZC/78kmhdZuMiMYgODugdolv02DvcWrpvd2TJAzzFuLT7mUUQy67DbB2k8sOWyM2a10S3lHH48rWxUgzdBrsGmMdGoOd3f3JaaWdR9u7oWG4JOf8Y2U3YnMRVAOAE4N0qUB6+PIEQOuAGi4= Received: by 10.65.176.7 with SMTP id d7mr192880qbp; Tue, 03 Oct 2006 20:11:26 -0700 (PDT) Received: by 10.64.179.8 with HTTP; Tue, 3 Oct 2006 20:11:25 -0700 (PDT) Message-ID: <84dead720610032011s716955f8q5351e1ca2a6c02f2@mail.gmail.com> Date: Wed, 4 Oct 2006 08:41:25 +0530 From: "Joseph Koshy" To: "Martin Cracauer" In-Reply-To: <20061004021401.GA69704@cons.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3131aa530609290721o267d55bakff4e801ef4000675@mail.gmail.com> <20061004021401.GA69704@cons.org> Cc: Jan Zacharias , Performance/tuning Subject: Re: Samba Performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Oct 2006 03:18:28 -0000 mc> Also make sure that you don't use a giant-locked driver mc> somewhere in the chain to the filessystem and that WITNESS mc> and friends are off. Is this a SMP machine? As a general suggestion, you can use hwpmc(4) and pmcstat(8) to understand where the bottlenecks are. For example, you can capture a system-wide profile and convert it to a set of gmon.out files, one per executable object (kernel, shared libraries, executables) in the system. -- FreeBSD Volunteer, http://people.freebsd.org/~jkoshy From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 16:30:32 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B245516A415 for ; Thu, 5 Oct 2006 16:30:32 +0000 (UTC) (envelope-from jbell@stelesys.com) Received: from stelesys.com (web3.stelesys.com [63.175.100.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D86A43D67 for ; Thu, 5 Oct 2006 16:30:04 +0000 (GMT) (envelope-from jbell@stelesys.com) Received: from [127.0.0.1] (helo=www.stelesys.com) by stelesys.com with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GVHTn-0003OX-CU for freebsd-performance@freebsd.org; Wed, 04 Oct 2006 20:52:51 -0400 Received: from 71.56.92.181 (SquirrelMail authenticated user jbell@stelesys.com) by www.stelesys.com with HTTP; Wed, 4 Oct 2006 20:52:51 -0400 (EDT) Message-ID: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> Date: Wed, 4 Oct 2006 20:52:51 -0400 (EDT) From: "Jerry Bell" To: freebsd-performance@freebsd.org User-Agent: SquirrelMail/1.4.6-rc1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 16:30:32 -0000 I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. I've looked through some of the lists here and have seen super-smack results in the 42k qps range on a 2 dual core opteron system. I'm able to get up to about 34k with the wide at the back of my server whilest rubbing the side of it. Here's what I've done: built both mysql 5.0 and 5.1 from ports with build_static and optimazations on. changed the clock to TSC added the following to my /etc/libmap.conf file: [mysqld] libc_r.so.5 libthr.so.2 libc_r.so.6 libthr.so.2 libthr.so.2 libthr.so.2 libpthread.so.1 libthr.so.2 libpthread.so.2 libthr.so.2 And this is about the best result I get: 2950# super-smack -d mysql select-key-mysql.smack 10 10000 Query Barrel Report for client smacker1 connect: max=200ms min=0ms avg= 20ms from 10 clients Query_type num_queries max_time min_time q_per_s select_index 200000 0 0 34440.88 Are there any other optimizations I need to do to mysql or freebsd? I'm still not 100% convinced that I've done everything I need to use libthr instead of libpthread. I expected the 2950 to be a bit closer to the 1.8Ghz opteron discussed here: http://article.gmane.org/gmane.os.freebsd.performance/1137/match=mysql Realistically, there's probably a bit of a performance advantage in the opteron, but I wouldn't expect it to be that large. Thanks! Jerry From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 16:56:36 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2733016A403 for ; Thu, 5 Oct 2006 16:56:36 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 119C743D78 for ; Thu, 5 Oct 2006 16:56:34 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 5 Oct 2006 12:56:33 -0400 Received: from 204.141.15.136 ([204.141.15.136] helo=postal.talkpoint.com) by ASSP-nospam ; 5 Oct 06 16:56:33 -0000 Received: from pleiades.nextvenue.com ([204.141.15.194]) by postal.talkpoint.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id TYVCBHVD; Thu, 5 Oct 2006 12:56:33 -0400 Date: Thu, 5 Oct 2006 12:56:33 -0400 From: Nick Evans To: "Jerry Bell" Message-ID: <20061005125633.03f93656@pleiades.nextvenue.com> In-Reply-To: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> X-Mailer: Sylpheed-Claws 2.4.0 (GTK+ 2.8.20; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Oct 2006 16:56:34.0094 (UTC) FILETIME=[36A624E0:01C6E89F] Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 16:56:36 -0000 On Wed, 4 Oct 2006 20:52:51 -0400 (EDT) "Jerry Bell" wrote: > I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. > I've looked through some of the lists here and have seen super-smack > results in the 42k qps range on a 2 dual core opteron system. I'm able to > get up to about 34k with the wide at the back of my server whilest rubbing > the side of it. > > Here's what I've done: > built both mysql 5.0 and 5.1 from ports with build_static and > optimazations on. > changed the clock to TSC > > added the following to my /etc/libmap.conf file: > [mysqld] > libc_r.so.5 libthr.so.2 > libc_r.so.6 libthr.so.2 > libthr.so.2 libthr.so.2 > libpthread.so.1 libthr.so.2 > libpthread.so.2 libthr.so.2 > > And this is about the best result I get: > 2950# super-smack -d mysql select-key-mysql.smack 10 10000 > Query Barrel Report for client smacker1 > connect: max=200ms min=0ms avg= 20ms from 10 clients > Query_type num_queries max_time min_time q_per_s > select_index 200000 0 0 34440.88 > > > Are there any other optimizations I need to do to mysql or freebsd? I'm > still not 100% convinced that I've done everything I need to use libthr > instead of libpthread. > > I expected the 2950 to be a bit closer to the 1.8Ghz opteron discussed > here: > http://article.gmane.org/gmane.os.freebsd.performance/1137/match=mysql > Realistically, there's probably a bit of a performance advantage in the > opteron, but I wouldn't expect it to be that large. > > Thanks! > > Jerry > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" If you build static you might not be using libthr and libmap AFAIK doesn't work with static binaries. Build it with only the extra optimizations and try again. You can check what threading lib it's using with ldd /usr/local/libexec/mysqld Nick From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 17:29:29 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D834B16A412 for ; Thu, 5 Oct 2006 17:29:29 +0000 (UTC) (envelope-from freebsd@lidstrom.eu) Received: from pne-smtpout1-sn2.hy.skanova.net (pne-smtpout1-sn2.hy.skanova.net [81.228.8.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id E6EA043DC7 for ; Thu, 5 Oct 2006 17:28:07 +0000 (GMT) (envelope-from freebsd@lidstrom.eu) Received: from mailgw.henriklidstrom.se (83.250.106.220) by pne-smtpout1-sn2.hy.skanova.net (7.2.075) id 4516FC41002F58EC; Thu, 5 Oct 2006 19:28:04 +0200 Received: from localhost (localhost.henriklidstrom.se [127.0.0.1]) by mailgw.henriklidstrom.se (Postfix) with ESMTP id 71DA1B831; Thu, 5 Oct 2006 19:27:57 +0200 (CEST) Received: from mailgw.henriklidstrom.se ([127.0.0.1]) by localhost (mailgw.henriklidstrom.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 34385-07; Thu, 5 Oct 2006 19:27:52 +0200 (CEST) Received: from [192.168.0.2] (henkezan.henriklidstrom.se [192.168.0.2]) by mailgw.henriklidstrom.se (Postfix) with ESMTP id DE4A5B821; Thu, 5 Oct 2006 19:27:52 +0200 (CEST) Message-ID: <452540A0.3060602@lidstrom.eu> Date: Thu, 05 Oct 2006 19:28:00 +0200 From: =?ISO-8859-1?Q?Henrik_Lidstr=F6m?= User-Agent: Thunderbird 1.5.0.7 (Windows/20060909) MIME-Version: 1.0 To: Jerry Bell References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> In-Reply-To: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at henriklidstrom.se Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 17:29:29 -0000 Jerry Bell skrev: > I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. > I've looked through some of the lists here and have seen super-smack > results in the 42k qps range on a 2 dual core opteron system. I'm able to > get up to about 34k with the wide at the back of my server whilest rubbing > the side of it. > > Isn't the disk subsystem of relevance here? Whats your setup? /Henrik From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 17:29:39 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0FDC216A49E for ; Thu, 5 Oct 2006 17:29:39 +0000 (UTC) (envelope-from hadara@bsd.ee) Received: from mx1.starman.ee (smtp-out2.starman.ee [85.253.0.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED5BF43E02 for ; Thu, 5 Oct 2006 17:28:45 +0000 (GMT) (envelope-from hadara@bsd.ee) Received: from depression.softematic.com (depression.softematic.com [62.65.205.81]) by mx1.starman.ee (Postfix) with ESMTP id 763BD23C435; Thu, 5 Oct 2006 20:28:42 +0300 (EEST) From: Sven Petai To: freebsd-performance@freebsd.org Date: Thu, 5 Oct 2006 20:28:39 +0300 User-Agent: KMail/1.9.3 References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> In-Reply-To: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200610052028.41041.hadara@bsd.ee> X-Virus-Scanned: by Amavisd-New at mx1.starman.ee Cc: Jerry Bell Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 17:29:39 -0000 On Thursday 05 October 2006 03:52, Jerry Bell wrote: > I expected the 2950 to be a bit closer to the 1.8Ghz opteron discussed > here: > http://article.gmane.org/gmane.os.freebsd.performance/1137/match=mysql Well he used several patches [1] from the current + ULE scheduler, which seems to give you better results in this particular benchmark, but sometimes seems to have some weird sideefects for realworld usage. Also keep in mind that default select-key smack is just very hard on some specific subsystems. The load it produces is rather unrealistic so better mysql supersmack scores might, but don't necessarily, translate into better real performance. You might get some idea on how different patches and kernelconfigs affect supersmack results on SMP from my latest benchmark run on 8 core machine: http://bsd.ee/~hadara/debug/mysql_sep_2006/stats.html With 8 cores my performance results are of course seriously pessimized by locking contention, I'm not sure how much of a problem it's with 4 cores, but it's very likely that rwatsons uds locking patch [2] that I used will help some in your case too, though I have no idea if it applies to 6.X tree without some hacking. ------- [1] http://lists.freebsd.org/pipermail/freebsd-performance/2006-April/001869.html [2] http://www.watson.org/~robert/freebsd/netperf/20060822-uds-fine-grain.diff From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 19:17:19 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CCCD916A403 for ; Thu, 5 Oct 2006 19:17:19 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from slimak.dkm.cz (slimak.dkm.cz [62.24.64.34]) by mx1.FreeBSD.org (Postfix) with SMTP id 182A343D62 for ; Thu, 5 Oct 2006 19:17:15 +0000 (GMT) (envelope-from 000.fbsd@quip.cz) Received: (qmail 48529 invoked by uid 0); 5 Oct 2006 19:17:10 -0000 Received: from grimm.quip.cz (HELO ?192.168.1.2?) (213.220.192.218) by slimak.dkm.cz with SMTP; 5 Oct 2006 19:17:10 -0000 Message-ID: <45255A36.5010108@quip.cz> Date: Thu, 05 Oct 2006 21:17:10 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, en, cs, en-us MIME-Version: 1.0 To: Jerry Bell References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> In-Reply-To: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 19:17:19 -0000 Jerry Bell wrote: > I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. > I've looked through some of the lists here and have seen super-smack > results in the 42k qps range on a 2 dual core opteron system. I'm able to > get up to about 34k with the wide at the back of my server whilest rubbing > the side of it. > > Here's what I've done: > built both mysql 5.0 and 5.1 from ports with build_static and > optimazations on. > changed the clock to TSC > > added the following to my /etc/libmap.conf file: > [mysqld] > libc_r.so.5 libthr.so.2 > libc_r.so.6 libthr.so.2 > libthr.so.2 libthr.so.2 > libpthread.so.1 libthr.so.2 > libpthread.so.2 libthr.so.2 [...] As Nick Evans said, you can't use static version of MySQL daemon if you want to use /etc/libmap.conf I tested both (static vs. dynamic with libmap.conf), dynamic with libthr performs much better than static on Dual Xeon 3GHz SMP system with 2GB of RAM. My /etc/libmap.conf is just [/usr/local/libexec/mysqld] libpthread.so.2 libthr.so.2 libpthread.so libthr.so Miroslav Lachman From owner-freebsd-performance@FreeBSD.ORG Thu Oct 5 23:24:06 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DEFE16A407 for ; Thu, 5 Oct 2006 23:24:06 +0000 (UTC) (envelope-from jbell@stelesys.com) Received: from stelesys.com (web3.stelesys.com [63.175.100.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id D77C043D45 for ; Thu, 5 Oct 2006 23:24:05 +0000 (GMT) (envelope-from jbell@stelesys.com) Received: from [127.0.0.1] (helo=www.stelesys.com) by stelesys.com with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GVcZQ-000Hh7-QB for freebsd-performance@freebsd.org; Thu, 05 Oct 2006 19:24:04 -0400 Received: from 71.56.92.181 (SquirrelMail authenticated user jbell@stelesys.com) by www.stelesys.com with HTTP; Thu, 5 Oct 2006 19:24:04 -0400 (EDT) Message-ID: <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> In-Reply-To: <45255A36.5010108@quip.cz> References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> <45255A36.5010108@quip.cz> Date: Thu, 5 Oct 2006 19:24:04 -0400 (EDT) From: "Jerry Bell" To: freebsd-performance@freebsd.org User-Agent: SquirrelMail/1.4.6-rc1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 23:24:06 -0000 Many thanks to all who responded. You are an incredibly smart group of people. The recompiling without static yielded much better results: 2950# super-smack -d mysql select-key-mysql.smack 10 10000 Query Barrel Report for client smacker1 connect: max=1ms min=0ms avg= 0ms from 10 clients Query_type num_queries max_time min_time q_per_s select_index 200000 0 0 51118.22 >From ~34k qps to ~51k qps is quite an improvement! I always thougt that compiling something static increased performance, but then that's probably true for things that have to startup and shutdown frequently. Thanks again. Jerry > Jerry Bell wrote: > >> I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. >> I've looked through some of the lists here and have seen super-smack >> results in the 42k qps range on a 2 dual core opteron system. I'm able >> to >> get up to about 34k with the wide at the back of my server whilest >> rubbing >> the side of it. >> >> Here's what I've done: >> built both mysql 5.0 and 5.1 from ports with build_static and >> optimazations on. >> changed the clock to TSC >> >> added the following to my /etc/libmap.conf file: >> [mysqld] >> libc_r.so.5 libthr.so.2 >> libc_r.so.6 libthr.so.2 >> libthr.so.2 libthr.so.2 >> libpthread.so.1 libthr.so.2 >> libpthread.so.2 libthr.so.2 > > [...] > > As Nick Evans said, you can't use static version of MySQL daemon if you > want to use /etc/libmap.conf > > I tested both (static vs. dynamic with libmap.conf), dynamic with libthr > performs much better than static on Dual Xeon 3GHz SMP system with 2GB > of RAM. > > My /etc/libmap.conf is just > > [/usr/local/libexec/mysqld] > libpthread.so.2 libthr.so.2 > libpthread.so libthr.so > > Miroslav Lachman > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Fri Oct 6 04:48:27 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id D64C516A407; Fri, 6 Oct 2006 04:48:26 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-performance@freebsd.org Date: Fri, 6 Oct 2006 12:48:22 +0800 User-Agent: KMail/1.8.2 References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> <45255A36.5010108@quip.cz> <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> In-Reply-To: <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200610061248.22136.davidxu@freebsd.org> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Cc: Jerry Bell Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 04:48:27 -0000 On Friday 06 October 2006 07:24, Jerry Bell wrote: > I always thougt that compiling something static increased performance, but > then that's probably true for things that have to startup and shutdown > frequently. > > Thanks again. > > Jerry > static compiling will link libpthread but not libthr. I found setting larger buffer in /etc/my.cnf will yield much better result than default configuration, also turnning off log-bin option makes a different on my machine. mysql 5.x is much better than 4.1 on select benchmark, almost extra 25% performance improvement I can get on athlon64 X2 3800+ running in 64-bit mode. David Xu From owner-freebsd-performance@FreeBSD.ORG Fri Oct 6 07:20:55 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0CF0C16A47C for ; Fri, 6 Oct 2006 07:20:55 +0000 (UTC) (envelope-from mv@thebeastie.org) Received: from p4.roq.com (ns1.ecoms.com [207.44.130.137]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2EAD43D72 for ; Fri, 6 Oct 2006 07:20:46 +0000 (GMT) (envelope-from mv@thebeastie.org) Received: from p4.roq.com (localhost.roq.com [127.0.0.1]) by p4.roq.com (Postfix) with ESMTP id 924624CD9D for ; Fri, 6 Oct 2006 07:21:03 +0000 (GMT) Received: from beaste5.jumbuck.com (ppp166-27.static.internode.on.net [150.101.166.27]) by p4.roq.com (Postfix) with ESMTP id D9AC04CD9A for ; Fri, 6 Oct 2006 07:21:02 +0000 (GMT) Received: from beaste5.jumbuck.com (beast5 [192.168.46.105]) by beaste5.jumbuck.com (Postfix) with ESMTP id ABFD220A1784; Fri, 6 Oct 2006 17:20:43 +1000 (EST) Received: from [192.168.46.102] (unknown [192.168.46.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by beaste5.jumbuck.com (Postfix) with ESMTP id 70BBC20A16E8; Fri, 6 Oct 2006 17:20:43 +1000 (EST) Message-ID: <452603CA.8070800@thebeastie.org> Date: Fri, 06 Oct 2006 17:20:42 +1000 From: Michael Vince User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.13) Gecko/20060727 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jerry Bell References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> <45255A36.5010108@quip.cz> <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> In-Reply-To: <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 07:20:55 -0000 Yeah the static compiling recommendations by MySQL documents are really more a linux thing more then anything else. The other other thing to check is to make sure you use larger buffer settings I recommend the large-my.cnf cp /usr/local/share/mysql/my-large.cnf /var/db/mysql/ Then restart MySQL. The disk IO doesn't play much into this as far as I know because the supersmack benchmark doesn't generate a large database to play with, also because its a med-high end Dell 1950 it should have the 256meg read and battery backed write cache controller card which would also hold a lot of that data without doing much real access. Yup there you have it, even though you didn't know about the best tweaks for MySQL you still managed to practically post some of the highest MySQL benchmarks posted here on performance out of the box with libthr, it also shows how much libthr gives extra performance when you have 4 CPU cores over most peoples posts with 1 or 2 CPU cores. Jerry Bell wrote: >Many thanks to all who responded. You are an incredibly smart group of >people. > >The recompiling without static yielded much better results: > >2950# super-smack -d mysql select-key-mysql.smack 10 10000 >Query Barrel Report for client smacker1 >connect: max=1ms min=0ms avg= 0ms from 10 clients >Query_type num_queries max_time min_time q_per_s >select_index 200000 0 0 51118.22 > >>From ~34k qps to ~51k qps is quite an improvement! > >I always thougt that compiling something static increased performance, but >then that's probably true for things that have to startup and shutdown >frequently. > >Thanks again. > >Jerry > > >>Jerry Bell wrote: >> >> >> >>>I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. >>>I've looked through some of the lists here and have seen super-smack >>>results in the 42k qps range on a 2 dual core opteron system. I'm able >>>to >>>get up to about 34k with the wide at the back of my server whilest >>>rubbing >>>the side of it. >>> >>>Here's what I've done: >>>built both mysql 5.0 and 5.1 from ports with build_static and >>>optimazations on. >>>changed the clock to TSC >>> >>>added the following to my /etc/libmap.conf file: >>>[mysqld] >>>libc_r.so.5 libthr.so.2 >>>libc_r.so.6 libthr.so.2 >>>libthr.so.2 libthr.so.2 >>>libpthread.so.1 libthr.so.2 >>>libpthread.so.2 libthr.so.2 >>> >>> >>[...] >> >>As Nick Evans said, you can't use static version of MySQL daemon if you >>want to use /etc/libmap.conf >> >>I tested both (static vs. dynamic with libmap.conf), dynamic with libthr >>performs much better than static on Dual Xeon 3GHz SMP system with 2GB >>of RAM. >> >>My /etc/libmap.conf is just >> >>[/usr/local/libexec/mysqld] >>libpthread.so.2 libthr.so.2 >>libpthread.so libthr.so >> >>Miroslav Lachman >>_______________________________________________ >>freebsd-performance@freebsd.org mailing list >>http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>To unsubscribe, send any mail to >>"freebsd-performance-unsubscribe@freebsd.org" >> >> >> > > >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > From owner-freebsd-performance@FreeBSD.ORG Fri Oct 6 07:57:42 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D273E16A407 for ; Fri, 6 Oct 2006 07:57:42 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.176.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3A3D543D46 for ; Fri, 6 Oct 2006 07:57:41 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.8/8.13.7) with ESMTP id k967vcdV064935 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 6 Oct 2006 09:57:38 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.8/8.13.3/Submit) id k967vcEF064934; Fri, 6 Oct 2006 09:57:38 +0200 (CEST) Date: Fri, 6 Oct 2006 09:57:38 +0200 From: Divacky Roman To: Jerry Bell Message-ID: <20061006075738.GA64854@stud.fit.vutbr.cz> References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.57 on 147.229.176.14 Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 07:57:42 -0000 On Wed, Oct 04, 2006 at 08:52:51PM -0400, Jerry Bell wrote: > I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. > I've looked through some of the lists here and have seen super-smack > results in the 42k qps range on a 2 dual core opteron system. I'm able to > get up to about 34k with the wide at the back of my server whilest rubbing > the side of it. > > Here's what I've done: > built both mysql 5.0 and 5.1 from ports with build_static and > optimazations on. > changed the clock to TSC > > added the following to my /etc/libmap.conf file: > [mysqld] > libc_r.so.5 libthr.so.2 > libc_r.so.6 libthr.so.2 > libthr.so.2 libthr.so.2 > libpthread.so.1 libthr.so.2 > libpthread.so.2 libthr.so.2 > > And this is about the best result I get: > 2950# super-smack -d mysql select-key-mysql.smack 10 10000 > Query Barrel Report for client smacker1 > connect: max=200ms min=0ms avg= 20ms from 10 clients > Query_type num_queries max_time min_time q_per_s > select_index 200000 0 0 34440.88 > > > Are there any other optimizations I need to do to mysql or freebsd? I'm > still not 100% convinced that I've done everything I need to use libthr > instead of libpthread. on UP people recomend to use TSC instead of acpi timer From owner-freebsd-performance@FreeBSD.ORG Fri Oct 6 13:16:32 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80C8416A417 for ; Fri, 6 Oct 2006 13:16:32 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from www.ebusiness-leidinger.de (jojo.ms-net.de [84.16.236.246]) by mx1.FreeBSD.org (Postfix) with ESMTP id A860843D46 for ; Fri, 6 Oct 2006 13:16:31 +0000 (GMT) (envelope-from Alexander@Leidinger.net) Received: from Andro-Beta.Leidinger.net (p54A5F0B9.dip.t-dialin.net [84.165.240.185]) (authenticated bits=0) by www.ebusiness-leidinger.de (8.13.6/8.13.6) with ESMTP id k96DFHd2073724; Fri, 6 Oct 2006 15:15:18 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from localhost (webmail.Leidinger.net [192.168.1.102]) by Andro-Beta.Leidinger.net (8.13.4/8.13.4) with ESMTP id k96DGS6F025143; Fri, 6 Oct 2006 15:16:28 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 06 Oct 2006 15:16:28 +0200 Message-ID: <20061006151628.egmzstaiqsw48kk8@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 06 Oct 2006 15:16:28 +0200 From: Alexander Leidinger To: Divacky Roman References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> <20061006075738.GA64854@stud.fit.vutbr.cz> In-Reply-To: <20061006075738.GA64854@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.3) / FreeBSD-7.0 X-Virus-Scanned: by amavisd-new Cc: freebsd-performance@freebsd.org, Jerry Bell Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 13:16:32 -0000 Quoting Divacky Roman (from Fri, 6 Oct =20 2006 09:57:38 +0200): > On Wed, Oct 04, 2006 at 08:52:51PM -0400, Jerry Bell wrote: >> I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. >> I've looked through some of the lists here and have seen super-smack >> results in the 42k qps range on a 2 dual core opteron system. I'm able t= o >> get up to about 34k with the wide at the back of my server whilest rubbin= g >> the side of it. >> >> Here's what I've done: >> built both mysql 5.0 and 5.1 from ports with build_static and >> optimazations on. >> changed the clock to TSC >> >> added the following to my /etc/libmap.conf file: >> [mysqld] >> libc_r.so.5 libthr.so.2 >> libc_r.so.6 libthr.so.2 >> libthr.so.2 libthr.so.2 >> libpthread.so.1 libthr.so.2 >> libpthread.so.2 libthr.so.2 >> >> And this is about the best result I get: >> 2950# super-smack -d mysql select-key-mysql.smack 10 10000 >> Query Barrel Report for client smacker1 >> connect: max=3D200ms min=3D0ms avg=3D 20ms from 10 clients >> Query_type num_queries max_time min_time q_per_s >> select_index 200000 0 0 34440.88 >> >> >> Are there any other optimizations I need to do to mysql or freebsd? I'm >> still not 100% convinced that I've done everything I need to use libthr >> instead of libpthread. > > on UP people recomend to use TSC instead of acpi timer He uses the TSC already... but nobody suggested the gettimeofday() =20 wrapper so far: http://wiki.freebsd.org/MySQL Bye, Alexander. --=20 Success is the sole earthly judge of right and wrong. =09=09-- Adolf Hitler, "Mein Kampf" http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-performance@FreeBSD.ORG Fri Oct 6 19:20:51 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E064916A4F4 for ; Fri, 6 Oct 2006 19:20:51 +0000 (UTC) (envelope-from jbell@stelesys.com) Received: from stelesys.com (web3.stelesys.com [63.175.100.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE59243D7B for ; Fri, 6 Oct 2006 19:20:50 +0000 (GMT) (envelope-from jbell@stelesys.com) Received: from [127.0.0.1] (helo=www.stelesys.com) by stelesys.com with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GVvFW-000464-7u; Fri, 06 Oct 2006 15:20:46 -0400 Received: from 209.134.164.20 (SquirrelMail authenticated user jbell@stelesys.com) by www.stelesys.com with HTTP; Fri, 6 Oct 2006 15:20:46 -0400 (EDT) Message-ID: <4472.209.134.164.20.1160162446.squirrel@www.stelesys.com> In-Reply-To: <452603CA.8070800@thebeastie.org> References: <3731.71.56.92.181.1160009571.squirrel@www.stelesys.com> <45255A36.5010108@quip.cz> <2840.71.56.92.181.1160090644.squirrel@www.stelesys.com> <452603CA.8070800@thebeastie.org> Date: Fri, 6 Oct 2006 15:20:46 -0400 (EDT) From: "Jerry Bell" To: "Michael Vince" User-Agent: SquirrelMail/1.4.6-rc1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-performance@freebsd.org Subject: Re: Help with improving mysql performance on 6.2PRE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 19:20:52 -0000 I have actually made the changes to my.cnf before I ran these. I expanded them quite a bit beyond what is in my-large.cnf. I need to pull them back in to save on some memory usage. I'm going to look at some of the other patches that have been suggested to me to see if they'll work and if they make a difference. Right now, my system only has the SAS5i card, with no memory. I have a PERC5i on order, but it's backordered and I'm thinking about whether I should go with a different card or not. It's relatively expensive, and I'm not sure how the performance stacks up, but I hear a lot of complaints about the management support of the PERC cards in Freebsd. Thanks to all again, Jerry > Yeah the static compiling recommendations by MySQL documents are really > more a linux thing more then anything else. > The other other thing to check is to make sure you use larger buffer > settings I recommend the large-my.cnf > cp /usr/local/share/mysql/my-large.cnf /var/db/mysql/ > Then restart MySQL. > > The disk IO doesn't play much into this as far as I know because the > supersmack benchmark doesn't generate a large database to play with, > also because its a med-high end Dell 1950 it should have the 256meg read > and battery backed write cache controller card which would also hold a > lot of that data without doing much real access. > > Yup there you have it, even though you didn't know about the best tweaks > for MySQL you still managed to practically post some of the highest > MySQL benchmarks posted here on performance out of the box with libthr, > it also shows how much libthr gives extra performance when you have 4 > CPU cores over most peoples posts with 1 or 2 CPU cores. > > > > Jerry Bell wrote: > >>Many thanks to all who responded. You are an incredibly smart group of >>people. >> >>The recompiling without static yielded much better results: >> >>2950# super-smack -d mysql select-key-mysql.smack 10 10000 >>Query Barrel Report for client smacker1 >>connect: max=1ms min=0ms avg= 0ms from 10 clients >>Query_type num_queries max_time min_time q_per_s >>select_index 200000 0 0 51118.22 >> >>>From ~34k qps to ~51k qps is quite an improvement! >> >>I always thougt that compiling something static increased performance, >> but >>then that's probably true for things that have to startup and shutdown >>frequently. >> >>Thanks again. >> >>Jerry >> >> >>>Jerry Bell wrote: >>> >>> >>> >>>>I have a Dell PE2950 with 2 dual core 3.73Ghz processors and 4G of ram. >>>>I've looked through some of the lists here and have seen super-smack >>>>results in the 42k qps range on a 2 dual core opteron system. I'm able >>>>to >>>>get up to about 34k with the wide at the back of my server whilest >>>>rubbing >>>>the side of it. >>>> >>>>Here's what I've done: >>>>built both mysql 5.0 and 5.1 from ports with build_static and >>>>optimazations on. >>>>changed the clock to TSC >>>> >>>>added the following to my /etc/libmap.conf file: >>>>[mysqld] >>>>libc_r.so.5 libthr.so.2 >>>>libc_r.so.6 libthr.so.2 >>>>libthr.so.2 libthr.so.2 >>>>libpthread.so.1 libthr.so.2 >>>>libpthread.so.2 libthr.so.2 >>>> >>>> >>>[...] >>> >>>As Nick Evans said, you can't use static version of MySQL daemon if you >>>want to use /etc/libmap.conf >>> >>>I tested both (static vs. dynamic with libmap.conf), dynamic with libthr >>>performs much better than static on Dual Xeon 3GHz SMP system with 2GB >>>of RAM. >>> >>>My /etc/libmap.conf is just >>> >>>[/usr/local/libexec/mysqld] >>>libpthread.so.2 libthr.so.2 >>>libpthread.so libthr.so >>> >>>Miroslav Lachman >>>_______________________________________________ >>>freebsd-performance@freebsd.org mailing list >>>http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>To unsubscribe, send any mail to >>>"freebsd-performance-unsubscribe@freebsd.org" >>> >>> >>> >> >> >>_______________________________________________ >>freebsd-performance@freebsd.org mailing list >>http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" >> >> > >