From owner-freebsd-performance@FreeBSD.ORG Mon Sep 19 03:37:44 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6871016A41F; Mon, 19 Sep 2005 03:37:44 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 431CA43D48; Mon, 19 Sep 2005 03:37:43 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from beastie.frontfree.net (unknown [211.71.95.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id EE576EB08D7; Mon, 19 Sep 2005 11:37:37 +0800 (CST) Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id DCD5413159E; Mon, 19 Sep 2005 11:37:33 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 84846-16; Mon, 19 Sep 2005 11:37:24 +0800 (CST) Received: from [10.217.12.235] (sina152-194.staff.sina.com.cn [61.135.152.194]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by beastie.frontfree.net (Postfix) with ESMTP id 214D3130C42; Mon, 19 Sep 2005 11:37:24 +0800 (CST) From: Xin LI To: freebsd-arch@FreeBSD.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-MlGidjHbWMxS/2Zg/9Vh" Organization: The FreeBSD Simplified Chinese Project Date: Mon, 19 Sep 2005 11:37:22 +0800 Message-Id: <1127101042.788.30.camel@spirit> Mime-Version: 1.0 X-Mailer: Evolution 2.2.3 FreeBSD GNOME Team Port X-Virus-Scanned: amavisd-new at frontfree.net Cc: freebsd-performance@FreeBSD.org Subject: Combine more operation within one system call: to do it, or not to do it? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Sep 2005 03:37:44 -0000 --=-MlGidjHbWMxS/2Zg/9Vh Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Dear folks, It seems that Microsoft has recently revised several of their APIs. One example is their ConnectEx(), as found in documentation [1]. The implementation is not so complex that it just combines more operation within one system call, however, this can reduce some unnecessary context switches as it's now possible to do more things within one system call. (For instance, when you connect to a server, you usually want to send some data as request). Shall we do something similar? Or do we already done something similar? [1] http://msdn.microsoft.com/library/default.asp?url=3D/library/en-us/winsock/= winsock/connectex_2.asp Cheers, --=20 Xin LI http://www.delphij.net/ --=-MlGidjHbWMxS/2Zg/9Vh Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBDLjJy/cVsHxFZiIoRAssxAKCD3rV/yiXY//MqGAMqNT6NjCpe0wCeMQ1V qlRLNc56p9nQwrCPLiBG7/E= =ZTQ/ -----END PGP SIGNATURE----- --=-MlGidjHbWMxS/2Zg/9Vh-- From owner-freebsd-performance@FreeBSD.ORG Mon Sep 19 09:42:43 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 035F816A41F; Mon, 19 Sep 2005 09:42:43 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from mail16.syd.optusnet.com.au (mail16.syd.optusnet.com.au [211.29.132.197]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A1C843D45; Mon, 19 Sep 2005 09:42:41 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (c220-239-19-236.belrs4.nsw.optusnet.com.au [220.239.19.236]) by mail16.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id j8J9gb4S007845 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Mon, 19 Sep 2005 19:42:39 +1000 Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.10/8.12.10) with ESMTP id j8J9gaSR054941; Mon, 19 Sep 2005 19:42:36 +1000 (EST) (envelope-from pjeremy@cirb503493.alcatel.com.au) Received: (from pjeremy@localhost) by cirb503493.alcatel.com.au (8.12.10/8.12.9/Submit) id j8J9gZ3j054940; Mon, 19 Sep 2005 19:42:35 +1000 (EST) (envelope-from pjeremy) Date: Mon, 19 Sep 2005 19:42:34 +1000 From: Peter Jeremy To: Xin LI Message-ID: <20050919094234.GG40237@cirb503493.alcatel.com.au> References: <1127101042.788.30.camel@spirit> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1127101042.788.30.camel@spirit> User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Mon, 19 Sep 2005 12:31:37 +0000 Cc: freebsd-performance@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Combine more operation within one system call: to do it, or not to do it? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Sep 2005 09:42:43 -0000 On Mon, 2005-Sep-19 11:37:22 +0800, Xin LI wrote: >It seems that Microsoft has recently revised several of their APIs. I think this is a regular occurence as part of their ongoing efforts to minimise backward and forward compatibility. > One example is their ConnectEx(), as found in documentation [1]. Does this represent a measurable improvement in a real-world situation? >Shall we do something similar? Looking at it from the application writer's POV: Implementing a special case for one OS (when that OS also supports the standard mechanism) requires additional effort and there needs to be good justification for expending that effort. Overall, orphan functionality is unlikely to be used. Unless you can convince several other vendors (*BSD, Linux or a commercial vendor) that the same functionality is worth implementing, you're better off not bothering. > Or do we already done something similar? How about sendto(2)? >http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/connectex_2.asp This doesn't work with lynx and I don't have my mozilla running. -- Peter Jeremy From owner-freebsd-performance@FreeBSD.ORG Tue Sep 20 02:34:27 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31D6916A41F; Tue, 20 Sep 2005 02:34:27 +0000 (GMT) (envelope-from peter@wemm.org) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8948343D46; Tue, 20 Sep 2005 02:34:26 +0000 (GMT) (envelope-from peter@wemm.org) Received: from fw.wemm.org (canning.wemm.org [192.203.228.65]) by canning.wemm.org (Postfix) with ESMTP id 451E82A945; Mon, 19 Sep 2005 19:34:26 -0700 (PDT) (envelope-from peter@wemm.org) Received: from overcee.wemm.org (overcee.wemm.org [10.0.0.3]) by fw.wemm.org (Postfix) with ESMTP id B5935E2B3; Mon, 19 Sep 2005 19:34:25 -0700 (PDT) (envelope-from peter@wemm.org) Received: from overcee.wemm.org (localhost [127.0.0.1]) by overcee.wemm.org (8.13.4/8.13.4) with ESMTP id j8K2YPpF041974; Mon, 19 Sep 2005 19:34:25 -0700 (PDT) (envelope-from peter@wemm.org) Received: from localhost (localhost [[UNIX: localhost]]) by overcee.wemm.org (8.13.4/8.13.1/Submit) id j8K2YOhk041971; Mon, 19 Sep 2005 19:34:24 -0700 (PDT) (envelope-from peter@wemm.org) X-Authentication-Warning: overcee.wemm.org: peter set sender to peter@wemm.org using -f From: Peter Wemm To: freebsd-arch@freebsd.org Date: Mon, 19 Sep 2005 19:34:23 -0700 User-Agent: KMail/1.8.1 References: <1127101042.788.30.camel@spirit> <20050919094234.GG40237@cirb503493.alcatel.com.au> In-Reply-To: <20050919094234.GG40237@cirb503493.alcatel.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200509191934.23991.peter@wemm.org> X-Mailman-Approved-At: Tue, 20 Sep 2005 05:22:20 +0000 Cc: Peter Jeremy , freebsd-performance@freebsd.org, Xin LI Subject: Re: Combine more operation within one system call: to do it, or not to do it? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2005 02:34:27 -0000 On Monday 19 September 2005 02:42 am, Peter Jeremy wrote: > On Mon, 2005-Sep-19 11:37:22 +0800, Xin LI wrote: > >It seems that Microsoft has recently revised several of their APIs. > > I think this is a regular occurence as part of their ongoing efforts > to minimise backward and forward compatibility. > > > One example is their ConnectEx(), as found in documentation [1]. > > Does this represent a measurable improvement in a real-world > situation? The other consideration is that our syscalls are generally pretty quick on most of our platforms. We don't normally context switch for a syscall - well, we save and restore registers, but that isn't too bad compared to the i386 tss and ldt etc switches for process context switches. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 01:46:48 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CFAD816A41F for ; Fri, 23 Sep 2005 01:46:48 +0000 (GMT) (envelope-from lists@natserv.com) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8392443D46 for ; Fri, 23 Sep 2005 01:46:44 +0000 (GMT) (envelope-from lists@natserv.com) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id 64E1A7DB0; Thu, 22 Sep 2005 21:46:43 -0400 (EDT) Date: Thu, 22 Sep 2005 21:46:43 -0400 (EDT) From: Francisco Reyes X-X-Sender: fran@zoraida.natserv.net To: mariano benedettini In-Reply-To: <15412.1126634818@www56.gmx.net> Message-ID: <20050922214142.N50836@zoraida.natserv.net> References: <15412.1126634818@www56.gmx.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 01:46:48 -0000 On Tue, 13 Sep 2005, mariano benedettini wrote: >91.3% idle CPU is not the problem. :-) > Mem: 1599M Active, 1704M Inact, 311M Wired, 189M Cache, 112M Buf, 14M Free > Swap: 2023M Total, 184K Used, 2023M Free Swap is not the problem. Do vmstat 10 Watch the output. In particular look at the first 3 columns. procs r b w 1 1 0 0 1 0 1 1 0 The left most column is CPU, the second column is disk IO. If you have a number in the "b" column and it never hits 0 you have an I/O problem. You HDs are not catching up. If you are using NFS and the "b" colun is not high and hits 0 some/all the time then the bottleneck is either the nfs connection or the nfs server. For example I have some servers that the "b" column would be between 20 and 60 for a while. I am currently working on removing some of the load of the machine. In my case more memory would help, but the computer vendor we bought the machine from has sent us the wrong memory 3 TIMES!! From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 01:52:24 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 556DF16A41F for ; Fri, 23 Sep 2005 01:52:24 +0000 (GMT) (envelope-from lists@natserv.com) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1076943D48 for ; Fri, 23 Sep 2005 01:52:24 +0000 (GMT) (envelope-from lists@natserv.com) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id 778537DE1 for ; Thu, 22 Sep 2005 21:52:23 -0400 (EDT) Date: Thu, 22 Sep 2005 21:52:23 -0400 (EDT) From: Francisco Reyes X-X-Sender: fran@zoraida.natserv.net To: FreeBSD Performance Message-ID: <20050922214709.Q50836@zoraida.natserv.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Finding what's causing I/O X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 01:52:24 -0000 Looking at vmstat I see the "b" colun never hits zero and it's usually between 5 and 20. Is there a way to find out which program(s) are causing the I/O? In some of the machines it was near trivial to find the culprit, but have a handfull of machines that I am not sure what the cause of I/O is. The machines are mailservers so they likely will not hold any particular file open long. From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 02:04:10 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A07C416A41F for ; Fri, 23 Sep 2005 02:04:10 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 30C0F43D45 for ; Fri, 23 Sep 2005 02:04:07 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j8N246rh079316; Thu, 22 Sep 2005 21:04:06 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <43336294.2020403@centtech.com> Date: Thu, 22 Sep 2005 21:04:04 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco Reyes References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> In-Reply-To: <20050922214142.N50836@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1098/Thu Sep 22 15:57:50 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org, mariano benedettini Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 02:04:10 -0000 Francisco Reyes wrote: > On Tue, 13 Sep 2005, mariano benedettini wrote: > >> 91.3% idle > > > CPU is not the problem. :-) > > >> Mem: 1599M Active, 1704M Inact, 311M Wired, 189M Cache, 112M Buf, 14M >> Free >> Swap: 2023M Total, 184K Used, 2023M Free > > > Swap is not the problem. > > > Do > vmstat 10 > > Watch the output. > In particular look at the first 3 columns. > procs > r b w > 1 1 0 > 0 1 0 > 1 1 0 > > The left most column is CPU, the second column is disk IO. > > If you have a number in the "b" column and it never hits 0 you have an > I/O problem. You HDs are not catching up. > > If you are using NFS and the "b" colun is not high and hits 0 some/all > the time then the bottleneck is either the nfs connection or the nfs > server. > > For example I have some servers that the "b" column would be between 20 > and 60 for a while. I am currently working on removing some of the load > of the machine. In my case more memory would help, but the computer > vendor we bought the machine from has sent us the wrong memory 3 TIMES!! Also, if it is an NFS server, one should check the cpu times on the nfsd processes. I've found that many times there aren't enough nfsd processes to take the load from many clients. Increasing the number (double it) often helps this. The max in 5.3 is 20, but you can easily change it and get around it. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 02:09:13 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DF96D16A41F for ; Fri, 23 Sep 2005 02:09:13 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 72AE143D45 for ; Fri, 23 Sep 2005 02:09:13 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: by xproxy.gmail.com with SMTP id t13so512412wxc for ; Thu, 22 Sep 2005 19:09:12 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=hAiTEiUBryAICSsrNpGt1kCdkXOleO3Q7IgFBrz+m5fo+SbypsYj3tppH5X/ZVG1fLqpk/Wl4zz3wcCOYx+4UMsWeIfogNfK/L1eT/xo7mltjPc8//B03KrmuMv6ximQROaQdT8RQziip0oyxjLwzAbFbti/1zaIGOXJvAmdYBA= Received: by 10.70.113.16 with SMTP id l16mr712113wxc; Thu, 22 Sep 2005 19:02:51 -0700 (PDT) Received: by 10.70.115.15 with HTTP; Thu, 22 Sep 2005 19:02:51 -0700 (PDT) Message-ID: <84dead7205092219023228cdf5@mail.gmail.com> Date: Fri, 23 Sep 2005 07:32:51 +0530 From: Joseph Koshy To: Francisco Reyes In-Reply-To: <20050922214709.Q50836@zoraida.natserv.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <20050922214709.Q50836@zoraida.natserv.net> Cc: FreeBSD Performance Subject: Re: Finding what's causing I/O X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joseph Koshy List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 02:09:14 -0000 > Is there a way to find out which program(s) are causing > the I/O? ktrace(8); you can use it to trace all descendants of 'init'. -- FreeBSD Volunteer, http://people.freebsd.org/~jkoshy From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 03:05:09 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6744416A41F for ; Fri, 23 Sep 2005 03:05:09 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 020C843D46 for ; Fri, 23 Sep 2005 03:05:08 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 67CFF5EE5; Thu, 22 Sep 2005 23:05:08 -0400 (EDT) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 36762-08; Thu, 22 Sep 2005 23:05:07 -0400 (EDT) Received: from [192.168.1.3] (pool-68-161-68-11.ny325.east.verizon.net [68.161.68.11]) by pi.codefab.com (Postfix) with ESMTP id 560F95D14; Thu, 22 Sep 2005 23:05:07 -0400 (EDT) Message-ID: <433370E4.8060708@mac.com> Date: Thu, 22 Sep 2005 23:05:08 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050728 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco References: <63f9d26505090417183dff415e@mail.gmail.com> <431C683B.1080803@mac.com> <20050922215326.B50836@zoraida.natserv.net> In-Reply-To: <20050922215326.B50836@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: Jeff Tchang , freebsd-performance@freebsd.org Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 03:05:09 -0000 Francisco wrote: > On Mon, 5 Sep 2005, Chuck Swiger wrote: >> Small writes are pretty much the worst-case scenario for RAID-5, > > Such as mail servers? So-so. RAID-5 is okay on a IMAP reader box, it's not so good for a pure SMTP relay, especially one that does virus scanning. > How about for a DB server which is mostly read only? If your DB claims to support a RAID-5 configuration-- some DBs will change their caching behavior to avoid thrashing a RAID-5 volume as much-- it might be OK. If you're going to run a big DB, you really ought to be designing the disk layout according to what the DB vendor recommends. >> normal to see a very significant performance drop-- by up to an order >> of magnitude-- from the performance of a bare drive. > > At which point Raid 5 starts to perform better? > 6,8,10 drives? Better for small writes? Never. Although good hardware and lots of RAM to cache with can help a lot. RAID systems have bus limitations on how wide they can go in terms of # of drives, also in how much real bus bandwidth is available for very wide configs. 8 drives is a common maximum width. > How about RAID 10 for a DB server? This is a much better choice, close to ideal. > I have been trying to convince the "powers that be" that SCSI would be > much better.. but the price difference is just too astronomical for the > capacities we need (500GB to 2 TB) > > Even 10K RPM IDE drives seem like would be a problem since they are > mostly small in size. Ten 72's would be in the right ballpark, that's about $2000. Ten of the cheapest reasonable 80GB ATA drives would be about $800. You could always ask: "How much is your data worth to your company, again?" You can get 146's for about $500 and even 300GB SCSI-3 drives exist. -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 07:05:22 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E54C116A41F for ; Fri, 23 Sep 2005 07:05:22 +0000 (GMT) (envelope-from ap@bnc.net) Received: from mailomat.net (mailomat.net [217.110.117.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 535EB43D45 for ; Fri, 23 Sep 2005 07:05:21 +0000 (GMT) (envelope-from ap@bnc.net) X-SpamCatcher-Score: 2 [X] Received: from [194.39.192.125] (account bnc-mail@mailrelay.mailomat.net HELO bnc.net) by mailomat.net (CommuniGate Pro SMTP 4.3.6) with ESMTPSA id 5550958; Fri, 23 Sep 2005 09:05:17 +0200 X-BNC-SpamCatcher-Score: 46 [XX] Received: from [194.39.192.247] (account ap HELO [194.39.192.247]) by bnc.net (CommuniGate Pro SMTP 4.3.5) with ESMTPSA id 1223321; Fri, 23 Sep 2005 09:05:16 +0200 In-Reply-To: <433370E4.8060708@mac.com> References: <63f9d26505090417183dff415e@mail.gmail.com> <431C683B.1080803@mac.com> <20050922215326.B50836@zoraida.natserv.net> <433370E4.8060708@mac.com> Mime-Version: 1.0 (Apple Message framework v734) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <531D1385-D05A-4536-B3A7-E69C674AC1B0@bnc.net> Content-Transfer-Encoding: 7bit From: Achim Patzner Date: Fri, 23 Sep 2005 09:05:11 +0200 To: Chuck Swiger X-Mailer: Apple Mail (2.734) Cc: freebsd-performance@freebsd.org Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 07:05:23 -0000 Am 23.09.2005 um 05:05 schrieb Chuck Swiger: >> I have been trying to convince the "powers that be" that SCSI >> would be much better.. but the price difference is just too >> astronomical for the capacities we need (500GB to 2 TB) >> Even 10K RPM IDE drives seem like would be a problem since they >> are mostly small in size. > > Ten 72's would be in the right ballpark, that's about $2000. Ten > of the cheapest reasonable 80GB ATA drives would be about $800. > > You could always ask: > > "How much is your data worth to your company, again?" > > You can get 146's for about $500 and even 300GB SCSI-3 drives exist. Yes. At rather incredible prices. One of our customes just bought 30 nearly identical machines (Intel E7520JR2 boards with loads of RAM and dual XEON). 11 are equipped with ICP Vortex SCSI controllers carrying a RAID level 5 set of 5 Fujitsu MAT3300NC plus a hot spare (which saved our lives already as 20 of those disks died on us already). The rest is equipped with S- ATA backplanes and ICP Vortex S-ATA RAID controllers with four disks of 160 to 500 GB each (and no hot spares). Neither are the SCSI disks faster nor more durable (we lost only one IDE disk up to now) - we compared similar RAID configurations first and the real reason thosee SCSI RAIDs were bought was the fact that you couldn't get more than four disks on an IDE Vortex. If you're asking me: Save the bucks on the hard drives and get some better disk controller instead. And no, I don't have any Adaptec shares nor did I really like ICP being bought by those manure spreaders of Adaptec (who designed some of the worlds most non-functional RAID controllers I've ever had the displeasure to work with). Achim From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:02:53 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B793A16A42A for ; Fri, 23 Sep 2005 12:02:53 +0000 (GMT) (envelope-from lists@natserv.com) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3893143D95 for ; Fri, 23 Sep 2005 12:02:50 +0000 (GMT) (envelope-from lists@natserv.com) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id 63F907DCD; Fri, 23 Sep 2005 08:02:49 -0400 (EDT) Date: Fri, 23 Sep 2005 08:02:49 -0400 (EDT) From: Francisco Reyes X-X-Sender: fran@zoraida.natserv.net To: Eric Anderson In-Reply-To: <43336294.2020403@centtech.com> Message-ID: <20050923080227.T58927@zoraida.natserv.net> References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org, mariano benedettini Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:02:53 -0000 On Thu, 22 Sep 2005, Eric Anderson wrote: > Also, if it is an NFS server, one should check the cpu times on the nfsd > processes. I've found that many times there aren't enough nfsd processes to > take the load from many clients. Increasing the number (double it) often > helps this. The max in 5.3 is 20, but you can easily change it and get > around it. What is the parameter to change the number of nfsd processes? From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:06:01 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8520816A41F for ; Fri, 23 Sep 2005 12:06:01 +0000 (GMT) (envelope-from lists@natserv.com) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id 39F1043D45 for ; Fri, 23 Sep 2005 12:06:01 +0000 (GMT) (envelope-from lists@natserv.com) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id 9FAC57DCD; Fri, 23 Sep 2005 08:06:00 -0400 (EDT) Date: Fri, 23 Sep 2005 08:06:00 -0400 (EDT) From: Francisco Reyes X-X-Sender: fran@zoraida.natserv.net To: Joseph Koshy In-Reply-To: <84dead7205092219023228cdf5@mail.gmail.com> Message-ID: <20050923080435.J58927@zoraida.natserv.net> References: <20050922214709.Q50836@zoraida.natserv.net> <84dead7205092219023228cdf5@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Performance Subject: Re: Finding what's causing I/O X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:06:01 -0000 On Fri, 23 Sep 2005, Joseph Koshy wrote: >> Is there a way to find out which program(s) are causing >> the I/O? > > ktrace(8); you can use it to trace all descendants of 'init'. Looking at the man page it's non-obvious how to use it (to me). Specially it seems one needs to indicate a pid or a command. How do I trace all programs? From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 02:14:00 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B25D716A41F for ; Fri, 23 Sep 2005 02:14:00 +0000 (GMT) (envelope-from francisco@natserv.net) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id 660CF43D45 for ; Fri, 23 Sep 2005 02:14:00 +0000 (GMT) (envelope-from francisco@natserv.net) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id B46AD7DE1; Thu, 22 Sep 2005 22:13:59 -0400 (EDT) Date: Thu, 22 Sep 2005 22:13:59 -0400 (EDT) From: Francisco X-X-Sender: fran@zoraida.natserv.net To: Chuck Swiger In-Reply-To: <431C683B.1080803@mac.com> Message-ID: <20050922215326.B50836@zoraida.natserv.net> References: <63f9d26505090417183dff415e@mail.gmail.com> <431C683B.1080803@mac.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Mailman-Approved-At: Fri, 23 Sep 2005 12:21:24 +0000 Cc: Jeff Tchang , freebsd-performance@freebsd.org Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 02:14:00 -0000 On Mon, 5 Sep 2005, Chuck Swiger wrote: > Small writes are pretty much the worst-case scenario for RAID-5, Such as mail servers? How about for a DB server which is mostly read only? > normal to see a very significant performance drop-- by up to an order of > magnitude-- from the performance of a bare drive. At which point Raid 5 starts to perform better? 6,8,10 drives? How about RAID 10 for a DB server? I have been trying to convince the "powers that be" that SCSI would be much better.. but the price difference is just too astronomical for the capacities we need (500GB to 2 TB) Even 10K RPM IDE drives seem like would be a problem since they are mostly small in size. From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:21:37 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CB2F16A41F for ; Fri, 23 Sep 2005 12:21:37 +0000 (GMT) (envelope-from shanu@shankerbalan.net) Received: from mailsrv.bangalore.corp.yahoo.com (mailsrv.bangalore.corp.yahoo.com [202.46.19.91]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7EB3043D49 for ; Fri, 23 Sep 2005 12:21:36 +0000 (GMT) (envelope-from shanu@shankerbalan.net) Received: from buffy.bangalore.corp.yahoo.com (buffy.bangalore.corp.yahoo.com [172.24.90.212]) by mailsrv.bangalore.corp.yahoo.com (8.13.4/8.13.4/pop-in) with ESMTP id j8NCKCsA067878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 23 Sep 2005 17:50:23 +0530 (IST) Received: from buffy.bangalore.corp.yahoo.com (localhost [127.0.0.1]) by buffy.bangalore.corp.yahoo.com (8.13.4/8.13.4) with ESMTP id j8NCKBvT082639 for ; Fri, 23 Sep 2005 17:50:11 +0530 (IST) (envelope-from shanu@shankerbalan.net) Received: (from shanker@localhost) by buffy.bangalore.corp.yahoo.com (8.13.4/8.13.4/Submit) id j8NCKBUN082638 for freebsd-performance@freebsd.org; Fri, 23 Sep 2005 17:50:11 +0530 (IST) (envelope-from shanu@shankerbalan.net) X-Authentication-Warning: buffy.bangalore.corp.yahoo.com: shanker set sender to shanu@shankerbalan.net using -f Date: Fri, 23 Sep 2005 17:50:11 +0530 From: Shanker Balan To: freebsd-performance@freebsd.org Message-ID: <20050923122011.GV33413@yahoo-inc.com> Mail-Followup-To: Shanker Balan , freebsd-performance@freebsd.org References: <20050922214709.Q50836@zoraida.natserv.net> <84dead7205092219023228cdf5@mail.gmail.com> <20050923080435.J58927@zoraida.natserv.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050923080435.J58927@zoraida.natserv.net> Organisation: http://www.yahoo.com/ User-Agent: Mutt/1.5.9i Subject: Re: Finding what's causing I/O X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:21:37 -0000 Hello, Francisco Reyes wrote, > On Fri, 23 Sep 2005, Joseph Koshy wrote: > > >>Is there a way to find out which program(s) are causing the I/O? > > > >ktrace(8); you can use it to trace all descendants of 'init'. > > Looking at the man page it's non-obvious how to use it (to me). > > > Specially it seems one needs to indicate a pid or a command. How do I > trace all programs? See top(1): m Toggle the display between 'cpu' and 'io' modes. -- Shanker Balan From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:22:34 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AFA7016A41F for ; Fri, 23 Sep 2005 12:22:34 +0000 (GMT) (envelope-from shanu@shankerbalan.net) Received: from mailsrv.bangalore.corp.yahoo.com (mailsrv.bangalore.corp.yahoo.com [202.46.19.91]) by mx1.FreeBSD.org (Postfix) with ESMTP id B4C6343D45 for ; Fri, 23 Sep 2005 12:22:33 +0000 (GMT) (envelope-from shanu@shankerbalan.net) Received: from buffy.bangalore.corp.yahoo.com (buffy.bangalore.corp.yahoo.com [172.24.90.212]) by mailsrv.bangalore.corp.yahoo.com (8.13.4/8.13.4/pop-in) with ESMTP id j8NCLxcF068953 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 23 Sep 2005 17:52:22 +0530 (IST) Received: from buffy.bangalore.corp.yahoo.com (localhost [127.0.0.1]) by buffy.bangalore.corp.yahoo.com (8.13.4/8.13.4) with ESMTP id j8NCLxW8082671 for ; Fri, 23 Sep 2005 17:51:59 +0530 (IST) (envelope-from shanu@shankerbalan.net) Received: (from shanker@localhost) by buffy.bangalore.corp.yahoo.com (8.13.4/8.13.4/Submit) id j8NCLxWP082670 for freebsd-performance@freebsd.org; Fri, 23 Sep 2005 17:51:59 +0530 (IST) (envelope-from shanu@shankerbalan.net) X-Authentication-Warning: buffy.bangalore.corp.yahoo.com: shanker set sender to shanu@shankerbalan.net using -f Date: Fri, 23 Sep 2005 17:51:59 +0530 From: Shanker Balan To: freebsd-performance@freebsd.org Message-ID: <20050923122159.GW33413@yahoo-inc.com> Mail-Followup-To: Shanker Balan , freebsd-performance@freebsd.org References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <20050923080227.T58927@zoraida.natserv.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050923080227.T58927@zoraida.natserv.net> Organisation: http://www.yahoo.com/ User-Agent: Mutt/1.5.9i Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:22:34 -0000 Hello, Francisco Reyes wrote, > On Thu, 22 Sep 2005, Eric Anderson wrote: > > >Also, if it is an NFS server, one should check the cpu times on the nfsd > >processes. I've found that many times there aren't enough nfsd processes > >to take the load from many clients. Increasing the number (double it) > >often helps this. The max in 5.3 is 20, but you can easily change it and > >get around it. > > What is the parameter to change the number of nfsd processes? See /etc/defaults/rc.conf: nfs_server_flags="-u -t -n 4" # Flags to nfsd (if enabled). nfsd(8) -n Specifies how many servers to create. -- Shanker Balan From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:23:20 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2312316A41F for ; Fri, 23 Sep 2005 12:23:20 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9D3E43D45 for ; Fri, 23 Sep 2005 12:23:19 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j8NCNIxH071297; Fri, 23 Sep 2005 07:23:18 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4333F3B4.600@centtech.com> Date: Fri, 23 Sep 2005 07:23:16 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco Reyes References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <20050923080227.T58927@zoraida.natserv.net> In-Reply-To: <20050923080227.T58927@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1098/Thu Sep 22 15:57:50 2005 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org, mariano benedettini Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:23:20 -0000 Francisco Reyes wrote: > On Thu, 22 Sep 2005, Eric Anderson wrote: > >> Also, if it is an NFS server, one should check the cpu times on the >> nfsd processes. I've found that many times there aren't enough nfsd >> processes to take the load from many clients. Increasing the number >> (double it) often helps this. The max in 5.3 is 20, but you can >> easily change it and get around it. > > > What is the parameter to change the number of nfsd processes? Use the -n flag to nfsd, so in /etc/rc.conf: nfs_server_flags="-u -t -n 1024" Yep, that's right, I have mine set to 1024. How I got to that number is simply by watching the cpu time on my nfsd processes. I do this: ps -auxw | grep nfsd | head -n 20 Which would reveal something like: root 410 4.2 0.0 1236 732 ?? S 13Sep05 1400:55.70 nfsd: server (nfsd) root 411 0.0 0.0 1236 732 ?? S 13Sep05 320:48.27 nfsd: server (nfsd) root 409 0.0 0.0 1352 924 ?? Is 13Sep05 0:00.47 nfsd: master (nfsd) root 412 0.0 0.0 1236 732 ?? S 13Sep05 83:29.31 nfsd: server (nfsd) root 413 0.0 0.0 1236 732 ?? S 13Sep05 39:18.94 nfsd: server (nfsd) root 414 0.0 0.0 1236 732 ?? S 13Sep05 25:16.79 nfsd: server (nfsd) root 416 0.0 0.0 1236 732 ?? S 13Sep05 19:39.53 nfsd: server (nfsd) root 417 0.0 0.0 1236 732 ?? S 13Sep05 15:23.14 nfsd: server (nfsd) root 418 0.0 0.0 1236 732 ?? S 13Sep05 13:05.99 nfsd: server (nfsd) root 419 0.0 0.0 1236 732 ?? S 13Sep05 11:05.15 nfsd: server (nfsd) root 420 0.0 0.0 1236 732 ?? S 13Sep05 9:59.39 nfsd: server (nfsd) root 421 0.0 0.0 1236 732 ?? S 13Sep05 8:39.97 nfsd: server (nfsd) root 422 0.0 0.0 1236 732 ?? S 13Sep05 7:34.38 nfsd: server (nfsd) root 423 0.0 0.0 1236 732 ?? S 13Sep05 6:56.37 nfsd: server (nfsd) root 424 0.0 0.0 1236 732 ?? S 13Sep05 6:40.99 nfsd: server (nfsd) root 425 0.0 0.0 1236 732 ?? S 13Sep05 6:08.44 nfsd: server (nfsd) root 426 0.0 0.0 1236 732 ?? S 13Sep05 5:40.81 nfsd: server (nfsd) root 427 0.0 0.0 1236 732 ?? S 13Sep05 6:34.98 nfsd: server (nfsd) root 428 0.0 0.0 1236 732 ?? S 13Sep05 4:34.36 nfsd: server (nfsd) root 429 0.0 0.0 1236 732 ?? S 13Sep05 3:17.89 nfsd: server (nfsd) That's on a very fast box, with a fiber channel connected 16 disk RAID0+1, with the fastest disks I can buy. Now, with slower disk back end, you'll probably see something more like this: root 438 0.0 0.1 1372 964 ?? Is 17Jun05 0:03.14 nfsd: master (nfsd) root 439 0.0 0.1 1224 744 ?? S 17Jun05 1010:27.86 nfsd: server (nfsd) root 440 0.0 0.1 1224 744 ?? I 17Jun05 231:17.30 nfsd: server (nfsd) root 441 0.0 0.1 1224 744 ?? I 17Jun05 125:17.14 nfsd: server (nfsd) root 442 0.0 0.1 1224 744 ?? I 17Jun05 76:30.62 nfsd: server (nfsd) root 444 0.0 0.1 1224 744 ?? I 17Jun05 94:18.99 nfsd: server (nfsd) root 445 0.0 0.1 1224 744 ?? I 17Jun05 54:48.31 nfsd: server (nfsd) root 446 0.0 0.1 1224 744 ?? I 17Jun05 63:40.74 nfsd: server (nfsd) root 447 0.0 0.1 1224 744 ?? I 17Jun05 38:29.15 nfsd: server (nfsd) root 448 0.0 0.1 1224 744 ?? I 17Jun05 46:12.29 nfsd: server (nfsd) root 449 0.0 0.1 1224 744 ?? I 17Jun05 15:24.99 nfsd: server (nfsd) root 450 0.0 0.1 1224 744 ?? I 17Jun05 13:20.40 nfsd: server (nfsd) root 451 0.0 0.1 1224 744 ?? I 17Jun05 28:09.07 nfsd: server (nfsd) root 452 0.0 0.1 1224 744 ?? I 17Jun05 11:19.46 nfsd: server (nfsd) root 453 0.0 0.1 1224 744 ?? I 17Jun05 19:34.98 nfsd: server (nfsd) root 454 0.0 0.1 1224 744 ?? I 17Jun05 9:59.46 nfsd: server (nfsd) root 455 0.0 0.1 1224 744 ?? I 17Jun05 8:36.71 nfsd: server (nfsd) root 456 0.0 0.1 1224 744 ?? I 17Jun05 7:28.79 nfsd: server (nfsd) root 457 0.0 0.1 1224 744 ?? I 17Jun05 6:18.29 nfsd: server (nfsd) root 458 0.0 0.1 1224 744 ?? I 17Jun05 5:43.77 nfsd: server (nfsd) See how much run time that 20th process has? I like to always have a couple with zero run time. So I double the number until I get there, then tune back a little. I've found that a very rough rule of thumb is: number of nfs clients * 1.8 = nfsd processes as long as you have the memory for it. You should consider about: nfsd processes * 2.5MB = memory desired for all the nfsd's Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:28:03 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 09BF216A420 for ; Fri, 23 Sep 2005 12:28:03 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9DB443D58 for ; Fri, 23 Sep 2005 12:28:00 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j8NCRxKv089983; Fri, 23 Sep 2005 07:27:59 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4333F4CE.2040109@centtech.com> Date: Fri, 23 Sep 2005 07:27:58 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco Reyes References: <20050922214709.Q50836@zoraida.natserv.net> <84dead7205092219023228cdf5@mail.gmail.com> <20050923080435.J58927@zoraida.natserv.net> In-Reply-To: <20050923080435.J58927@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1098/Thu Sep 22 15:57:50 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: FreeBSD Performance Subject: Re: Finding what's causing I/O X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:28:03 -0000 Francisco Reyes wrote: > On Fri, 23 Sep 2005, Joseph Koshy wrote: > >>> Is there a way to find out which program(s) are causing >>> the I/O? >> >> >> ktrace(8); you can use it to trace all descendants of 'init'. > > > Looking at the man page it's non-obvious how to use it (to me). > > > Specially it seems one needs to indicate a pid or a command. How do I > trace all programs? Maybe you provide the init pid, and the -i option. I played with this a bit last night, and found out I really love this tool! Here's what I did to play with it: (find pid of a bash shell running - was 1268) In another shell: ktrace -tni -ip 1268 In ktraced shell: cd / cd /tmp touch t cat t rm t In ktrace shell window: ktrace -C kdump | less That should give you a quick idea how to use it. The man page is pretty decent. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 12:32:39 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A122216A41F for ; Fri, 23 Sep 2005 12:32:39 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3665043D48 for ; Fri, 23 Sep 2005 12:32:38 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j8NCWckB071432; Fri, 23 Sep 2005 07:32:38 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4333F5E5.80709@centtech.com> Date: Fri, 23 Sep 2005 07:32:37 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco References: <63f9d26505090417183dff415e@mail.gmail.com> <431C683B.1080803@mac.com> <20050922215326.B50836@zoraida.natserv.net> In-Reply-To: <20050922215326.B50836@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1098/Thu Sep 22 15:57:50 2005 on mh2.centtech.com X-Virus-Status: Clean Cc: Jeff Tchang , freebsd-performance@freebsd.org Subject: Re: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 12:32:39 -0000 Francisco wrote: > On Mon, 5 Sep 2005, Chuck Swiger wrote: > >> Small writes are pretty much the worst-case scenario for RAID-5, > > > Such as mail servers? > How about for a DB server which is mostly read only? > >> normal to see a very significant performance drop-- by up to an order >> of magnitude-- from the performance of a bare drive. > > > > At which point Raid 5 starts to perform better? > 6,8,10 drives? > > > How about RAID 10 for a DB server? > I have been trying to convince the "powers that be" that SCSI would be > much better.. but the price difference is just too astronomical for the > capacities we need (500GB to 2 TB) > > Even 10K RPM IDE drives seem like would be a problem since they are > mostly small in size. I have a 16 disk SATA (WD Raptor 74GB drives) build with a RAID0+1 (maybe called a RAID10 by others) connected via fiber channel, and I get extremely fast data rates with it. A RAID0+1 is much faster at writes than a RAID5, and I believe faster at reads too. I've gotten 180-190MB/s from this disk, which is probably the most I could ask for on a 2gbit connection. For databases, this is a great solution (I have a MySQL db running on one - never disk bound, ever). I bought the array here: http://www.acnc.com They are FreeBSD friendly (and even support it too!) and have tools for FreeBSD as well as all the other OS's too. I've had great luck so far with them. One recommendation - get 1GB cache on the boxes - you'll see huge performance improvements for very little cost. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 19:55:14 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 227AF16A41F for ; Fri, 23 Sep 2005 19:55:14 +0000 (GMT) (envelope-from marianobe@gmx.net) Received: from postino4.prima.com.ar (postino4.prima.com.ar [200.42.0.162]) by mx1.FreeBSD.org (Postfix) with SMTP id 2B5CE43D46 for ; Fri, 23 Sep 2005 19:55:12 +0000 (GMT) (envelope-from marianobe@gmx.net) Received: (qmail 38548 invoked from network); 23 Sep 2005 19:55:08 -0000 Received: from unknown (HELO ?192.168.1.2?) (mbenedettini@200.122.3.70) by postino4.prima.com.ar with SMTP; 23 Sep 2005 19:55:08 -0000 Message-ID: <43345D9A.8040105@gmx.net> Date: Fri, 23 Sep 2005 16:55:06 -0300 From: Mariano Benedettini User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050425) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Anderson References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> In-Reply-To: <43336294.2020403@centtech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Francisco Reyes , freebsd-performance@freebsd.org Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 19:55:14 -0000 Thanks for all the replies. It's not a HD problem. On monday I'll increase the number of nfsd processes and the number of nfsiod on the client, setting both to 50, I think that the nfs performance will be much better :-) Mariano. Eric Anderson wrote: > Francisco Reyes wrote: > >> On Tue, 13 Sep 2005, mariano benedettini wrote: >> >>> 91.3% idle >> >> >> >> CPU is not the problem. :-) >> >> >>> Mem: 1599M Active, 1704M Inact, 311M Wired, 189M Cache, 112M Buf, 14M >>> Free >>> Swap: 2023M Total, 184K Used, 2023M Free >> >> >> >> Swap is not the problem. >> >> >> Do >> vmstat 10 >> >> Watch the output. >> In particular look at the first 3 columns. >> procs >> r b w >> 1 1 0 >> 0 1 0 >> 1 1 0 >> >> The left most column is CPU, the second column is disk IO. >> >> If you have a number in the "b" column and it never hits 0 you have an >> I/O problem. You HDs are not catching up. >> >> If you are using NFS and the "b" colun is not high and hits 0 some/all >> the time then the bottleneck is either the nfs connection or the nfs >> server. >> >> For example I have some servers that the "b" column would be between >> 20 and 60 for a while. I am currently working on removing some of the >> load of the machine. In my case more memory would help, but the >> computer vendor we bought the machine from has sent us the wrong >> memory 3 TIMES!! > > > Also, if it is an NFS server, one should check the cpu times on the nfsd > processes. I've found that many times there aren't enough nfsd > processes to take the load from many clients. Increasing the number > (double it) often helps this. The max in 5.3 is 20, but you can easily > change it and get around it. > > Eric > > > > From owner-freebsd-performance@FreeBSD.ORG Fri Sep 23 20:26:01 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9A1DA16A41F for ; Fri, 23 Sep 2005 20:26:01 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B40F743D76 for ; Fri, 23 Sep 2005 20:25:52 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j8NKPpPA000412; Fri, 23 Sep 2005 15:25:51 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <433464CE.4010603@centtech.com> Date: Fri, 23 Sep 2005 15:25:50 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mariano Benedettini References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <43345D9A.8040105@gmx.net> In-Reply-To: <43345D9A.8040105@gmx.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1098/Thu Sep 22 15:57:50 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: Francisco Reyes , freebsd-performance@freebsd.org Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2005 20:26:01 -0000 Mariano Benedettini wrote: > Thanks for all the replies. It's not a HD problem. > On monday I'll increase the number of nfsd processes and the number of > nfsiod on the client, setting both to 50, > I think that the nfs performance will be much better :-) 50 nfsiod's may be a bit overkill, but you should experiment to find out. You should also increase the rsize and wsize parameters on the mount options for better efficiency. Eric > Eric Anderson wrote: > >> Francisco Reyes wrote: >> >>> On Tue, 13 Sep 2005, mariano benedettini wrote: >>> >>>> 91.3% idle >>> >>> >>> >>> >>> CPU is not the problem. :-) >>> >>> >>>> Mem: 1599M Active, 1704M Inact, 311M Wired, 189M Cache, 112M Buf, >>>> 14M Free >>>> Swap: 2023M Total, 184K Used, 2023M Free >>> >>> >>> >>> >>> Swap is not the problem. >>> >>> >>> Do >>> vmstat 10 >>> >>> Watch the output. >>> In particular look at the first 3 columns. >>> procs >>> r b w >>> 1 1 0 >>> 0 1 0 >>> 1 1 0 >>> >>> The left most column is CPU, the second column is disk IO. >>> >>> If you have a number in the "b" column and it never hits 0 you have >>> an I/O problem. You HDs are not catching up. >>> >>> If you are using NFS and the "b" colun is not high and hits 0 >>> some/all the time then the bottleneck is either the nfs connection or >>> the nfs server. >>> >>> For example I have some servers that the "b" column would be between >>> 20 and 60 for a while. I am currently working on removing some of the >>> load of the machine. In my case more memory would help, but the >>> computer vendor we bought the machine from has sent us the wrong >>> memory 3 TIMES!! >> >> >> >> Also, if it is an NFS server, one should check the cpu times on the >> nfsd processes. I've found that many times there aren't enough nfsd >> processes to take the load from many clients. Increasing the number >> (double it) often helps this. The max in 5.3 is 20, but you can >> easily change it and get around it. >> >> Eric >> >> >> >> -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Sat Sep 24 00:53:48 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CE2216A421 for ; Sat, 24 Sep 2005 00:53:48 +0000 (GMT) (envelope-from lists@natserv.com) Received: from zoraida.natserv.net (p65-147.acedsl.com [66.114.65.147]) by mx1.FreeBSD.org (Postfix) with ESMTP id E599643D6E for ; Sat, 24 Sep 2005 00:53:44 +0000 (GMT) (envelope-from lists@natserv.com) Received: from localhost (localhost.natserv.net [127.0.0.1]) by zoraida.natserv.net (Postfix) with ESMTP id 297AE7DCD; Fri, 23 Sep 2005 20:53:44 -0400 (EDT) Date: Fri, 23 Sep 2005 20:53:44 -0400 (EDT) From: Francisco Reyes X-X-Sender: fran@zoraida.natserv.net To: Eric Anderson In-Reply-To: <433464CE.4010603@centtech.com> Message-ID: <20050923205327.B62667@zoraida.natserv.net> References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <43345D9A.8040105@gmx.net> <433464CE.4010603@centtech.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org, Mariano Benedettini Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2005 00:53:48 -0000 On Fri, 23 Sep 2005, Eric Anderson wrote: > You should also increase the rsize and wsize parameters on the mount options > for better efficiency. On the server? From owner-freebsd-performance@FreeBSD.ORG Sat Sep 24 03:52:50 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8407616A41F for ; Sat, 24 Sep 2005 03:52:50 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 22D8843D5C for ; Sat, 24 Sep 2005 03:52:49 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j8O3qmFK007878; Fri, 23 Sep 2005 22:52:48 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4334CD8E.5050907@centtech.com> Date: Fri, 23 Sep 2005 22:52:46 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050914 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francisco Reyes References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <43345D9A.8040105@gmx.net> <433464CE.4010603@centtech.com> <20050923205327.B62667@zoraida.natserv.net> In-Reply-To: <20050923205327.B62667@zoraida.natserv.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1099/Fri Sep 23 15:29:28 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2005 03:52:50 -0000 Francisco Reyes wrote: > On Fri, 23 Sep 2005, Eric Anderson wrote: > >> You should also increase the rsize and wsize parameters on the mount >> options for better efficiency. > > > On the server? On the client (in /etc/fstab or on the command line with -o). Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Sat Sep 24 18:16:55 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5020516A41F for ; Sat, 24 Sep 2005 18:16:55 +0000 (GMT) (envelope-from marianobe@gmx.net) Received: from postino5.prima.com.ar (postino5b.prima.com.ar [200.42.0.178]) by mx1.FreeBSD.org (Postfix) with SMTP id 5D94743D49 for ; Sat, 24 Sep 2005 18:16:53 +0000 (GMT) (envelope-from marianobe@gmx.net) Received: (qmail 63838 invoked from network); 24 Sep 2005 18:16:47 -0000 Received: from unknown (HELO ?192.168.1.2?) (mbenedettini@200.122.4.73) by postino5.prima.com.ar with SMTP; 24 Sep 2005 18:16:47 -0000 Message-ID: <4335980E.8060903@gmx.net> Date: Sat, 24 Sep 2005 15:16:46 -0300 From: Mariano Benedettini User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050425) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Anderson References: <15412.1126634818@www56.gmx.net> <20050922214142.N50836@zoraida.natserv.net> <43336294.2020403@centtech.com> <43345D9A.8040105@gmx.net> <433464CE.4010603@centtech.com> In-Reply-To: <433464CE.4010603@centtech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Francisco Reyes , freebsd-performance@freebsd.org Subject: Re: High load average mail server 5.3-RELEASE X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2005 18:16:55 -0000 I've found on discussion lists that some people also tested values near 80 or 100. I think I have CPU and RAM to start with a value of 50. The rsize and wsize values are both 32768. Thanks in advance, Mariano. Eric Anderson wrote: > Mariano Benedettini wrote: > >> Thanks for all the replies. It's not a HD problem. >> On monday I'll increase the number of nfsd processes and the number of >> nfsiod on the client, setting both to 50, >> I think that the nfs performance will be much better :-) > > > 50 nfsiod's may be a bit overkill, but you should experiment to find out. > > You should also increase the rsize and wsize parameters on the mount > options for better efficiency. > > Eric > > > >> Eric Anderson wrote: >> >>> Francisco Reyes wrote: >>> >>>> On Tue, 13 Sep 2005, mariano benedettini wrote: >>>> >>>>> 91.3% idle >>>> >>>> >>>> >>>> >>>> >>>> CPU is not the problem. :-) >>>> >>>> >>>>> Mem: 1599M Active, 1704M Inact, 311M Wired, 189M Cache, 112M Buf, >>>>> 14M Free >>>>> Swap: 2023M Total, 184K Used, 2023M Free >>>> >>>> >>>> >>>> >>>> >>>> Swap is not the problem. >>>> >>>> >>>> Do >>>> vmstat 10 >>>> >>>> Watch the output. >>>> In particular look at the first 3 columns. >>>> procs >>>> r b w >>>> 1 1 0 >>>> 0 1 0 >>>> 1 1 0 >>>> >>>> The left most column is CPU, the second column is disk IO. >>>> >>>> If you have a number in the "b" column and it never hits 0 you have >>>> an I/O problem. You HDs are not catching up. >>>> >>>> If you are using NFS and the "b" colun is not high and hits 0 >>>> some/all the time then the bottleneck is either the nfs connection >>>> or the nfs server. >>>> >>>> For example I have some servers that the "b" column would be between >>>> 20 and 60 for a while. I am currently working on removing some of >>>> the load of the machine. In my case more memory would help, but the >>>> computer vendor we bought the machine from has sent us the wrong >>>> memory 3 TIMES!! >>> >>> >>> >>> >>> Also, if it is an NFS server, one should check the cpu times on the >>> nfsd processes. I've found that many times there aren't enough nfsd >>> processes to take the load from many clients. Increasing the number >>> (double it) often helps this. The max in 5.3 is 20, but you can >>> easily change it and get around it. >>> >>> Eric >>> >>> >>> >>> > >