From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 06:24:32 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5BDB16A4CE for ; Mon, 22 Nov 2004 06:24:32 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.207]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70DB443D58 for ; Mon, 22 Nov 2004 06:24:32 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id r35so205671rna for ; Sun, 21 Nov 2004 22:24:32 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=cbqeuQKndDVXg6E4JaaOVR7yhm3XJGLw50pH0oDZZ+cXIT5gVHF4H00DB+xrR8Eabco1mOqek/nkWyUC4iZSPnhB3qmV6uBedoRo5L1CP2t1ajtYuaxk6oSEw757auX6sPIWKhIGA1TpEp7E5+azL0Zq67Uj1IdwW5tZt4Fvnho= Received: by 10.38.13.70 with SMTP id 70mr708511rnm; Sun, 21 Nov 2004 22:24:31 -0800 (PST) Received: by 10.38.151.41 with HTTP; Sun, 21 Nov 2004 22:24:31 -0800 (PST) Message-ID: <18f60194041121222468ea996e@mail.gmail.com> Date: Sun, 21 Nov 2004 23:24:31 -0700 From: Aaron Glenn To: Thomas Graham In-Reply-To: <1509.147.8.17.110.1101094111.squirrel@sml.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> <20041119152642.GA5143@octanews.net> <18f60194041119104939fe9156@mail.gmail.com> <1509.147.8.17.110.1101094111.squirrel@sml.dyndns.org> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 06:24:32 -0000 On Mon, 22 Nov 2004 11:28:31 +0800 (HKT), Thomas Graham wrote: > is it indicate the network cable problem ? Can't remember if I posted this to the list, but I ftp'ed /dev/zero from another box on another subnet two switches away at 93Mbps sustained (for three minutes). The cable, network card, network driver, and network I/O is fine. I'm positive I'm hitting the limitations of Apache or thttpd - but I don't know why. Any trace'ing tips, perhaps? Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 14:43:15 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24D7A16A4CE for ; Mon, 22 Nov 2004 14:43:15 +0000 (GMT) Received: from web41208.mail.yahoo.com (web41208.mail.yahoo.com [66.218.93.41]) by mx1.FreeBSD.org (Postfix) with SMTP id E7C7143D2D for ; Mon, 22 Nov 2004 14:43:14 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 75063 invoked by uid 60001); 22 Nov 2004 14:43:14 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=nn0mBf7xoEeIQlIILt7qxWDyAbDKgcqrzTQbrOEwxZXvcpVIzRAmd/SYrezAGw7k9LU+SxY2E6igaCZf6mW/ppYYGT+M5hlS7bI0U1Xcc52OivaPwrLka9C/HoctviynLxUH3OWpGgBWhEoz2Fu1PRvSOMkBO9dslJ8BtF2ac3o= ; Message-ID: <20041122144314.75061.qmail@web41208.mail.yahoo.com> Received: from [83.129.242.96] by web41208.mail.yahoo.com via HTTP; Mon, 22 Nov 2004 06:43:14 PST Date: Mon, 22 Nov 2004 06:43:14 -0800 (PST) From: Arne "Wörner" To: freebsd-performance@freebsd.org In-Reply-To: <18f60194041121222468ea996e@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 14:43:15 -0000 > Can't remember if I posted this to the list, but I ftp'ed /dev/zero > from another box on another subnet two switches away at 93Mbps > sustained (for three minutes). The cable, network card, network > driver, and network I/O is fine. I'm positive I'm hitting the > limitations of Apache or thttpd - but I don't know why. > > Any trace'ing tips, perhaps? > Hmm. You could try to run thttpd without the sendfile-syscall (which is suspicious to me, because I did not know it before): > cd /usr/ports/www/thttpd > make configure > vi work/thttpd-2.25b/config.h [remove the line, that contains USE_SENDFILE] > make > nm work/thttpd-2.25b/thttpd.o | grep sendfile | wc -l [the result should be "0"] Since thttpd tries to throttle for some reason (I thought the OS would slow down some send requests by inserting packets of other send requests; or isn't there time sharing in the ethernet drivers?), it might be interesting (a) to increase the values for max_bytes in lines 1740, or (b) to use just one single HTTP connection during the test phase (in that case throttling should be disabled)... -Arne __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 16:45:56 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7B2EA16A4CE for ; Mon, 22 Nov 2004 16:45:56 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24A9643D1D for ; Mon, 22 Nov 2004 16:45:56 +0000 (GMT) (envelope-from fehwalker@gmail.com) Received: by wproxy.gmail.com with SMTP id 36so206902wri for ; Mon, 22 Nov 2004 08:45:55 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=onH9Fasata9C77TRYmA947VXY/6hcjLbZbcM5e1VbPU23VYEV7SRtWwuUz+OU1wXmMU9wUItCePOehKvSfFFMrYDVq3exuHlhW4Mszed4fKmY9tKvDUEWpFpjr9bJ9Rq1phSQmrBv9zfaGMr2ehL7LYX1p6fSp05aH4XD2t3+sU= Received: by 10.54.20.74 with SMTP id 74mr813354wrt; Mon, 22 Nov 2004 08:45:55 -0800 (PST) Received: by 10.54.19.65 with HTTP; Mon, 22 Nov 2004 08:45:55 -0800 (PST) Message-ID: <35de0c30041122084519ef3859@mail.gmail.com> Date: Mon, 22 Nov 2004 11:45:55 -0500 From: Bryan Fullerton To: freebsd-performance@freebsd.org In-Reply-To: <18f60194041121222468ea996e@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> <20041119152642.GA5143@octanews.net> <18f60194041119104939fe9156@mail.gmail.com> <1509.147.8.17.110.1101094111.squirrel@sml.dyndns.org> <18f60194041121222468ea996e@mail.gmail.com> Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Bryan Fullerton List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 16:45:56 -0000 On Sun, 21 Nov 2004 23:24:31 -0700, Aaron Glenn wrote: > Can't remember if I posted this to the list, but I ftp'ed /dev/zero > from another box on another subnet two switches away at 93Mbps > sustained (for three minutes). The cable, network card, network > driver, and network I/O is fine. I'm positive I'm hitting the > limitations of Apache or thttpd - but I don't know why. You could try asking the thttpd list. Bryan From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 22:34:11 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A4BB16A4CE for ; Mon, 22 Nov 2004 22:34:11 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id A37E043D3F for ; Mon, 22 Nov 2004 22:34:10 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so289026rne for ; Mon, 22 Nov 2004 14:34:10 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=ZWBZvn5zxRQiFXMEucMh/vCbeXNGqHzn37cqjVppiyX6wIivSKn6hobXZQX2ddiJW84V1BDC44pKQk8mYpVjz5upxrYFO5mEEUn3tzb1RU9S+DdW1UKdV5Fo6L+UGDxLFQnl1D7ulgNcgNMNKaYyaA1xT5q34ZWi95ibdx5QftA= Received: by 10.38.72.14 with SMTP id u14mr223485rna; Mon, 22 Nov 2004 14:34:01 -0800 (PST) Received: by 10.38.151.41 with HTTP; Mon, 22 Nov 2004 14:33:34 -0800 (PST) Message-ID: <18f6019404112214334457cd22@mail.gmail.com> Date: Mon, 22 Nov 2004 14:33:34 -0800 From: Aaron Glenn To: freebsd-performance@freebsd.org In-Reply-To: <35de0c30041122084519ef3859@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> <20041119152642.GA5143@octanews.net> <18f60194041119104939fe9156@mail.gmail.com> <1509.147.8.17.110.1101094111.squirrel@sml.dyndns.org> <18f60194041121222468ea996e@mail.gmail.com> <35de0c30041122084519ef3859@mail.gmail.com> Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 22:34:11 -0000 On Mon, 22 Nov 2004 11:45:55 -0500, Bryan Fullerton wrote: > > You could try asking the thttpd list. Of course. But why does iostat report 1.3MB/s on a completely idle system? Is something eating my disk I/O? If so, how do I find out what, precisely, is? Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 22:34:15 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08EAE16A4D1 for ; Mon, 22 Nov 2004 22:34:15 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id A87AA43D5C for ; Mon, 22 Nov 2004 22:34:14 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so289044rne for ; Mon, 22 Nov 2004 14:34:14 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=KOSYkNSQ92/3k+BcXyT/C5Xgge68VYo4Oak0H8J6XMz7mEUwVqxgLT9/0PTITHva/aopNsF2FKlipLG2MzaRIDRj6a9S8grnUG7NLG7G7OjJILSrYGhCA0swVHPPyEOr2EKIxQcEVD4249GpILNJcAOVDcKRgrO82ktHhxrYoe4= Received: by 10.38.72.14 with SMTP id u14mr223531rna; Mon, 22 Nov 2004 14:34:05 -0800 (PST) Received: by 10.38.151.41 with HTTP; Mon, 22 Nov 2004 14:33:34 -0800 (PST) Message-ID: <18f6019404112214334457cd22@mail.gmail.com> Date: Mon, 22 Nov 2004 14:33:34 -0800 From: Aaron Glenn To: freebsd-performance@freebsd.org In-Reply-To: <35de0c30041122084519ef3859@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> <20041119152642.GA5143@octanews.net> <18f60194041119104939fe9156@mail.gmail.com> <1509.147.8.17.110.1101094111.squirrel@sml.dyndns.org> <18f60194041121222468ea996e@mail.gmail.com> <35de0c30041122084519ef3859@mail.gmail.com> Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 22:34:15 -0000 On Mon, 22 Nov 2004 11:45:55 -0500, Bryan Fullerton wrote: > > You could try asking the thttpd list. Of course. But why does iostat report 1.3MB/s on a completely idle system? Is something eating my disk I/O? If so, how do I find out what, precisely, is? Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 22:38:24 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 22E0716A4CE for ; Mon, 22 Nov 2004 22:38:24 +0000 (GMT) Received: from web14123.mail.yahoo.com (web14123.mail.yahoo.com [66.163.171.114]) by mx1.FreeBSD.org (Postfix) with SMTP id D309A43D3F for ; Mon, 22 Nov 2004 22:38:23 +0000 (GMT) (envelope-from cguttesen@yahoo.dk) Received: (qmail 15009 invoked by uid 60001); 22 Nov 2004 22:38:23 -0000 Message-ID: <20041122223823.15007.qmail@web14123.mail.yahoo.com> Received: from [194.248.174.50] by web14123.mail.yahoo.com via HTTP; Mon, 22 Nov 2004 23:38:23 CET Date: Mon, 22 Nov 2004 23:38:23 +0100 (CET) From: Claus Guttesen To: Aaron Glenn , freebsd-performance@freebsd.org In-Reply-To: <18f6019404112214334457cd22@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 22:38:24 -0000 > Of course. But why does iostat report 1.3MB/s on a > completely idle > system? Is something eating my disk I/O? If so, how > do I find out > what, precisely, is? Try top and press 'm', this will show you io-stats. Claus From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 22:45:31 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F45F16A4CE for ; Mon, 22 Nov 2004 22:45:31 +0000 (GMT) Received: from web41213.mail.yahoo.com (web41213.mail.yahoo.com [66.218.93.46]) by mx1.FreeBSD.org (Postfix) with SMTP id 429E343D58 for ; Mon, 22 Nov 2004 22:45:31 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 65942 invoked by uid 60001); 22 Nov 2004 22:45:31 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=gxBLNGFtAmITTjZqdbANdAq0/LrGHKfUZNoq9aCsaoWNpvd3pIk26r2K2MeUMH+k8tY2dK7qsy5AX0BsG8mOKoncwQ7WhXA/y2R9zoeroUhgo5OUsrlYHJq0/BqLMo0IdRXFV2mOQMHj+1TttYEZzN0wRY6pSHhUDls9oRV71+M= ; Message-ID: <20041122224530.65940.qmail@web41213.mail.yahoo.com> Received: from [83.129.242.96] by web41213.mail.yahoo.com via HTTP; Mon, 22 Nov 2004 14:45:30 PST Date: Mon, 22 Nov 2004 14:45:30 -0800 (PST) From: Arne "Wörner" To: Aaron Glenn In-Reply-To: <20041122223823.15007.qmail@web14123.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 22:45:31 -0000 > > Of course. But why does iostat report 1.3MB/s on a > > completely idle system? Is something eating my disk I/O? If so, how > > do I find out what, precisely, is? > Hmm... Since there should not be so much I/O on an idle system, I would like to ask: Did you use - iostat or - iostat 2 or so? If I use "iostat" I just see the average. If I use "iostat 2" I see first the average and then the iostat of the last 2 seconds: > iostat 2 tty ad0 ad1 da0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 2 4342 28.04 1 0.04 36.95 4 0.15 0.01 0 0.00 18 73 6 2 0 12 115 0.00 0 0.00 44.67 7 0.33 0.00 0 0.00 20 74 3 3 0 5 32 16.00 2 0.04 41.26 8 0.31 0.00 0 0.00 21 71 5 3 0 0 38 7.00 1 0.01 50.00 7 0.37 0.00 0 0.00 24 69 5 2 0 ^C -Arne __________________________________ Do you Yahoo!? The all-new My Yahoo! - Get yours free! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Mon Nov 22 22:55:17 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C74716A4CE for ; Mon, 22 Nov 2004 22:55:17 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19D8943D2F for ; Mon, 22 Nov 2004 22:55:17 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so290491rne for ; Mon, 22 Nov 2004 14:55:11 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=XHil3RyDv18Twe2QK2wqfuRmbh3bK8z4ivHpsVTL6rSD88exUTRgehpTXlX2tnuN35Art3S/k9NfcKvaDEWxxHbf36PUkZQ2RcTFFOXkLvHga0fnt96BgTo5qClfKeagznHsO4cWT5ldIMCXlR8v8jWw1mBRKkNaTPrxcVOUi9A= Received: by 10.38.102.74 with SMTP id z74mr1085585rnb; Mon, 22 Nov 2004 14:55:10 -0800 (PST) Received: by 10.38.151.41 with HTTP; Mon, 22 Nov 2004 14:55:10 -0800 (PST) Message-ID: <18f60194041122145543bda4a3@mail.gmail.com> Date: Mon, 22 Nov 2004 14:55:10 -0800 From: Aaron Glenn To: =?ISO-8859-1?Q?Arne_W=F6rner?= In-Reply-To: <20041122224530.65940.qmail@web41213.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable References: <20041122223823.15007.qmail@web14123.mail.yahoo.com> <20041122224530.65940.qmail@web41213.mail.yahoo.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2004 22:55:17 -0000 On Mon, 22 Nov 2004 14:45:30 -0800 (PST), Arne W=F6rner wrote: >=20 > If I use "iostat" I just see the average. If I use "iostat 2" I see first= the > average and then the iostat of the last 2 seconds: I read the iostat man page *after* sending to the list. Next time I'll read the relevant man page *before* sending (-: Hopefully the thttpd list has some ideas. Disabling the USE_SENDFILE macro in config.h actually decreased thoroughput, and filled my /var/log/messages with: Nov 22 15:23:20 d thttpd[12304]: mmc panic - freeing all unreferenced maps Nov 22 15:23:20 d thttpd[12304]: mmap - Cannot allocate memory Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Tue Nov 23 06:06:56 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A99516A4CE for ; Tue, 23 Nov 2004 06:06:56 +0000 (GMT) Received: from daemon.kr.FreeBSD.org (www.kr.freebsd.org [211.115.73.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6CE2843D5D for ; Tue, 23 Nov 2004 06:06:55 +0000 (GMT) (envelope-from cjh@kr.FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) by daemon.kr.FreeBSD.org (Postfix) with ESMTP id BA98C1A6F1; Tue, 23 Nov 2004 15:06:50 +0900 (KST) Received: from daemon.kr.FreeBSD.org ([127.0.0.1]) by localhost (daemon.kr.freebsd.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 86664-14; Tue, 23 Nov 2004 15:06:46 +0900 (KST) Received: from localhost (gradius [211.44.63.164]) by daemon.kr.FreeBSD.org (Postfix) with ESMTP id 6E8EF1A6E4; Tue, 23 Nov 2004 15:06:45 +0900 (KST) Date: Tue, 23 Nov 2004 15:06:47 +0900 (KST) Message-Id: <20041123.150647.41628290.cjh@kr.FreeBSD.org> To: aaron.glenn@gmail.com From: CHOI Junho In-Reply-To: <18f60194041122145543bda4a3@mail.gmail.com> References: <20041122223823.15007.qmail@web14123.mail.yahoo.com> <20041122224530.65940.qmail@web41213.mail.yahoo.com> <18f60194041122145543bda4a3@mail.gmail.com> Organization: Korea FreeBSD Users Group X-URL: http://www.kr.FreeBSD.org/~cjh X-Mailer: Mew version 4.1.50 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: by amavisd-new at kr.FreeBSD.org cc: freebsd-performance@freebsd.org cc: arne_woerner@yahoo.com Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 06:06:56 -0000 From: Aaron Glenn Subject: Re: I've ran out of ideas Date: Mon, 22 Nov 2004 14:55:10 -0800 > On Mon, 22 Nov 2004 14:45:30 -0800 (PST), Arne W=F6rner > Hopefully the thttpd list has some ideas. Disabling the USE_SENDFILE > macro in config.h actually decreased thoroughput, and filled my > /var/log/messages with: > = > Nov 22 15:23:20 d thttpd[12304]: mmc panic - freeing all unreferenced= maps > Nov 22 15:23:20 d thttpd[12304]: mmap - Cannot allocate memory > = > Regards, > aaron.glenn This message is logged then thttpd's memory management routime('mmc') failed to allocate new memory for incoming request. Maybe your files is too many and too big to fit in your RAM. thttpd's memory allocation try to mmap() first in case of the requested file. USE_SENDFILE patch changes this behavior to use sendfile() first, so when you use sendfile(), this message will not occur. You need to check how much memory thttpd use, If you have too many file to serve at a certain period. thttpd's mmc use timeout(10 sec?) and reference count to flush mmap() allocation. If you have 1G memory and total size of requested file is over 1G in 10 seconds, this message can occur. You can check this with 'top', seeing RES field for thttpd when it's busy use. If mmc failed, that request denied. mmc tries to dealloc all mmap'ed memory(first message) and try to allocate new one(second message). If sendfile() failed(usually for allocation failure to kernel's sendfile buffer), it try to wait until allocate buffer even in nonblocking mode(SFALLOC state when you see 'top'), so it delays further processing, finally performance decreased. If your thttpd memory usage is relatively low when you see RES field of top's output(non-sendfile mode), you need to increase two values in config.h: #define DESIRED_MAX_MAPPED_FILES 1000 #define DESIRED_MAX_MAPPED_BYTES 1000000000 Increase two values to fit your case. This is mmc's limit of mmap() allocation. -- CHOI Junho cjh@[kr.]FreeBSD.org Key fingerprint =3D 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 From owner-freebsd-performance@FreeBSD.ORG Wed Nov 24 13:26:42 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 82EAA16A4CE for ; Wed, 24 Nov 2004 13:26:42 +0000 (GMT) Received: from mx2.mail.ru (mx2.mail.ru [194.67.23.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0515643D58 for ; Wed, 24 Nov 2004 13:26:42 +0000 (GMT) (envelope-from my-subs@mail.ru) Received: from [212.143.159.12] (port=62834 helo=ant.zoo) by mx2.mail.ru with smtp id 1CWxAN-0001Uw-00 for freebsd-performance@freebsd.org; Wed, 24 Nov 2004 16:26:39 +0300 Date: Wed, 24 Nov 2004 15:26:34 +0200 From: Alexander Portnoy To: freebsd-performance@freebsd.org Message-Id: <20041124152634.713b9e87.my-subs@mail.ru> In-Reply-To: References: X-Mailer: Sylpheed version 0.9.99-gtk2-20041024 (GTK+ 2.4.13; i386-portbld-freebsd5.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam: Not detected Subject: Re: openssl speed performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2004 13:26:42 -0000 On Fri, 01 Oct 2004 10:35:12 +0200 "seb.arana@aj-form.com" wrote: > hi everyone ! > i've read some post here and i'm very impressed with the openssl speed > results of one member here (called Alexander Portnoy). > > i've done a "openssl speed" test (ie: localhost# openssl speed ) and > here are the results : > >................... >................... > > the sources of kernel are RELENG_5_2 (p10 applied) > > someone know why my computer (p4 2.4Ghz, 1Go, 120Go SATA RAID1 on > promise chipset, MB asus p4c800, FreeBSD 5.2.1R patched) cant perf any > more, or better than the Alexander results ? > > Thanks for ideas :) > G'Day ! > Seb. The difference in dsa/rsa tests is really significant. I suggest You to run the test from FreeBSD-5.3-FixIt-CD: ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/ISO-IMAGES/5.3/5.3-RELEASE-i386-disc2.iso This will tell You where to search the source of the problem: hardware or software misconfiguration. There are results of my system (desktop system "Compaq Evo D310": Pentium-IV 2400MHz 512KB L2 cache 533MHz FSB w/o HTT, 256MB DDR266 RAM, Intel-845 chipset based MB) booted from the CD: OpenSSL 0.9.7d 17 Mar 2004 built on: Thu Nov 4 22:14:58 UTC 2004 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx) compiler: cc available timing options: USE_TOD HZ=128 [sysconf value] timing function used: getrusage The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes md2 1362.64k 3041.19k 4259.50k 4860.64k 4996.67k mdc2 3898.81k 4962.63k 5314.68k 5405.95k 5467.41k md4 8146.22k 29710.53k 85335.46k 161228.01k 215170.06k md5 6843.76k 22698.38k 60892.60k 102615.32k 135737.57k hmac(md5) 8732.93k 27387.84k 69306.50k 107352.48k 136973.98k sha1 6591.18k 20041.62k 47024.28k 71131.32k 83852.90k rmd160 5325.49k 15179.73k 32200.50k 44719.27k 50435.42k rc4 79043.08k 86925.21k 89144.49k 89913.07k 89584.36k des cbc 40417.84k 40708.89k 40847.24k 40822.23k 40855.28k des ede3 15917.71k 15924.98k 16021.33k 15990.69k 15991.21k idea cbc 0.00 0.00 0.00 0.00 0.00 rc2 cbc 11591.88k 11980.88k 12657.38k 12522.76k 12882.24k rc5-32/12 cbc 82272.88k 84006.21k 84055.62k 84241.06k 84521.27k blowfish cbc 70684.67k 72304.03k 72528.48k 73193.48k 72030.05k cast cbc 55150.82k 59284.21k 60207.17k 59226.29k 59215.20k aes-128 cbc 55924.89k 52807.86k 53857.58k 54445.88k 54388.11k aes-192 cbc 48574.60k 46551.92k 46923.55k 47635.50k 47667.10k aes-256 cbc 43947.24k 42262.12k 42969.64k 43403.60k 43262.21k sign verify sign/s verify/s rsa 512 bits 0.0010s 0.0001s 954.7 10151.5 rsa 1024 bits 0.0050s 0.0003s 200.7 3660.4 rsa 2048 bits 0.0297s 0.0009s 33.7 1129.8 rsa 4096 bits 0.1987s 0.0031s 5.0 322.7 sign verify sign/s verify/s dsa 512 bits 0.0008s 0.0010s 1186.1 977.6 dsa 1024 bits 0.0025s 0.0031s 400.4 325.6 dsa 2048 bits 0.0083s 0.0100s 120.2 99.8 From owner-freebsd-performance@FreeBSD.ORG Wed Nov 24 13:43:50 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B594D16A4CE for ; Wed, 24 Nov 2004 13:43:50 +0000 (GMT) Received: from mx2.mail.ru (mx2.mail.ru [194.67.23.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 38BA143D1F for ; Wed, 24 Nov 2004 13:43:50 +0000 (GMT) (envelope-from my-subs@mail.ru) Received: from [212.143.159.12] (port=56884 helo=ant.zoo) by mx2.mail.ru with smtp id 1CWxQx-0008g0-00 for freebsd-performance@freebsd.org; Wed, 24 Nov 2004 16:43:47 +0300 Date: Wed, 24 Nov 2004 15:43:41 +0200 From: Alexander Portnoy To: freebsd-performance@freebsd.org Message-Id: <20041124154341.56a789f6.my-subs@mail.ru> In-Reply-To: References: X-Mailer: Sylpheed version 0.9.99-gtk2-20041024 (GTK+ 2.4.13; i386-portbld-freebsd5.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam: Not detected Subject: Re: openssl speed performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2004 13:43:50 -0000 Compare Your results with the results on the Athlon based notebook "HP Pavilion ze4523ea" (AMD Athlon-XP Mobile 2400+ 1.8GHz 512KB L2 cache, 256MB DDR266 RAM, ALi chipset based MB, Seagate ST93012A HDD) running FreeBSD 5.3-RELEASE-p1: OpenSSL 0.9.7d 17 Mar 2004 built on: Sun Nov 21 08:04:55 IST 2004 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx) compiler: cc available timing options: USE_TOD HZ=128 [sysconf value] timing function used: getrusage The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes md2 1461.94k 3121.67k 4379.01k 4851.33k 5012.79k mdc2 4341.62k 5241.45k 5506.69k 5579.09k 5609.63k md4 12511.55k 43928.95k 126317.65k 241420.53k 329338.90k md5 9830.87k 32750.66k 87214.71k 149973.51k 190908.51k hmac(md5) 12351.54k 39313.73k 99331.66k 160855.86k 195563.49k sha1 10219.65k 30966.68k 70337.86k 103062.14k 119734.11k rmd160 7233.74k 21121.28k 46230.28k 65910.78k 74993.56k rc4 142942.15k 160620.87k 165003.87k 166941.84k 167535.53k des cbc 46259.23k 48243.77k 48732.54k 48924.13k 48979.40k des ede3 17330.48k 17607.44k 17713.47k 17740.34k 17747.83k idea cbc 25333.27k 26590.33k 26929.46k 27016.77k 27042.04k rc2 cbc 15750.27k 16400.24k 16570.88k 16617.35k 16673.05k rc5-32/12 cbc 126410.07k 142507.29k 147297.94k 148236.66k 148743.07k blowfish cbc 69932.54k 75323.41k 76268.81k 76740.34k 76872.74k cast cbc 65229.65k 69262.32k 70198.63k 70588.47k 70699.30k aes-128 cbc 38300.38k 39192.03k 39692.53k 39922.20k 39851.56k aes-192 cbc 33457.23k 34129.29k 34598.41k 34603.94k 34628.37k aes-256 cbc 29625.08k 30190.37k 30486.14k 30560.85k 30580.12k sign verify sign/s verify/s rsa 512 bits 0.0007s 0.0001s 1349.3 15545.4 rsa 1024 bits 0.0033s 0.0002s 300.5 5910.7 rsa 2048 bits 0.0186s 0.0005s 53.7 1877.0 rsa 4096 bits 0.1206s 0.0018s 8.3 545.2 sign verify sign/s verify/s dsa 512 bits 0.0006s 0.0007s 1733.9 1452.1 dsa 1024 bits 0.0016s 0.0019s 643.8 527.4 dsa 2048 bits 0.0050s 0.0060s 198.4 165.8 There are my configuration files: ============== /boot/loader.conf: libiconv_load="YES" cd9660_load="YES" # ISO 9660 filesystem cd9660_iconv_load="YES" linprocfs_load="YES" # Linux compatibility process filesystem msdosfs_load="YES" # FAT-12/16/32 msdosfs_iconv_load="YES" nfs_load="YES" # NFS nfs_client_load="YES" nfs_server_load="YES" ntfs_load="YES" # NTFS ntfs_iconv_load="YES" unionfs_load="YES" # Union filesystem linux_load="YES" # Linux emulation if_ath_load="YES" if_sis_load="YES" # Silicon Integrated Systems SiS 900/7016 snd_pcm_load="YES" snd_t4dwave_load="YES" # t4dwave usb_load="YES" # USB subsystem agp_load="YES" # agp module random_load="YES" # Random device libmchain_load="YES" cardbus_load="YES" cbb_load="YES" mem_load="YES" io_load="YES" hw.ata.atapi_dma=1 =========== Kernel config: machine i386 cpu I686_CPU # aka Pentium Pro(tm) options CPU_ATHLON_SSE_HACK options CPU_ENABLE_SSE device isa device pci device npx device pmtimer # Adjust system timer at wakeup time ident ze4523ea-53-modular options INCLUDE_CONFIG_FILE # Include this file in kernel options SCHED_4BSD options COMPAT_43 options COMPAT_FREEBSD4 options SYSVSHM options SYSVSEM options SYSVMSG options DDB options KDB options KTRACE #kernel tracing options INET #Internet communications protocols options IPSEC #IP security options IPSEC_ESP #IP security (crypto; define w/ IPSEC) options NETSMBCRYPTO #encrypted password support for SMB device ether #Generic Ethernet device loop #Network loopback device device bpf #Berkeley packet filter options PPP_BSDCOMP #PPP BSD-compress support options PPP_DEFLATE #PPP zlib/deflate/gzip support options PPP_FILTER #enable bpf filtering (needs bpf) options IPFIREWALL_VERBOSE #enable logging to syslogd(8) options IPFIREWALL_VERBOSE_LIMIT=100 #limit verbosity options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default options IPDIVERT #divert sockets options IPFILTER #ipfilter support options IPFILTER_LOG #ipfilter logging options IPSTEALTH #support for stealth forwarding options TCP_DROP_SYNFIN #drop TCP packets with SYN+FIN options FFS #Fast filesystem options SOFTUPDATES options UFS_EXTATTR options UFS_EXTATTR_AUTOSTART options UFS_ACL options UFS_DIRHASH options QUOTA #enable disk quotas options _KPOSIX_PRIORITY_SCHEDULING device scbus #base SCSI code device da #SCSI direct access devices (aka disks) device cd #SCSI CD-ROMs device pass #CAM passthrough driver device pty #Pseudo ttys device atkbdc device atkbd options KBD_INSTALL_CDEV # install a CDEV entry in /dev device psm device vga device sc options SC_HISTORY_SIZE=2000 # number of history buffer lines options SC_PIXEL_MODE # add support for the raster text mode options SC_TWOBUTTON_MOUSE device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives device atapicam # emulate ATAPI devices as SCSI ditto via CAM # needs CAM to be present (scbus & pass) options ATA_STATIC_ID device sio options PANIC_REBOOT_WAIT_TIME=(-1) ============= /etc/make.conf: CPUTYPE?=athlon-xp CFLAGS= -O -pipe COPTFLAGS= -O -pipe NO_BLUETOOTH= true # do not build Bluetooth related stuff NO_I4B= true # do not build isdn4bsd package NO_VINUM= true # do not build Vinum utilities NOATM= true # do not build ATM related programs and libraries NOGAMES= true # do not build games (games/ subdir) NOINET6= true # do not build IPv6 related programs and libraries NOPROFILE= true # Avoid compiling profiled libraries MAKE_IDEA= YES # IDEA (128 bit symmetric encryption) COMPAT4X= yes From owner-freebsd-performance@FreeBSD.ORG Wed Nov 24 19:17:07 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 23E7616A4CE; Wed, 24 Nov 2004 19:17:07 +0000 (GMT) Received: from otter3.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D64643D4C; Wed, 24 Nov 2004 19:17:06 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by otter3.centtech.com (8.12.3/8.12.3) with ESMTP id iAOJH3OJ066087; Wed, 24 Nov 2004 13:17:04 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <41A4DE2A.6090200@centtech.com> Date: Wed, 24 Nov 2004 13:16:58 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.3) Gecko/20041110 X-Accept-Language: en-us, en MIME-Version: 1.0 To: akhthar@carmatec.com References: <200411250002.37764.akhthar@carmatec.com> In-Reply-To: <200411250002.37764.akhthar@carmatec.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org Subject: Re: Mbuf errors X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2004 19:17:07 -0000 Akhthar Parvez. K wrote: > Hi All, > > Mysql service is going down continously in my system due to lack of memory > space. > > I checked the messages log and found the following error message. > > All mbuf clusters exhausted, please see tuning(7). > > I have no idea about mbuf cluster. Can anyone please help me to fix the issue. > > I hope the information below will help you. [..snip..] > Following is the output of netstat -m > > 3797/14672/26624 mbufs in use (current/peak/max): > 3791 mbufs allocated to data > 2 mbufs allocated to fragment reassembly queue headers > 4 mbufs allocated to socket names and addresses > 3704/6656/6656 mbuf clusters in use (current/peak/max) > 16980 Kbytes allocated to network (85% of mb_map in use) > 106522 requests for memory denied > 1545 requests for memory delayed > 0 calls to protocol drain routines > > Please let me know if you need more info regarding this. > Thank you in advance. This isn't really security related, more performance related, so I'm moving it to that list (freebsd-performance@). The answer to your question is right in your email! In the tuning man page, it says: kern.ipc.nmbclusters may be adjusted to increase the number of network mbufs the system is willing to allocate. Each cluster represents approx- imately 2K of memory, so a value of 1024 represents 2M of kernel memory reserved for network buffers. You can do a simple calculation to figure out how many you need. If you have a web server which maxes out at 1000 simultaneous connections, and each connection eats a 16K receive and 16K send buffer, you need approximately 32MB worth of network buffers to deal with it. A good rule of thumb is to multiply by 2, so 32MBx2 = 64MB/2K = 32768. So for this case you would want to set kern.ipc.nmbclusters to 32768. We recommend values between 1024 and 4096 for machines with mod- erates amount of memory, and between 4096 and 32768 for machines with greater amounts of memory. Under no circumstances should you specify an arbitrarily high value for this parameter, it could lead to a boot-time crash. The -m option to netstat(1) may be used to observe network clus- ter use. Older versions of FreeBSD do not have this tunable and require that the kernel config(8) option NMBCLUSTERS be set instead. More and more programs are using the sendfile(2) system call to transmit files over the network. The kern.ipc.nsfbufs sysctl controls the number of file system buffers sendfile(2) is allowed to use to perform its work. This parameter nominally scales with kern.maxusers so you should not need to modify this parameter except under extreme circumstances. See the TUNING section in the sendfile(2) manual page for details. So basically you need to use sysctl to adjust (increase) the kern.ipc.nmbclusters number. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology When in doubt, mumble; when in trouble, delegate; when in charge, ponder ------------------------------------------------------------------------