From owner-freebsd-performance@FreeBSD.ORG Mon Aug 22 07:11:42 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F46316A41F for ; Mon, 22 Aug 2005 07:11:42 +0000 (GMT) (envelope-from atr@ngw.compcent.usu.ru) Received: from relay4.usu.ru (relay4.usu.ru [194.226.235.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E90A43D45 for ; Mon, 22 Aug 2005 07:11:40 +0000 (GMT) (envelope-from atr@ngw.compcent.usu.ru) Received: from relay4.usu.ru (localhost [127.0.0.1]) by relay4.usu.ru (PostSystem) with ESMTP id 0C829167FD for ; Mon, 22 Aug 2005 13:11:36 +0600 (YEKST) Received: from usu2.usu.ru (usu2.usu.ru [194.226.237.16]) by relay4.usu.ru (PostSystem) with ESMTP id 0228E167FC for ; Mon, 22 Aug 2005 13:11:35 +0600 (YEKST) Received: from localhost.usu2.usu.ru (localhost.usu2.usu.ru [127.0.0.1]) by usu2.usu.ru (Postfix) with ESMTP id 1A4DDA7A32 for ; Mon, 22 Aug 2005 13:11:34 +0600 (YEKST) Received: from labfer.usu.ru (labfer.usu.ru [194.226.237.37]) by usu2.usu.ru (Postfix) with ESMTP id C7E9AA7A31 for ; Mon, 22 Aug 2005 13:11:33 +0600 (YEKST) Received: from labfer.usu.ru (localhost [127.0.0.1]) by labfer.usu.ru (8.13.3/8.13.3) with ESMTP id j7M7BX6P039683 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 22 Aug 2005 13:11:33 +0600 (YEKST) (envelope-from atr@ngw.compcent.usu.ru) Received: (from atr@localhost) by labfer.usu.ru (8.13.3/8.13.1/Submit) id j7M7BWCI039682; Mon, 22 Aug 2005 13:11:32 +0600 (YEKST) (envelope-from atr@ngw.compcent.usu.ru) X-Authentication-Warning: labfer.usu.ru: atr set sender to atr@ngw.compcent.usu.ru using -f Date: Mon, 22 Aug 2005 13:11:32 +0600 From: Alexander Trubin To: freebsd-performance@freebsd.org Message-ID: <20050822071132.GA29339@spm.usu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Virus-Scanned: ClamAV version 0.83, clamav-milter version 0.83 on labfer.usu.ru X-Virus-Status: Clean X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.15; AVE: 6.31.1.0; VDF: 6.31.1.152; host: usu2.usu.ru) X-Virus-Scanned: ClamAV using ClamSMTP Subject: page faults X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2005 07:11:42 -0000 hi i have a server amd64 athlon 3000+ with 1GB ram (dmesg,config can be found here: http://spm.usu.ru/_amd64/). FreeBSD 5.4 amd64 running this script while server is idle: while 1 ls > /dev/null end gives me the following output of vmstat -w 1 hst# vmstat -w 1 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 ad6 in sy cs us sy id 0 1 0 550608 86704 0 0 0 0 0 0 0 0 2174 176 2116 0 2 98 1 1 0 550848 86596 25111 0 0 0 22879 0 0 0 2177 59365 4057 17 35 48 1 1 0 550608 86704 50879 0 0 0 46406 0 0 0 2159 119742 5873 38 62 0 0 1 1 550608 86704 50898 0 0 0 46407 0 0 0 2153 119890 5872 38 62 0 1 1 0 550844 86616 50912 0 0 0 46408 0 0 0 2156 120020 5774 36 64 0 1 1 0 550608 86704 50884 0 0 0 46408 0 0 0 2162 119745 5841 33 67 0 ... doing the same thing on freebsd 5.4,4.11 on i386 produces much less number of flt(page faults) which rarely goes above 4000-5000. I'm just trying to tune my apache with cgi version of php, which now have not so good performance. Is the high number of page faults ok on amd64? If not, what can be the reason and how can i reduce it ? i can provide any additional information. thanks in advance. -- Alexandr S Trubin From owner-freebsd-performance@FreeBSD.ORG Mon Aug 22 15:04:50 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D542F16A420 for ; Mon, 22 Aug 2005 15:04:50 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from mail.pgt.mpt.gov.br (mail.pgt.mpt.gov.br [200.157.62.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 50A5943D60 for ; Mon, 22 Aug 2005 15:04:49 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from [10.0.0.136] (516e.pgt.mpt.gov.br [10.0.0.136]) by mail.pgt.mpt.gov.br (8.13.1/8.13.1) with ESMTP id j7MF4lA3002766 for ; Mon, 22 Aug 2005 12:04:47 -0300 (BRST) (envelope-from tuliogs@pgt.mpt.gov.br) Message-ID: <4309EA1B.6090108@pgt.mpt.gov.br> Date: Mon, 22 Aug 2005 12:07:07 -0300 From: =?ISO-8859-1?Q?Tulio_Guimar=E3es_da_Silva?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: FreeBSD Mailing Lists References: In-Reply-To: Content-Type: multipart/mixed; boundary="------------000408050802020700020706" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Slow apache response X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2005 15:04:51 -0000 This is a multi-part message in MIME format. --------------000408050802020700020706 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit A bit late, but... :) Shane Ambler wrote: >On 18/8/05 12:40 AM, "Shane Ambler" wrote: > >>n 17/8/05 9:43 PM, "Claus Guttesen" wrote: >> >> >>>>Apart from apache there is sendmail and ssh running (and the basics such as >>>>tty's, cron and syslog) >>>>All pages are php. >>>>Any ideas on how I can get response times up? >>>> >>>> >>>Are you running apache 1.3 or 2.0? Is httpd.conf configured *not* to >>>do reverse dns-look-up, 'HostnameLookups Off'. Try setting 'KeepAlive >>>Off' if set to on. >>> >>erver version: Apache/1.3.33 (Unix) >>Server built: Feb 10 2005 12:34:22 >> >>KeepAlive on >>HostnameLookup off >> >>I'll try changing KeepAlive >> >Setting KeepAlive off seemed to make improvements initially but it has >slowed down again (2 hours later) - maybe I need to look at caching? > > Keepalive Off shouldn´t really help much if Apache is not topping the maximum number of processes. For what it seems (to me), the problem is not directly apache-related, but maybe PHP, Mysql or some other "peripheral" system, since CPU and proccess usage is not the bottleneck. Anyway, there are a few tricks I can suggest - excuses if some are too elementary, but these could also help other people: ;) 1) Apache: check "Min/Max SpareServers", "Startservers" (which I guess, from your first message, it´s set to 150) and (important) "MaxClients". If they´re set to reasonable values, then the problem is probably elsewhere; 2) PHP: try varying some settings in phi.ini (usually in /usr/local/etc). Ports intallations suggest tuning "output_buffering" and I would also suggest you to look at memory_limit. It could be also a good idea to enable error logging to figure out if other settings are insufficient; 3) Mysql: there seems to be some performance issues with this one. There´s a not-so-old thread discussing Mysql performance in FBSD compared to Linux, and even though our Mysql usage in FBSD here at work is almost to be ignored, :) there are some good performance tips in there that helped; 4) and, finally, OS&hardware: try to see how much memory is effectively being used and by who; maybe some apps (like PHP) are auto-limiting their memory use. Also, there has been a few issues with I/O in 5.4, specially with RAID and fxp subsystems - e.g.: I just raised by about 20-30% disk transfer rates on a DL380 and by 50% on a ML-110 with Highpoint RR1820A. ;) Don´t the logs show any anormality (massive crashes, timeouts, etc.)? I hope this all could help. Have luck, Tulio G. da Silva --------------000408050802020700020706-- From owner-freebsd-performance@FreeBSD.ORG Mon Aug 22 19:09:20 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAB5E16A42B for ; Mon, 22 Aug 2005 19:09:20 +0000 (GMT) (envelope-from oxy@field.hu) Received: from dumballah.tvnet.hu (dumballah.tvnet.hu [195.38.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6ADE943D45 for ; Mon, 22 Aug 2005 19:09:18 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (localhost.localdomain [127.0.0.1]) by dumballah.tvnet.hu (Postfix) with ESMTP id 5E27210180A for ; Mon, 22 Aug 2005 21:09:16 +0200 (CEST) Received: from dumballah.tvnet.hu ([127.0.0.1]) by localhost (dumballah.tvnet.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 32570-10 for ; Mon, 22 Aug 2005 21:09:15 +0200 (CEST) Received: from oxy (adsl-9b-159.pool.tvnet.hu [217.197.177.159]) by dumballah.tvnet.hu (Postfix) with SMTP id 03EC81017A2 for ; Mon, 22 Aug 2005 21:09:15 +0200 (CEST) Message-ID: <001801c5a74d$003fd1b0$0201a8c0@oxy> From: "OxY" To: Date: Mon, 22 Aug 2005 21:09:20 +0200 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-Virus-Scanned: by amavisd-new at tvnet.hu Cc: Subject: mysql 4.1.13 performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2005 19:09:21 -0000 hi! i am writing you to give me please some hints & tricks to improve my stock mysql performance.. i know there was a topic about fbsd vs linux mysql perf, but what i need is just a few compile and .cnf options to improve the stuff as much as possible, not to chase linux.. thanks guys! From owner-freebsd-performance@FreeBSD.ORG Tue Aug 23 01:27:15 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C583516A41F for ; Tue, 23 Aug 2005 01:27:15 +0000 (GMT) (envelope-from decibel@decibel.org) Received: from flake.decibel.org (flake.decibel.org [67.100.216.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D22243D45 for ; Tue, 23 Aug 2005 01:27:15 +0000 (GMT) (envelope-from decibel@decibel.org) Received: by flake.decibel.org (Postfix, from userid 1001) id DC9DE152AD; Mon, 22 Aug 2005 20:27:13 -0500 (CDT) Date: Mon, 22 Aug 2005 20:27:13 -0500 From: "Jim C. Nasby" To: Michael VInce Message-ID: <20050823012713.GN17203@decibel.org> References: <42F6B4CD.5030703@roq.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42F6B4CD.5030703@roq.com> X-Operating-System: FreeBSD 4.11-RELEASE-p10 i386 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.9i Cc: freebsd-performance@freebsd.org, Claus Guttesen Subject: Re: changing max_connections in postgresql on FreeBSD 5.4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2005 01:27:15 -0000 On Mon, Aug 08, 2005 at 11:26:37AM +1000, Michael VInce wrote: > Also note that I have a MySQL server with a similar table setup and it > performs a lot better. If you provide example queries that are performing worse than in MySQL along with the output of EXPLAIN ANALYZE on them to the PostgreSQL performance tuning list I'm sure you'd get some help. Include details about your machine configuration and postgresql.conf variables you've changed too. Also, make sure you're actually running MySQL in such a way that it's ACID compliant; it's very easy to configure it so that your data isn't safe at all, which would be an apples-oranges comparison to PostgreSQL. -- Jim C. Nasby, Database Architect decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Tue Aug 23 01:30:17 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B74CE16A41F for ; Tue, 23 Aug 2005 01:30:17 +0000 (GMT) (envelope-from decibel@decibel.org) Received: from flake.decibel.org (flake.decibel.org [67.100.216.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 419CF43D53 for ; Tue, 23 Aug 2005 01:30:17 +0000 (GMT) (envelope-from decibel@decibel.org) Received: by flake.decibel.org (Postfix, from userid 1001) id 530DE152AF; Mon, 22 Aug 2005 20:30:16 -0500 (CDT) Date: Mon, 22 Aug 2005 20:30:16 -0500 From: "Jim C. Nasby" To: OxY Message-ID: <20050823013016.GO17203@decibel.org> References: <001801c5a74d$003fd1b0$0201a8c0@oxy> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <001801c5a74d$003fd1b0$0201a8c0@oxy> X-Operating-System: FreeBSD 4.11-RELEASE-p10 i386 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.9i Cc: performance@freebsd.org Subject: Re: mysql 4.1.13 performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2005 01:30:17 -0000 On Mon, Aug 22, 2005 at 09:09:20PM +0200, OxY wrote: > i know there was a topic about fbsd vs linux mysql perf, but > what i need is just a few compile and .cnf options to improve the stuff > as much as possible, not to chase linux.. Then why aren't you asking on the mysql lists? -- Jim C. Nasby, Database Architect decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Tue Aug 23 02:08:44 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B09516A420 for ; Tue, 23 Aug 2005 02:08:44 +0000 (GMT) (envelope-from david.mao@thomson.net) Received: from dmzraw4.extranet.tce.com (dmzraw4.extranet.tce.com [157.254.234.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5134143D46 for ; Tue, 23 Aug 2005 02:08:43 +0000 (GMT) (envelope-from david.mao@thomson.net) Received: from indyvss2.am.thmulti.com (unknown [157.254.92.61]) by dmzraw4.extranet.tce.com (Postfix) with ESMTP id 3981F8C7 for ; Tue, 23 Aug 2005 02:08:42 +0000 (GMT) Received: from localhost (localhost [127.0.0.1]) by indyvss2.am.thmulti.com (Postfix) with ESMTP id 21FA71D1F for ; Tue, 23 Aug 2005 02:08:42 +0000 (GMT) Received: from indyvss2.am.thmulti.com ([127.0.0.1]) by localhost (indyvss2 [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 17807-01-95 for ; Tue, 23 Aug 2005 02:08:40 +0000 (GMT) Received: from smtprelay2.indy.tce.com (smtprelay2.indy.tce.com [157.254.96.95]) by indyvss2.am.thmulti.com (Postfix) with ESMTP id 359711CEB for ; Tue, 23 Aug 2005 02:08:40 +0000 (GMT) Received: from boulsmailbh02.eu.thmulti.com (localhost [127.0.0.1]) by smtprelay2.indy.tce.com (8.12.9/8.12.8) with ESMTP id j7N28cgt022655 for ; Tue, 23 Aug 2005 02:08:39 GMT Received: from tahksmail02.ap.thmulti.com ([141.11.12.26]) by boulsmailbh02.eu.thmulti.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 23 Aug 2005 04:08:38 +0200 Received: from tahksmail01.ap.thmulti.com ([141.11.13.38]) by tahksmail02.ap.thmulti.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 23 Aug 2005 10:07:18 +0800 Received: from bjngsmail01.ap.thmulti.com ([10.11.70.35]) by tahksmail01.ap.thmulti.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 23 Aug 2005 10:07:18 +0800 content-class: urn:content-classes:message MIME-Version: 1.0 X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Date: Tue, 23 Aug 2005 10:07:14 +0800 Message-ID: <31021C278A7A6B4AB95E9A085C3552180FF74E@bjngsmail01> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: polling in 4.11 vs 5.4 Thread-Index: AcWnh2CfitWU/R+xStKqCTLmU1TG2Q== From: "Mao Shou Yan" To: X-OriginalArrivalTime: 23 Aug 2005 02:07:18.0549 (UTC) FILETIME=[63BA1450:01C5A787] X-Virus-Scanned: amavisd-new at thomson.net Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: polling in 4.11 vs 5.4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2005 02:08:44 -0000 Hi, everybody, I hope this is the right place that I post to! When I enabled polling in 5.4 and 4.11, I found that CPU load in 5.4 is much higher than 4.11. For example, suppose HZ is 5000, in 5.4 the idle CPU is about 87%, but in 4.11, the idle CPU is about 99.9%. Is this right? Or the statistic is wrong in 4.11? BTW, ACPI is disabled under 5.4. I'm looking forward your reply. Any response is welcome! =20 David.Mao From owner-freebsd-performance@FreeBSD.ORG Tue Aug 23 02:24:31 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF07616A41F for ; Tue, 23 Aug 2005 02:24:30 +0000 (GMT) (envelope-from david.mao@thomson.net) Received: from dmzraw4.extranet.tce.com (dmzraw4.extranet.tce.com [157.254.234.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4054643D48 for ; Tue, 23 Aug 2005 02:24:30 +0000 (GMT) (envelope-from david.mao@thomson.net) Received: from indyvss2.am.thmulti.com (unknown [157.254.92.61]) by dmzraw4.extranet.tce.com (Postfix) with ESMTP id 86A689CE for ; Tue, 23 Aug 2005 02:24:29 +0000 (GMT) Received: from localhost (localhost [127.0.0.1]) by indyvss2.am.thmulti.com (Postfix) with ESMTP id 7562B16F67 for ; Tue, 23 Aug 2005 02:24:29 +0000 (GMT) Received: from indyvss2.am.thmulti.com ([127.0.0.1]) by localhost (indyvss2 [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 21415-01-10 for ; Tue, 23 Aug 2005 02:24:26 +0000 (GMT) Received: from indysmailcs01.am.thmulti.com (indysmailcs01.am.thmulti.com [157.254.96.5]) by indyvss2.am.thmulti.com (Postfix) with ESMTP id 40C6516F66 for ; Tue, 23 Aug 2005 02:24:26 +0000 (GMT) Received: from indysmailbh01.am.thmulti.com ([157.254.96.4]) by indysmailcs01.am.thmulti.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 22 Aug 2005 21:24:26 -0500 Received: from tahkexch2k.ap.thmulti.com ([141.11.13.12]) by indysmailbh01.am.thmulti.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 22 Aug 2005 21:24:26 -0500 Received: from tahksmail01.ap.thmulti.com ([141.11.13.38]) by tahkexch2k.ap.thmulti.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 23 Aug 2005 10:24:02 +0800 Received: from bjngsmail01.ap.thmulti.com ([10.11.70.35]) by tahksmail01.ap.thmulti.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 23 Aug 2005 10:24:01 +0800 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Date: Tue, 23 Aug 2005 10:20:27 +0800 Message-ID: <31021C278A7A6B4AB95E9A085C3552180FF765@bjngsmail01> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: polling in 4.11 vs 5.4 Thread-Index: AcWnh2CfitWU/R+xStKqCTLmU1TG2QAAa7pQ From: "Mao Shou Yan" To: X-OriginalArrivalTime: 23 Aug 2005 02:24:01.0802 (UTC) FILETIME=[B9B656A0:01C5A789] X-Virus-Scanned: amavisd-new at thomson.net Subject: RE: polling in 4.11 vs 5.4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2005 02:24:31 -0000 BTW, I found that context switch in 5.4 is much higher than 5.4 while = enabling polling. (Using systat -vmstat 1) -----Original Message----- From: owner-freebsd-performance@freebsd.org = [mailto:owner-freebsd-performance@freebsd.org] On Behalf Of Mao Shou Yan Sent: 2005=C4=EA8=D4=C223=C8=D5 10:07 To: freebsd-performance@freebsd.org Subject: polling in 4.11 vs 5.4 Hi, everybody, I hope this is the right place that I post to! When I enabled polling in 5.4 and 4.11, I found that CPU load in 5.4 is much higher than 4.11. For example, suppose HZ is 5000, in 5.4 the idle CPU is about 87%, but in 4.11, the idle CPU is about 99.9%. Is this right? Or the statistic is wrong in 4.11? BTW, ACPI is disabled under 5.4. I'm looking forward your reply. Any response is welcome! =20 David.Mao _______________________________________________ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to = "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 10:17:34 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C7A1A16A41F for ; Wed, 24 Aug 2005 10:17:34 +0000 (GMT) (envelope-from kometen@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDD6943D46 for ; Wed, 24 Aug 2005 10:17:32 +0000 (GMT) (envelope-from kometen@gmail.com) Received: by wproxy.gmail.com with SMTP id i22so63930wra for ; Wed, 24 Aug 2005 03:17:32 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=SU5fHT6lvhpg8jb7w1ag/oTSeA9AOFwORU8LeW1gTtqceOctvpqFe2PxbJraFzI4YPbrs1WAshPD85Fz++lvri8x3QTdGhsGfpIT+kyBPWMVGo19ZhPhhvtU/EV8BWup6H5sCDUd6s2WRMno3NoEKPOrR0pMC59vs5F16AX5Gr4= Received: by 10.54.100.1 with SMTP id x1mr423889wrb; Wed, 24 Aug 2005 03:17:32 -0700 (PDT) Received: by 10.54.144.11 with HTTP; Wed, 24 Aug 2005 03:17:31 -0700 (PDT) Message-ID: Date: Wed, 24 Aug 2005 12:17:32 +0200 From: Claus Guttesen To: FreeBSD Mailing Lists Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: Sun v40z vs. Tyan Transport TX46 as postgresql-server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 10:17:34 -0000 Hi. I'm looking at upgrading my db-server. It's currently running FreeBSD 5.4-release and postgresql 7.4.8 on a quad TX46 with 4 GB RAM. I've searched for comparative benchmarks pitting the TX46 agains a Sun v40z, but haven't found any that indicates what system does fare the best. The planned upgrade will have 8 GB RAM and will be running FreeBSD 6.0 when that is available. The postgresql-performance-list only mentions the v40z against the Nocona-c= pu. regards Claus From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 16:34:07 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 49CB716A41F for ; Wed, 24 Aug 2005 16:34:07 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 8298A43D48 for ; Wed, 24 Aug 2005 16:34:06 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bio35.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.130.35) by matrix01.home.net.pl with SMTP; Wed, 24 Aug 2005 16:34:03 -0000 Date: Wed, 24 Aug 2005 18:33:57 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <1168719770.20050824183357@adeon.lublin.pl> To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 24 Aug 2005 16:47:47 +0000 Subject: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 16:34:07 -0000 Hello, I had to unpack a lot of tar archives and I occasional noticed terrible bad performance on freebsd5. This is my test file: 854251520 24 Sie 12:13 mysql-m.tgz There are some real MySQL tables in it, it has been done with tar -cvf. This archive contains about 146.000 small files. --------------------------------------- Unpacking it on FreeBSD5 gives me such results: # time tar -xf mysql-m.tgz 2.130u 20.187s 7:02.69 5.2% 41+382k 13097+8205io 0pf+0w ...so 7 minutes of real time. This is today's FreeBSD 5.4-STABLE but I also tried 5.4-release. That server is brand new Dell PE2850 with Seagate ST373207L SCSI drive, no raid. Parition is default UFS2 mounted with noatime, softupdates on. This is Dual Xeon 2.8, 2GB ram. My sysctls: vfs.ufs.dirhash_maxmem=16777216 (much better than default 2MB) machdep.hyperthreading_allowed=1 (better dd results) kern.maxfiles=65536 kern.maxfilesperproc=65536 vfs.read_max=16 I've tried a few different kernels - I had results like 6-15 minutes of real time. 6 minutes was the best. I've also tried standard sysctls, and with/without make.conf cputype. I've tested bsdtar and gtar from ports (btw. gtar seems to be faster). I didn't noticed slowness of any other things on this server. I've quiet good DD results. Diskinfo shows: # diskinfo -tv /dev/da3s1d /dev/da3s1d 512 # sectorsize 73402366464 # mediasize in bytes (68G) 143363997 # mediasize in sectors 8923 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Seek times: Full stroke: 250 iter in 3.107691 sec = 12.431 msec Half stroke: 250 iter in 2.223141 sec = 8.893 msec Quarter stroke: 500 iter in 3.595413 sec = 7.191 msec Short forward: 400 iter in 1.832164 sec = 4.580 msec Short backward: 400 iter in 2.047162 sec = 5.118 msec Seq outer: 2048 iter in 0.315528 sec = 0.154 msec Seq inner: 2048 iter in 0.320899 sec = 0.157 msec Transfer rates: outside: 102400 kbytes in 1.341191 sec = 76350 kbytes/sec middle: 102400 kbytes in 1.536625 sec = 66640 kbytes/sec inside: 102400 kbytes in 2.731570 sec = 37488 kbytes/sec Soon I'll be able to post results from my other server powered by FreeBSD 4.11 with poor SCSI, maybe in the night. ---------------------------------------- This is result from Gentoo Linux on 2.6.x hardened kernel: # time tar -xf mysql-m.tgz real 1m3.944s user 0m1.702s sys 0m15.794s Only ~one minute! Six times faster than on a FreeBSD. I'm not a linux fan, and I don't want to tell you how good linux is, but I would to find out what causes such bad results on my favourite FreeBSD... This one is noname server with Dual Xeon 2.8, 2GB ram. It has SATA Maxtor drive. Partition is ext3 -T news, -O dir_index, with atime! I've similar results with ReiserFS. ------------------------------------------ Both systems was good optimized - as good as I can. All services (like mysql, apache) was terminated before my tests. Both FreeBSD's locate.updatedb and Gentoo's updatedb disabled. Thank you for any opinions or hints how to tune FreeBSD a bit. Best Regards, JG From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 17:00:18 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 86F9A16A41F for ; Wed, 24 Aug 2005 17:00:18 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (www1.multiplay.co.uk [212.42.16.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE1E743D46 for ; Wed, 24 Aug 2005 17:00:17 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from stevenp4 ([193.123.241.40]) by multiplay.co.uk (multiplay.co.uk [212.42.16.7]) (MDaemon.PRO.v8.1.0.R) with ESMTP id md50001821631.msg for ; Wed, 24 Aug 2005 17:53:35 +0100 Message-ID: <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> From: "Steven Hartland" To: "JG" , References: <1168719770.20050824183357@adeon.lublin.pl> Date: Wed, 24 Aug 2005 18:00:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Spam-Processed: multiplay.co.uk, Wed, 24 Aug 2005 17:53:35 +0100 (not processed: message from valid local sender) X-MDRemoteIP: 193.123.241.40 X-Return-Path: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-performance@freebsd.org X-MDAV-Processed: multiplay.co.uk, Wed, 24 Aug 2005 17:53:36 +0100 Cc: Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 17:00:18 -0000 Might be silly but do u get similar results if u: 1. expand to a memory backed disk 2. expand to /dev/null Steve ----- Original Message ----- From: "JG" > I had to unpack a lot of tar archives and I occasional noticed terrible > bad performance on freebsd5. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 17:49:50 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE1BE16A41F for ; Wed, 24 Aug 2005 17:49:50 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 0463443D45 for ; Wed, 24 Aug 2005 17:49:49 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bio35.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.130.35) by matrix01.home.net.pl with SMTP; Wed, 24 Aug 2005 17:49:46 -0000 Date: Wed, 24 Aug 2005 19:49:39 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <818587040.20050824194939@adeon.lublin.pl> To: "Steven Hartland" In-Reply-To: <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 17:49:51 -0000 > Might be silly but do u get similar results if u: > 1. expand to a memory backed disk > 2. expand to /dev/null Hello, Thank you for test advice. FreeBSD: # time tar -xf mysql-m.tgz -O > /dev/null 1.125u 3.007s 0:04.13 99.7% 41+323k 0+0io 0pf+0w # time tar -xf mysql-m.tgz -O > /dev/null 1.194u 3.013s 0:04.22 99.5% 41+321k 1+0io 0pf+0w So the same result twice. BTW It's fast. ------------------------------- Gentoo: # time tar -xf mysql-m.tgz -O > /dev/null real 0m7.586s user 0m0.906s sys 0m1.352s # time tar -xf mysql-m.tgz -O > /dev/null real 0m1.640s user 0m0.833s sys 0m0.808s Seems like a cached. First time 7,5 sec, each next time it takes only 1.6secs. I checked it with the other file and result is the same. Anyway, it's only a few secs difference, so the problem must be with write. Any other hints? :) Thank You, JG From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 17:57:46 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 294D316A41F for ; Wed, 24 Aug 2005 17:57:46 +0000 (GMT) (envelope-from fullermd@over-yonder.net) Received: from mortis.over-yonder.net (adsl-157-21-158.jan.bellsouth.net [70.157.21.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 91BE443D46 for ; Wed, 24 Aug 2005 17:57:44 +0000 (GMT) (envelope-from fullermd@over-yonder.net) Received: by mortis.over-yonder.net (Postfix, from userid 100) id 7AF8C2109C; Wed, 24 Aug 2005 12:57:43 -0500 (CDT) Date: Wed, 24 Aug 2005 12:57:43 -0500 From: "Matthew D. Fuller" To: JG Message-ID: <20050824175742.GA85180@over-yonder.net> References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> <818587040.20050824194939@adeon.lublin.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <818587040.20050824194939@adeon.lublin.pl> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.9i-fullermd.2 Cc: freebsd-performance@freebsd.org, Steven Hartland Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 17:57:46 -0000 On Wed, Aug 24, 2005 at 07:49:39PM +0200 I heard the voice of JG, and lo! it spake thus: > > Anyway, it's only a few secs difference, so the problem must be with > write. Any other hints? :) Gentoo is using GNU tar, 5.4 is using bsdtar. Try installing gtar on FreeBSD and see what the time is. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 17:58:50 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B5D216A421 for ; Wed, 24 Aug 2005 17:58:50 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from mail.pgt.mpt.gov.br (mail.pgt.mpt.gov.br [200.157.62.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id D466D43D69 for ; Wed, 24 Aug 2005 17:58:45 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from [10.0.0.136] (516e.pgt.mpt.gov.br [10.0.0.136]) by mail.pgt.mpt.gov.br (8.13.1/8.13.1) with ESMTP id j7OHwhew088945 for ; Wed, 24 Aug 2005 14:58:44 -0300 (BRST) (envelope-from tuliogs@pgt.mpt.gov.br) Message-ID: <430CB5D2.1060103@pgt.mpt.gov.br> Date: Wed, 24 Aug 2005 15:00:50 -0300 From: =?ISO-8859-1?Q?Tulio_Guimar=E3es_da_Silva?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> In-Reply-To: <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> Content-Type: multipart/mixed; boundary="------------090702060006050300040908" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 17:58:50 -0000 This is a multi-part message in MIME format. --------------090702060006050300040908 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Hi, I´ve got the same kind of problem, not only with DDS-[234] tapes, but also with "all-powerful-with-bells-and-whistles" AIT-3 units, with controllers ranging from Adaptec stock 2940 to PCI-X Ultra-320... almost same results. The problems seems to lie in tar itself; I read there´s something to do with block sizes, but using -b with larger values got me not much more than corrupt or incomplete data. :( The only way I got to have decent transfer rates AND reliability was to filter *archiving* through dd, including block sizes. For example, to archive: # tar -zcpf - /usr/local | dd of=/dev/sa0 bs=64k and to restore: # tar -b 128 -zxvf /dev/sa0 The above is particullarly true for remote transfers; if you´re using tar over rsh/rmt (-f host:/path), you´ll surely prefer simple "rsh/tar" with output redirection. ;) Note that block sizes in tar count as 512-byte ones, while in dd they can be specified as Kilobytes or even megabytes. Besides speed, there´s a sensible boost on storage space when using dd-block-sized transfers. The apparent reason for that is tar actually uses -b 20 (10kb) blocks, while 10GB+ tapes usually expect larger sizes. For AIT-3, I didn´t notice any real good improvement past 128kb block sizes; I didn´t experiment enough with DDS-4 because our test tape drive got a heart attack and quit... BTW, it returned 2 weeks ago and I didn´t give it any attention; it may be a little depressed by now, so I guess I´ll return it to test beds. :) Anyway, I wouldn´t try anything lower than 32kb blocks on it. I can´t remember if I did any test pointing to /dev/null, as mr. Hartland suggested, nor from /dev/zero or /dev/random... it´s worth a try. I´ll post new results as soon as I get´em (if any). ;) Have luck, Tulio G. da Silva Steven Hartland wrote: > Might be silly but do u get similar results if u: > 1. expand to a memory backed disk > 2. expand to /dev/null > > Steve > ----- Original Message ----- From: "JG" > >> I had to unpack a lot of tar archives and I occasional noticed terrible >> bad performance on freebsd5. > --------------090702060006050300040908-- From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 18:02:42 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CEA6016A41F for ; Wed, 24 Aug 2005 18:02:42 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from mail.pgt.mpt.gov.br (mail.pgt.mpt.gov.br [200.157.62.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3FEA43D48 for ; Wed, 24 Aug 2005 18:02:39 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from [10.0.0.136] (516e.pgt.mpt.gov.br [10.0.0.136]) by mail.pgt.mpt.gov.br (8.13.1/8.13.1) with ESMTP id j7OI2ciT089185 for ; Wed, 24 Aug 2005 15:02:39 -0300 (BRST) (envelope-from tuliogs@pgt.mpt.gov.br) Message-ID: <430CB6CC.3050808@pgt.mpt.gov.br> Date: Wed, 24 Aug 2005 15:05:00 -0300 From: =?ISO-8859-1?Q?Tulio_Guimar=E3es_da_Silva?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> <818587040.20050824194939@adeon.lublin.pl> <20050824175742.GA85180@over-yonder.net> In-Reply-To: <20050824175742.GA85180@over-yonder.net> Content-Type: multipart/mixed; boundary="------------080402040605030802080905" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 18:02:42 -0000 This is a multi-part message in MIME format. --------------080402040605030802080905 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Matthew D. Fuller wrote: >On Wed, Aug 24, 2005 at 07:49:39PM +0200 I heard the voice of >JG, and lo! it spake thus: > > >>Anyway, it's only a few secs difference, so the problem must be with >>write. Any other hints? :) >> >> > >Gentoo is using GNU tar, 5.4 is using bsdtar. Try installing gtar on >FreeBSD and see what the time is. > Tip: gtar is already available by default on complete installations at /usr/bin/gtar. ;) I didn´t notice much speed difference using it, though. :( --------------080402040605030802080905-- From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 18:03:07 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F83416A41F for ; Wed, 24 Aug 2005 18:03:07 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 390A343D49 for ; Wed, 24 Aug 2005 18:03:06 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bio35.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.130.35) by matrix01.home.net.pl with SMTP; Wed, 24 Aug 2005 18:03:02 -0000 Date: Wed, 24 Aug 2005 20:02:55 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <84677990.20050824200255@adeon.lublin.pl> To: "Matthew D. Fuller" In-Reply-To: <20050824175742.GA85180@over-yonder.net> References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> <818587040.20050824194939@adeon.lublin.pl> <20050824175742.GA85180@over-yonder.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 18:03:07 -0000 > Gentoo is using GNU tar, 5.4 is using bsdtar. Try installing gtar on > FreeBSD and see what the time is. "I've tested bsdtar and gtar from ports (btw. gtar seems to be faster)." It's not big difference. JG From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 18:27:48 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1299A16A41F for ; Wed, 24 Aug 2005 18:27:48 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: from smtp107.sbc.mail.mud.yahoo.com (smtp107.sbc.mail.mud.yahoo.com [68.142.198.206]) by mx1.FreeBSD.org (Postfix) with SMTP id 7D3FE43D46 for ; Wed, 24 Aug 2005 18:27:47 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: (qmail 45448 invoked from network); 24 Aug 2005 18:27:46 -0000 Received: from unknown (HELO optimator.noacks.org) (noacks@swbell.net@70.240.231.0 with login) by smtp107.sbc.mail.mud.yahoo.com with SMTP; 24 Aug 2005 18:27:46 -0000 Received: from localhost (localhost [127.0.0.1]) by optimator.noacks.org (Postfix) with ESMTP id 42CCF60EB; Wed, 24 Aug 2005 13:27:45 -0500 (CDT) Received: from optimator.noacks.org ([127.0.0.1]) by localhost (optimator.noacks.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 07464-16; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) Received: from compgeek.noacks.org (compgeek [192.168.1.10]) by optimator.noacks.org (Postfix) with ESMTP id D2FA060E8; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) by compgeek.noacks.org (8.13.4/8.13.4) with ESMTP id j7OIRf0q002347; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) (envelope-from noackjr@alumni.rice.edu) Message-ID: <430CBC18.6040902@alumni.rice.edu> Date: Wed, 24 Aug 2005 13:27:36 -0500 From: Jonathan Noack User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050727) X-Accept-Language: en-us, en MIME-Version: 1.0 To: JG References: <1168719770.20050824183357@adeon.lublin.pl> In-Reply-To: <1168719770.20050824183357@adeon.lublin.pl> X-Enigmail-Version: 0.92.0.0 OpenPGP: id=991D8195; url=http://www.noacks.org/cert/noackjr.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigAAFC06154F4DB0DF41AB9E66" X-Virus-Scanned: amavisd-new at noacks.org Cc: freebsd-performance@freebsd.org Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: noackjr@alumni.rice.edu List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 18:27:48 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigAAFC06154F4DB0DF41AB9E66 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 08/24/05 11:33, JG wrote: > I had to unpack a lot of tar archives and I occasional noticed terrible > bad performance on freebsd5. > > This is my test file: > > 854251520 24 Sie 12:13 mysql-m.tgz > > There are some real MySQL tables in it, it has been done with tar > -cvf. This archive contains about 146.000 small files. > > --------------------------------------- > Unpacking it on FreeBSD5 gives me such results: > > # time tar -xf mysql-m.tgz > 2.130u 20.187s 7:02.69 5.2% 41+382k 13097+8205io 0pf+0w > ...so 7 minutes of real time. > > This is today's FreeBSD 5.4-STABLE but I also tried 5.4-release. > > That server is brand new Dell PE2850 with Seagate ST373207L SCSI > drive, no raid. Parition is default UFS2 mounted with noatime, softupdates on. Will noatime make a difference when unpacking a tar archive (assuming an otherwise idle system, at least)? My understanding of atime is that it might slow down the disk for later accesses due to atime writes, but when creating files it shouldn't have any effect. Is that not correct? > This is Dual Xeon 2.8, 2GB ram. > > My sysctls: > vfs.ufs.dirhash_maxmem=16777216 (much better than default 2MB) > machdep.hyperthreading_allowed=1 (better dd results) Your other settings appear ok, but I'd turn off hyperthreading. Almost every FreeBSD/HT test has shown that it reduces performance because the scheduler is not HT-aware. When the system is relatively idle (single dd running, for example), it might not pessimize things, but it will most likely slow you down under load. > kern.maxfiles=65536 > kern.maxfilesperproc=65536 > vfs.read_max=16 > > > > ---------------------------------------- > This is result from Gentoo Linux on 2.6.x hardened kernel: > # time tar -xf mysql-m.tgz > > real 1m3.944s > user 0m1.702s > sys 0m15.794s > > Only ~one minute! Six times faster than on a FreeBSD. I'm not a linux > fan, and I don't want to tell you how good linux is, but I would to > find out what causes such bad results on my favourite FreeBSD... This sounds like you're running into the old "lemming syncer" problem. There is currently some work on disk schedulers (even a Summer of Code project), but it will most likely not make it into 5.x. -- Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195 --------------enigAAFC06154F4DB0DF41AB9E66 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFDDLwdUFz01pkdgZURAm35AJ9cLy4yFsZVOddNKWhx+P8xa+L35wCglRkt VcQCEMX22H2TqJbzfm9oTBk= =Zdqt -----END PGP SIGNATURE----- --------------enigAAFC06154F4DB0DF41AB9E66-- From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 18:31:45 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F7B516A41F for ; Wed, 24 Aug 2005 18:31:45 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from mail.pgt.mpt.gov.br (mail.pgt.mpt.gov.br [200.157.62.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id BAC4543D49 for ; Wed, 24 Aug 2005 18:31:44 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from [10.0.0.136] (516e.pgt.mpt.gov.br [10.0.0.136]) by mail.pgt.mpt.gov.br (8.13.1/8.13.1) with ESMTP id j7OIVhwr090428 for ; Wed, 24 Aug 2005 15:31:43 -0300 (BRST) (envelope-from tuliogs@pgt.mpt.gov.br) Message-ID: <430CBD9D.8020702@pgt.mpt.gov.br> Date: Wed, 24 Aug 2005 15:34:05 -0300 From: =?ISO-8859-1?Q?Tulio_Guimar=E3es_da_Silva?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <1168719770.20050824183357@adeon.lublin.pl> <068901c5a8cd$4b8a15f0$7f06000a@int.mediasurface.com> <430CB5D2.1060103@pgt.mpt.gov.br> In-Reply-To: <430CB5D2.1060103@pgt.mpt.gov.br> Content-Type: multipart/mixed; boundary="------------070006030104000000040300" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 18:31:45 -0000 This is a multi-part message in MIME format. --------------070006030104000000040300 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit I must be working too much... I can´t see where I took the DDS-thing from... O_o Sorry, now I see it´s a disk-only operation. >X( Anyway, you could try the dd thing; something like # dd if=mysql-m.tgz | tar -zxvf - and see if it makes any diference. There was a relatively new thread in April/May dealing with poor read/write results on 5.x branch.. the primary "target" was RAID, but maybe there´s a bunch of possible tuneups that may be applied to IDE and SCSI disks. Have you tried´em? Sorry again for the blatant misunderstanding. Tulio G. Silva Tulio Guimarães da Silva wrote: > Hi, > I´ve got the same kind of problem, not only with DDS-[234] tapes, but > also with "all-powerful-with-bells-and-whistles" AIT-3 units, with > controllers ranging from Adaptec stock 2940 to PCI-X Ultra-320... > almost same results. > The problems seems to lie in tar itself; I read there´s something to > do with block sizes, but using -b with larger values got me not much > more than corrupt or incomplete data. :( The only way I got to have > decent transfer rates AND reliability was to filter *archiving* > through dd, including block sizes. For example, to archive: > > # tar -zcpf - /usr/local | dd of=/dev/sa0 bs=64k > > and to restore: > # tar -b 128 -zxvf /dev/sa0 > > The above is particullarly true for remote transfers; if you´re using > tar over rsh/rmt (-f host:/path), you´ll surely prefer simple > "rsh/tar" with output redirection. ;) > Note that block sizes in tar count as 512-byte ones, while in dd they > can be specified as Kilobytes or even megabytes. Besides speed, > there´s a sensible boost on storage space when using dd-block-sized > transfers. The apparent reason for that is tar actually uses -b 20 > (10kb) blocks, while 10GB+ tapes usually expect larger sizes. > For AIT-3, I didn´t notice any real good improvement past 128kb block > sizes; I didn´t experiment enough with DDS-4 because our test tape > drive got a heart attack and quit... BTW, it returned 2 weeks ago and > I didn´t give it any attention; it may be a little depressed by now, > so I guess I´ll return it to test beds. :) Anyway, I wouldn´t try > anything lower than 32kb blocks on it. > I can´t remember if I did any test pointing to /dev/null, as mr. > Hartland suggested, nor from /dev/zero or /dev/random... it´s worth a > try. > I´ll post new results as soon as I get´em (if any). ;) > Have luck, > > Tulio G. da Silva > > Steven Hartland wrote: > >> Might be silly but do u get similar results if u: >> 1. expand to a memory backed disk >> 2. expand to /dev/null >> >> Steve >> ----- Original Message ----- From: "JG" >> >>> I had to unpack a lot of tar archives and I occasional noticed terrible >>> bad performance on freebsd5. >> >> >------------------------------------------------------------------------ > >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > --------------070006030104000000040300-- From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 21:00:40 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1D75E16A41F for ; Wed, 24 Aug 2005 21:00:40 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 399D343D45 for ; Wed, 24 Aug 2005 21:00:36 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bib251.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.117.251) by matrix01.home.net.pl with SMTP; Wed, 24 Aug 2005 21:00:32 -0000 Date: Wed, 24 Aug 2005 23:00:23 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <1626642863.20050824230023@adeon.lublin.pl> To: Jonathan Noack In-Reply-To: <430CBC18.6040902@alumni.rice.edu> References: <1168719770.20050824183357@adeon.lublin.pl> <430CBC18.6040902@alumni.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 21:00:40 -0000 > Will noatime make a difference when unpacking a tar archive (assuming an > otherwise idle system, at least)? My understanding of atime is that it > might slow down the disk for later accesses due to atime writes, but > when creating files it shouldn't have any effect. Is that not correct? Yes, you're right. > Your other settings appear ok, but I'd turn off hyperthreading. Almost > every FreeBSD/HT test has shown that it reduces performance because the > scheduler is not HT-aware. When the system is relatively idle (single > dd running, for example), it might not pessimize things, but it will > most likely slow you down under load. HTT is a big performance hit for me, mainly for webserver. I've much worse diskinfo results without HTT, don't know why. Tar is upacking a bit longer with HTT disabled. > This sounds like you're running into the old "lemming syncer" problem. > There is currently some work on disk schedulers (even a Summer of Code > project), but it will most likely not make it into 5.x. Agrh.. Regards, Jarek From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 21:52:01 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 782A016A41F for ; Wed, 24 Aug 2005 21:52:01 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id BB42D43D55 for ; Wed, 24 Aug 2005 21:52:00 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bib251.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.117.251) by matrix01.home.net.pl with SMTP; Wed, 24 Aug 2005 21:51:56 -0000 Date: Wed, 24 Aug 2005 23:51:48 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <469999288.20050824235148@adeon.lublin.pl> To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 21:52:01 -0000 > Anyway, you could try the dd thing; something like > # dd if=mysql-m.tgz | tar -zxvf - > and see if it makes any diference. Not for me, I've very similar result. Is there way to do something reverse - extract tar file to stdout or sth. and then "dd" it to files? JG From owner-freebsd-performance@FreeBSD.ORG Thu Aug 25 06:05:39 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA83D16A41F for ; Thu, 25 Aug 2005 06:05:39 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [68.142.200.97]) by mx1.FreeBSD.org (Postfix) with SMTP id 4909243D45 for ; Thu, 25 Aug 2005 06:05:39 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 81099 invoked by uid 60001); 25 Aug 2005 06:05:38 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=j8w66T27+rDzu4NZPV/gQYF+Gip5cgFXt5XmAR3UTMrn7on42jqwjcDTDdUYLdMFVxp+GNCrLbZm3f6Z7R6QPJDs2Yaum2tfOEauFjVJpkPptav70c7aC+Py1WPZAEJFKuCL2cJeo+rW72QvPci4hW6JfQgxxvhSlWRenCbZn2U= ; Message-ID: <20050825060538.81097.qmail@web30304.mail.mud.yahoo.com> Received: from [213.54.78.141] by web30304.mail.mud.yahoo.com via HTTP; Wed, 24 Aug 2005 23:05:38 PDT Date: Wed, 24 Aug 2005 23:05:38 -0700 (PDT) From: Arne "Wörner" To: JG , freebsd-performance@freebsd.org In-Reply-To: <469999288.20050824235148@adeon.lublin.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2005 06:05:39 -0000 --- JG wrote: > > Anyway, you could try the dd thing; something like > > # dd if=mysql-m.tgz | tar -zxvf - > > and see if it makes any diference. > > Not for me, I've very similar result. Is there way to do > something > reverse - extract tar file to stdout or sth. and then "dd" it to > files? > I say, what about time sh -c "dd if=mysql-m.tgz bs=1m | gunzip | tar xf -" ? :-) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Thu Aug 25 09:30:03 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3114416A41F for ; Thu, 25 Aug 2005 09:30:03 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.10.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFA2943D45 for ; Thu, 25 Aug 2005 09:30:01 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.4/8.13.3) with ESMTP id j7P9TvC7049944 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Thu, 25 Aug 2005 11:29:57 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.4/8.13.3/Submit) id j7P9Tvkb049943; Thu, 25 Aug 2005 11:29:57 +0200 (CEST) Date: Thu, 25 Aug 2005 11:29:57 +0200 From: Divacky Roman To: JG Message-ID: <20050825092957.GA49525@stud.fit.vutbr.cz> References: <1168719770.20050824183357@adeon.lublin.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1168719770.20050824183357@adeon.lublin.pl> User-Agent: Mutt/1.4.2i X-Scanned-By: MIMEDefang 2.49 on 147.229.10.14 Cc: freebsd-performance@freebsd.org Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2005 09:30:03 -0000 cannot this be related to low value of nswbufs (fixed in curennt and I think releng_6)? On Wed, Aug 24, 2005 at 06:33:57PM +0200, JG wrote: > Hello, > > I had to unpack a lot of tar archives and I occasional noticed terrible > bad performance on freebsd5. > > This is my test file: > > 854251520 24 Sie 12:13 mysql-m.tgz > > There are some real MySQL tables in it, it has been done with tar > -cvf. This archive contains about 146.000 small files. > > --------------------------------------- > Unpacking it on FreeBSD5 gives me such results: > > # time tar -xf mysql-m.tgz > 2.130u 20.187s 7:02.69 5.2% 41+382k 13097+8205io 0pf+0w > ...so 7 minutes of real time. > > This is today's FreeBSD 5.4-STABLE but I also tried 5.4-release. > > That server is brand new Dell PE2850 with Seagate ST373207L SCSI > drive, no raid. Parition is default UFS2 mounted with noatime, softupdates on. > This is Dual Xeon 2.8, 2GB ram. > > My sysctls: > vfs.ufs.dirhash_maxmem=16777216 (much better than default 2MB) > machdep.hyperthreading_allowed=1 (better dd results) > kern.maxfiles=65536 > kern.maxfilesperproc=65536 > vfs.read_max=16 > > I've tried a few different kernels - I had results like 6-15 minutes > of real time. 6 minutes was the best. > > I've also tried standard sysctls, and with/without make.conf cputype. > I've tested bsdtar and gtar from ports (btw. gtar seems to be faster). > > I didn't noticed slowness of any other things on this server. I've quiet good DD results. > > Diskinfo shows: > # diskinfo -tv /dev/da3s1d > /dev/da3s1d > 512 # sectorsize > 73402366464 # mediasize in bytes (68G) > 143363997 # mediasize in sectors > 8923 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > > Seek times: > Full stroke: 250 iter in 3.107691 sec = 12.431 msec > Half stroke: 250 iter in 2.223141 sec = 8.893 msec > Quarter stroke: 500 iter in 3.595413 sec = 7.191 msec > Short forward: 400 iter in 1.832164 sec = 4.580 msec > Short backward: 400 iter in 2.047162 sec = 5.118 msec > Seq outer: 2048 iter in 0.315528 sec = 0.154 msec > Seq inner: 2048 iter in 0.320899 sec = 0.157 msec > Transfer rates: > outside: 102400 kbytes in 1.341191 sec = 76350 kbytes/sec > middle: 102400 kbytes in 1.536625 sec = 66640 kbytes/sec > inside: 102400 kbytes in 2.731570 sec = 37488 kbytes/sec > > > Soon I'll be able to post results from my other server powered by > FreeBSD 4.11 with poor SCSI, maybe in the night. > > ---------------------------------------- > This is result from Gentoo Linux on 2.6.x hardened kernel: > # time tar -xf mysql-m.tgz > > real 1m3.944s > user 0m1.702s > sys 0m15.794s > > Only ~one minute! Six times faster than on a FreeBSD. I'm not a linux > fan, and I don't want to tell you how good linux is, but I would to > find out what causes such bad results on my favourite FreeBSD... > > This one is noname server with Dual Xeon 2.8, 2GB ram. It has > SATA Maxtor drive. Partition is ext3 -T news, -O dir_index, with > atime! I've similar results with ReiserFS. > > ------------------------------------------ > > Both systems was good optimized - as good as I can. All services (like > mysql, apache) was terminated before my tests. Both FreeBSD's > locate.updatedb and Gentoo's updatedb disabled. > > > Thank you for any opinions or hints how to tune FreeBSD a bit. > > Best Regards, > JG > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Thu Aug 25 20:41:56 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8E3BF16A420 for ; Thu, 25 Aug 2005 20:41:56 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id F23FA43D49 for ; Thu, 25 Aug 2005 20:41:54 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bhs170.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.108.170) by matrix01.home.net.pl with SMTP; Thu, 25 Aug 2005 20:41:50 -0000 Date: Thu, 25 Aug 2005 22:41:49 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <1476333643.20050825224149@adeon.lublin.pl> To: =?ISO-8859-1?B?QXJuZSBX9nJuZXI=?= In-Reply-To: <20050825092210.25239.qmail@web30312.mail.mud.yahoo.com> References: <134970360.20050825104636@adeon.lublin.pl> <20050825092210.25239.qmail@web30312.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-performance@freebsd.org Subject: Re[6]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2005 20:41:56 -0000 > Now I say, what about > # cpio -i < mysql-m.tgz > (assuming that mysql-m.tgz is in "tar" format)? Hello, idea is good, but the result is the same :/ JG From owner-freebsd-performance@FreeBSD.ORG Thu Aug 25 21:12:59 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE60016A41F for ; Thu, 25 Aug 2005 21:12:59 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 1FB7043D55 for ; Thu, 25 Aug 2005 21:12:58 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bhs170.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.108.170) by matrix01.home.net.pl with SMTP; Thu, 25 Aug 2005 21:12:55 -0000 Date: Thu, 25 Aug 2005 23:12:54 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <1887930059.20050825231254@adeon.lublin.pl> To: Divacky Roman In-Reply-To: <20050825092957.GA49525@stud.fit.vutbr.cz> References: <1168719770.20050824183357@adeon.lublin.pl> <20050825092957.GA49525@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2005 21:12:59 -0000 > cannot this be related to low value of nswbufs (fixed in curennt and I think > releng_6)? Do you mean http://www.freebsd.org/cgi/query-pr.cgi?pr=84903 or sth else? Jarek G From owner-freebsd-performance@FreeBSD.ORG Thu Aug 25 22:46:35 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F47916A41F for ; Thu, 25 Aug 2005 22:46:35 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 766D343D5A for ; Thu, 25 Aug 2005 22:46:32 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bhs170.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.108.170) by matrix01.home.net.pl with SMTP; Thu, 25 Aug 2005 22:46:28 -0000 Date: Fri, 26 Aug 2005 00:46:27 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <1595762165.20050826004627@adeon.lublin.pl> To: freebsd-performance@freebsd.org In-Reply-To: <20050825092957.GA49525@stud.fit.vutbr.cz> References: <1168719770.20050824183357@adeon.lublin.pl> <20050825092957.GA49525@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2005 22:46:35 -0000 > Unpacking it on FreeBSD5 gives me such results: > > # time tar -xf mysql-m.tgz > 2.130u 20.187s 7:02.69 5.2% 41+382k 13097+8205io 0pf+0w > ...so 7 minutes of real time. I've checked the same file on *different* server, but it has very similar hardware (not Dell, but also Dual Xeon 2.8, SCSI, 2GB ram, etc...). It was on FreeBSD 4.10. # time tar -xf mysql-m.tgz 0.244u 33.724s 4:25.26 12.8% 374+291k 14216+1828io 8pf+0w During this test server was loaded (load averange about 1.00, Apache, MySQL running). I wasn't able to turn deamons off but I suspect this result could be much better than it was. So is my problem only FreeBSD5 releated? Seems to be. JG From owner-freebsd-performance@FreeBSD.ORG Fri Aug 26 12:11:49 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CD2316A41F for ; Fri, 26 Aug 2005 12:11:49 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from v00061.home.net.pl (list.pl [212.85.96.61]) by mx1.FreeBSD.org (Postfix) with SMTP id 0C2C143D48 for ; Fri, 26 Aug 2005 12:11:47 +0000 (GMT) (envelope-from jarek@adeon.lublin.pl) Received: from bhq206.neoplus.adsl.tpnet.pl (jarek.list@home@83.28.106.206) by matrix01.home.net.pl with SMTP; Fri, 26 Aug 2005 12:11:42 -0000 Date: Fri, 26 Aug 2005 14:11:39 +0200 From: JG X-Mailer: The Bat! (v3.0) Professional X-Priority: 3 (Normal) Message-ID: <113976400.20050826141139@adeon.lublin.pl> To: Divacky Roman In-Reply-To: <20050826093209.GA17465@stud.fit.vutbr.cz> References: <1168719770.20050824183357@adeon.lublin.pl> <20050825092957.GA49525@stud.fit.vutbr.cz> <1887930059.20050825231254@adeon.lublin.pl> <20050826093209.GA17465@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re[2]: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: JG List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2005 12:11:49 -0000 >> Do you mean http://www.freebsd.org/cgi/query-pr.cgi?pr=84903 or sth >> else? > yes, this I patched my kernel as described but It has no any effect on untarring performance. Anyway thank you for your hint, I belive this can help in tuning other things. # time gtar -xf mysql-m.tgz 1.207u 16.371s 7:01.06 4.1% 164+278k 8030+7176io 4pf+0w JG From owner-freebsd-performance@FreeBSD.ORG Sat Aug 27 21:39:26 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B3EE16A41F for ; Sat, 27 Aug 2005 21:39:26 +0000 (GMT) (envelope-from pavelkraynyukhov@mail.ru) Received: from mx3.mail.ru (mx3.mail.ru [194.67.23.149]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A5C643D45 for ; Sat, 27 Aug 2005 21:39:26 +0000 (GMT) (envelope-from pavelkraynyukhov@mail.ru) Received: from [82.83.224.40] (port=12912 helo=ChainOfWorlds) by mx3.mail.ru with asmtp id 1E98Oa-000KHr-00 for freebsd-performance@freebsd.org; Sun, 28 Aug 2005 01:39:24 +0400 From: Pavel Kraynyukhov To: freebsd-performance@freebsd.org In-Reply-To: <20050827120024.07BC516A420@hub.freebsd.org> References: <20050827120024.07BC516A420@hub.freebsd.org> Content-Type: text/plain; charset=koi8-r Date: Sat, 27 Aug 2005 23:39:14 +0200 Message-Id: <1125178755.6487.2.camel@ChainOfWorlds.InstantHQ.COM> Mime-Version: 1.0 X-Mailer: Evolution 2.2.3 Content-Transfer-Encoding: 8bit Subject: Re: freebsd-performance Digest, Vol 30, Issue 16 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2005 21:39:26 -0000 Have you tried to disable any optimizations in CFLAGS and CXXFLAGS of /etc/make.conf and rebuild the system ? I had bad expirience with it (enabled optimization options) - fairly downgrade of performance on my desktop ... ÷ óÂÔ, 27/08/2005 × 12:00 +0000, freebsd-performance-request@freebsd.org ÐÉÛÅÔ: > Send freebsd-performance mailing list submissions to > freebsd-performance@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > or, via email, send a message with subject or body 'help' to > freebsd-performance-request@freebsd.org > > You can reach the person managing the list at > freebsd-performance-owner@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-performance digest..." > > > Today's Topics: > > 1. Re[2]: slow tar performance on fbsd5 (JG) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 26 Aug 2005 14:11:39 +0200 > From: JG > Subject: Re[2]: slow tar performance on fbsd5 > To: Divacky Roman > Cc: freebsd-performance@freebsd.org > Message-ID: <113976400.20050826141139@adeon.lublin.pl> > Content-Type: text/plain; charset=us-ascii > > >> Do you mean http://www.freebsd.org/cgi/query-pr.cgi?pr=84903 or sth > >> else? > > yes, this > > I patched my kernel as described but It has no any effect on untarring performance. > Anyway thank you for your hint, I belive this can help in tuning other things. > > # time gtar -xf mysql-m.tgz > 1.207u 16.371s 7:01.06 4.1% 164+278k 8030+7176io 4pf+0w > > > JG > > > > > > ------------------------------ > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > End of freebsd-performance Digest, Vol 30, Issue 16 > *************************************************** >