From owner-freebsd-performance@FreeBSD.ORG Wed Aug 24 18:27:48 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1299A16A41F for ; Wed, 24 Aug 2005 18:27:48 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: from smtp107.sbc.mail.mud.yahoo.com (smtp107.sbc.mail.mud.yahoo.com [68.142.198.206]) by mx1.FreeBSD.org (Postfix) with SMTP id 7D3FE43D46 for ; Wed, 24 Aug 2005 18:27:47 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: (qmail 45448 invoked from network); 24 Aug 2005 18:27:46 -0000 Received: from unknown (HELO optimator.noacks.org) (noacks@swbell.net@70.240.231.0 with login) by smtp107.sbc.mail.mud.yahoo.com with SMTP; 24 Aug 2005 18:27:46 -0000 Received: from localhost (localhost [127.0.0.1]) by optimator.noacks.org (Postfix) with ESMTP id 42CCF60EB; Wed, 24 Aug 2005 13:27:45 -0500 (CDT) Received: from optimator.noacks.org ([127.0.0.1]) by localhost (optimator.noacks.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 07464-16; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) Received: from compgeek.noacks.org (compgeek [192.168.1.10]) by optimator.noacks.org (Postfix) with ESMTP id D2FA060E8; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) by compgeek.noacks.org (8.13.4/8.13.4) with ESMTP id j7OIRf0q002347; Wed, 24 Aug 2005 13:27:42 -0500 (CDT) (envelope-from noackjr@alumni.rice.edu) Message-ID: <430CBC18.6040902@alumni.rice.edu> Date: Wed, 24 Aug 2005 13:27:36 -0500 From: Jonathan Noack User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050727) X-Accept-Language: en-us, en MIME-Version: 1.0 To: JG References: <1168719770.20050824183357@adeon.lublin.pl> In-Reply-To: <1168719770.20050824183357@adeon.lublin.pl> X-Enigmail-Version: 0.92.0.0 OpenPGP: id=991D8195; url=http://www.noacks.org/cert/noackjr.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigAAFC06154F4DB0DF41AB9E66" X-Virus-Scanned: amavisd-new at noacks.org Cc: freebsd-performance@freebsd.org Subject: Re: slow tar performance on fbsd5 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: noackjr@alumni.rice.edu List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 18:27:48 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigAAFC06154F4DB0DF41AB9E66 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 08/24/05 11:33, JG wrote: > I had to unpack a lot of tar archives and I occasional noticed terrible > bad performance on freebsd5. > > This is my test file: > > 854251520 24 Sie 12:13 mysql-m.tgz > > There are some real MySQL tables in it, it has been done with tar > -cvf. This archive contains about 146.000 small files. > > --------------------------------------- > Unpacking it on FreeBSD5 gives me such results: > > # time tar -xf mysql-m.tgz > 2.130u 20.187s 7:02.69 5.2% 41+382k 13097+8205io 0pf+0w > ...so 7 minutes of real time. > > This is today's FreeBSD 5.4-STABLE but I also tried 5.4-release. > > That server is brand new Dell PE2850 with Seagate ST373207L SCSI > drive, no raid. Parition is default UFS2 mounted with noatime, softupdates on. Will noatime make a difference when unpacking a tar archive (assuming an otherwise idle system, at least)? My understanding of atime is that it might slow down the disk for later accesses due to atime writes, but when creating files it shouldn't have any effect. Is that not correct? > This is Dual Xeon 2.8, 2GB ram. > > My sysctls: > vfs.ufs.dirhash_maxmem=16777216 (much better than default 2MB) > machdep.hyperthreading_allowed=1 (better dd results) Your other settings appear ok, but I'd turn off hyperthreading. Almost every FreeBSD/HT test has shown that it reduces performance because the scheduler is not HT-aware. When the system is relatively idle (single dd running, for example), it might not pessimize things, but it will most likely slow you down under load. > kern.maxfiles=65536 > kern.maxfilesperproc=65536 > vfs.read_max=16 > > > > ---------------------------------------- > This is result from Gentoo Linux on 2.6.x hardened kernel: > # time tar -xf mysql-m.tgz > > real 1m3.944s > user 0m1.702s > sys 0m15.794s > > Only ~one minute! Six times faster than on a FreeBSD. I'm not a linux > fan, and I don't want to tell you how good linux is, but I would to > find out what causes such bad results on my favourite FreeBSD... This sounds like you're running into the old "lemming syncer" problem. There is currently some work on disk schedulers (even a Summer of Code project), but it will most likely not make it into 5.x. -- Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195 --------------enigAAFC06154F4DB0DF41AB9E66 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFDDLwdUFz01pkdgZURAm35AJ9cLy4yFsZVOddNKWhx+P8xa+L35wCglRkt VcQCEMX22H2TqJbzfm9oTBk= =Zdqt -----END PGP SIGNATURE----- --------------enigAAFC06154F4DB0DF41AB9E66--