From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 07:01:32 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 568FB16A41F for ; Sun, 7 Aug 2005 07:01:32 +0000 (GMT) (envelope-from josh.carroll@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB23F43D6B for ; Sun, 7 Aug 2005 07:01:31 +0000 (GMT) (envelope-from josh.carroll@gmail.com) Received: by wproxy.gmail.com with SMTP id i8so818104wra for ; Sun, 07 Aug 2005 00:01:30 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=NnLQ09ryQktNvZKk2hjQkiKMAp/xpZ+MyRL3eMidWngI2sQlfKMDyxF7wbGd94QYm/fjPKKYxJlMBVQVPLZsgGFjp8142FEWlFPnlOexgy/2UYsa3dbhf0z4pKIQhD311uiGOMtirTa5BrXGfQhPVa9Nnj93rOaavzpkMidnnG4= Received: by 10.54.46.30 with SMTP id t30mr3984691wrt; Sun, 07 Aug 2005 00:01:30 -0700 (PDT) Received: by 10.54.14.51 with HTTP; Sun, 7 Aug 2005 00:01:30 -0700 (PDT) Message-ID: <8cb6106e05080700016c4a065a@mail.gmail.com> Date: Sun, 7 Aug 2005 00:01:30 -0700 From: Josh Carroll To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: disk read vs. write performance in 5.4-RELEASE-p6 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: josh.carroll@psualum.com List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 07:01:32 -0000 Hello, After reading man tuning, I began poking around at my IDE drives to see how their performance was in FreeBSD. I noticed that writes are quite slow (on the order of 15MB/s) compared to reads (55MB/s). In some initial googling, I saw a thread from early 2005 about 5.3 and performance problems. I ran the same dd test to check if it corroborated my results and sure enough it did. Before I get to the results, the box is a Pentium 4 3.0 Ghz on an Intel 865PERL motherboard running 5.4-RELEASE-p6. The kernel is a custom kernel, but the results are the same with my kernel and with GENERIC. The IDE drives in question (all PATA) are two 80GB drives (one 7200 rpm, one 5400 rpm) and a 120GB 7200rpm drive. The 80GB 7200rpm drive (ad0) is the primary master, the 80GB 5400rpm drive (ad1) is the primary slave. The 120GB drive (ad2) is the secondary master, with a DVD burner as the secondary slave. Here are the dd results: ad0(write): 1073741824 bytes transferred in 67.897021 secs (15814270 bytes/= sec) ad0(read): 1073741824 bytes transferred in 19.861379 secs (54061796 bytes/s= ec) ad1(write): 1073741824 bytes transferred in 73.247236 secs (14659145 bytes/= sec) ad1(read): 1073741824 bytes transferred in 26.235017 secs (40927811 bytes/s= ec) ad2(write): 1073741824 bytes transferred in 66.088393 secs (16247056 bytes/= sec) ad2(read): 1073741824 bytes transferred in 21.449289 secs (50059553 bytes/s= ec) The read numbers are right about where I would expect, and the 5400 rpm drive is slightly slower as I'd expect. However, all there drives write at roughly the same speed. All around 15MB/s. And that's the crux of the problem. I'm pretty sure this isn't "normal", as the numbers look rather low. I had a few others test their IDE drives and they reported at least 30MB/s. syctl values look kosher: # sysctl hw.ata hw.ata.atapi_dma: 1 hw.ata.wc: 1 hw.ata.ata_dma: 1 As does the atacontrol output: # atacontrol mode 0 Master =3D UDMA100 Slave =3D UDMA100 # atacontrol mode 1 Master =3D UDMA100 Slave =3D UDMA33 # atacontrol info 0 Master: ad0 ATA/ATAPI revision 6 Slave: ad1 ATA/ATAPI revision 5 # atacontrol info 1 Master: ad2 ATA/ATAPI revision 6 Slave: acd0 ATA/ATAPI revision 5 Finally, here is my dmesg output: Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 =09The Regents of the University of California. All rights reserved. FreeBSD 5.4-RELEASE-p6 #3: Thu Aug 4 23:53:33 PDT 2005 root@deblin.org:/usr/src/sys/i386/compile/DEBLIN Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (3000.12-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf41 Stepping =3D 1 Features=3D0xbfebfbff Hyperthreading: 2 logical CPUs real memory =3D 1073676288 (1023 MB) avail memory =3D 1036939264 (988 MB) ACPI APIC Table: ioapic0 irqs 0-23 on motherboard npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Overriding SCI Interrupt from IRQ 9 to IRQ 20 acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 acpi_throttle0: on cpu0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 nvidia0: mem 0xf7d80000-0xf7dfffff,0xe8000000-0xefffffff,0xfd000000-0xfdffffff irq 16 at device 0.0 on pci1 uhci0: port 0xcc00-0xcc1f irq 16 at device 29.0 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd000-0xd01f irq 19 at device 29.1 on pci0 usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xd400-0xd41f irq 18 at device 29.2 on pci0 usb2: on uhci2 usb2: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xd800-0xd81f irq 16 at device 29.3 on pci0 usb3: on uhci3 usb3: USB revision 1.0 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xfebff800-0xfebffbff irq 23 at device 29.7 on pci0 usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered pcib2: at device 30.0 on pci0 pci2: on pcib2 em0: port 0xbc00-0xbc3f mem 0xfeac0000-0xfeadffff,0xfeae0000-0xfeafffff irq 17 at device 1.0 on pci2 em0: Ethernet address: 00:0e:0c:6c:b9:16 em0: Speed:N/A Duplex:N/A fxp0: port 0xb800-0xb81f mem 0xfe900000-0xfe9fffff,0xf7eff000-0xf7efffff irq 18 at device 2.0 on pci2 miibus0: on fxp0 nsphy0: on miibus0 nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:a0:c9:1f:ac:f8 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xfc00-0xfc0f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 pci0: at device 31.3 (no driver attached) atkbdc0: port 0x64,0x60 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 psm0: irq 12 on atkbdc0 psm0: model IntelliMouse, device ID 3 fdc0: port 0x3f7,0x3f4-0x3f5,0x3f2-0x3f3 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acp= i0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x3e8-0x3ef irq 4 on acpi0 sio1: type 16550A pmtimer0 on isa0 orm0: at iomem 0xe0000-0xe0fff,0xce800-0xcf7ff,0xc0000-0xce7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: parallel port not found. uhid0: APC Back-UPS ES 500 FW:801.e5.D USB FW:e5, rev 1.10/1.06, addr 2, iclass 3/0 Timecounter "TSC" frequency 3000123473 Hz quality 800 Timecounters tick every 10.000 msec ad0: 76319MB [155061/16/63] at ata0-master UDMA100 ad1: 76319MB [155061/16/63] at ata0-slave UDMA100 ad2: 152627MB [310101/16/63] at ata1-master UDMA100 acd0: CDRW at ata1-slave UDMA33 cd0 at ata1 bus 0 target 1 lun 0 cd0: Removable CD-ROM SCSI-0 device=20 cd0: 33.000MB/s transfers cd0: cd present [348875 x 2048 byte records] Thanks in advance! Josh From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 01:49:58 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6916D16A41F for ; Sun, 7 Aug 2005 01:49:55 +0000 (GMT) (envelope-from pfgshield-freebsd@yahoo.com) Received: from web51610.mail.yahoo.com (web51610.mail.yahoo.com [206.190.38.215]) by mx1.FreeBSD.org (Postfix) with SMTP id 7037043F32 for ; Sun, 7 Aug 2005 01:18:50 +0000 (GMT) (envelope-from pfgshield-freebsd@yahoo.com) Received: (qmail 77472 invoked by uid 60001); 7 Aug 2005 01:18:49 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=ttuuGz/vLrv+/1K/+SnDaBTZwWpqy03YfXowWNt4A4qR0ZR+wy2l/i8eAeFaLi0ztk6PPnHFgKhAbI3Z3uvm0+/GnYvf67W4N3EXq0hFGePitR3V4+RAo/joAUJT862o7vt8N2WEWOmGOkfS5FhtzKyqQ13H19Gl9vmpLXkV+JM= ; Message-ID: <20050807011849.77470.qmail@web51610.mail.yahoo.com> Received: from [208.30.41.90] by web51610.mail.yahoo.com via HTTP; Sun, 07 Aug 2005 03:18:49 CEST Date: Sun, 7 Aug 2005 03:18:49 +0200 (CEST) From: To: freebsd-performance@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 X-Mailman-Approved-At: Sun, 07 Aug 2005 14:47:51 +0000 Cc: Subject: LibMicro: Portable Microbenchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pfgshield-freebsd@yahoo.com List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 01:49:58 -0000 Hi guys; This looks pretty interesting... "LibMicro is a portable set of microbenchmarks that many Solaris engineers used during Solaris 10 development to measure the performance of various system and library calls. LibMicro was developed by Bart Smaalders and Phil Harman as part of their If another OS is faster it's a Solaris bug performance campaign. LibMicro is being released as Open Source under the CDDL license. The libMicro sources include an awk script that generates color coded html pages to easily compare various OS revs, processors and other differences in a quick manner." http://www.opensolaris.org/os/community/performance/libmicro/ ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 18:47:18 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E449716A41F for ; Sun, 7 Aug 2005 18:47:18 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B17043D46 for ; Sun, 7 Aug 2005 18:47:18 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from beastie.frontfree.net (unknown [219.239.99.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 24D3EEB4F73 for ; Mon, 8 Aug 2005 02:47:15 +0800 (CST) Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id BFCF81351B6 for ; Mon, 8 Aug 2005 02:47:13 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 61558-14 for ; Mon, 8 Aug 2005 02:47:07 +0800 (CST) Received: by beastie.frontfree.net (Postfix, from userid 1001) id 23F28131F9D; Mon, 8 Aug 2005 02:47:07 +0800 (CST) Date: Mon, 8 Aug 2005 02:47:07 +0800 From: Xin LI To: freebsd-performance@FreeBSD.org Message-ID: <20050807184707.GA61714@frontfree.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Q68bSM7Ycu6FN28Q" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-GPG-key-ID/Fingerprint: 0xCAEEB8C0 / 43B8 B703 B8DD 0231 B333 DC28 39FB 93A0 CAEE B8C0 X-GPG-Public-Key: http://www.delphij.net/delphij.asc X-Operating-System: FreeBSD beastie.frontfree.net 5.4-RELEASE-p6 FreeBSD 5.4-RELEASE-p6 #4: Thu Jul 28 10:59:26 CST 2005 delphij@beastie.frontfree.net:/usr/obj/usr/src/sys/BEASTIE i386 X-URL: http://www.delphij.net X-By: delphij@beastie.frontfree.net X-Location: Beijing, China X-Virus-Scanned: amavisd-new at frontfree.net Cc: Subject: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 18:47:19 -0000 --Q68bSM7Ycu6FN28Q Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Bcc'ed to -developers@, so this can be discussed in a public list] Hi, It seems that vfs.ufs.dirhash_maxmem is set to 2MB. I think this value is slightly too small for modern machines: - There are many applications that relies on small files. CVS, maildir, etc. For these applications a typical need of dirhash would be much larger than 2MB. - The RAM equiped with modern computers are growing fast. - Increasing dirhash_maxmem does not bring too much overhead on small systems, as the system would automatically recycle unused entities. If the memory was not freed in time, then it is usually because the system is busy accessing a zillion of small files. Moreover, it is possible for the user to change the default value back to a smaller value if the file indexing is not their performance bottleneck. My proposal is to increase the default dirhash_maxmem value to at least 32MB or 64MB. Any objections? Cons for this, discussed in -developer: - dirhash does not implements automatical mechanism to reduce memory usage in response to system memory pressure, and benefits mainly to large directories, e.g. Maildirs. Cheers, --=20 Xin LI http://www.delphij.net/ See complete headers for GPG key and other information. --Q68bSM7Ycu6FN28Q Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFC9lcr/cVsHxFZiIoRAqjnAJ9Mh58qF+USy9KM/2EslptVZvuFawCfaqQO tnyAc8m/HEwtCG6hGAfaGlE= =/cHX -----END PGP SIGNATURE----- --Q68bSM7Ycu6FN28Q-- From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 18:57:47 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 61A4B16A41F for ; Sun, 7 Aug 2005 18:57:47 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0AC2943D48 for ; Sun, 7 Aug 2005 18:57:47 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 71C295CBA; Sun, 7 Aug 2005 14:57:46 -0400 (EDT) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 02474-10; Sun, 7 Aug 2005 14:57:45 -0400 (EDT) Received: from [192.168.1.3] (pool-68-161-79-217.ny325.east.verizon.net [68.161.79.217]) by pi.codefab.com (Postfix) with ESMTP id 157EE5C68; Sun, 7 Aug 2005 14:57:44 -0400 (EDT) Message-ID: <42F659AE.80905@mac.com> Date: Sun, 07 Aug 2005 14:57:50 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050801 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Xin LI References: <20050807184707.GA61714@frontfree.net> In-Reply-To: <20050807184707.GA61714@frontfree.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-performance@FreeBSD.org Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 18:57:47 -0000 Xin LI wrote: > It seems that vfs.ufs.dirhash_maxmem is set to 2MB. I think this value > is slightly too small for modern machines: [ ... ] > My proposal is to increase the default dirhash_maxmem value to at least > 32MB or 64MB. Any objections? You are undoubtedly right that allocating only 2MB for dirhash on a modern machine which has, for example, 1GB of RAM is too small. On the other hand, I've got several firewall boxes with only 128MB, and it's not reasonable to simply dedicate up to 64MB (half!) to dirhash without paying more attention to the amount of physical memory that is actually available. How big should dirhash_maxmem be? 5-10% of available RAM, perhaps? -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 19:05:51 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD1E916A41F for ; Sun, 7 Aug 2005 19:05:51 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 22F2443D48 for ; Sun, 7 Aug 2005 19:05:51 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from beastie.frontfree.net (unknown [219.239.99.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 1BD9BEB10D0 for ; Mon, 8 Aug 2005 03:05:47 +0800 (CST) Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id 0A8BB135B01; Mon, 8 Aug 2005 03:05:43 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 61932-05; Mon, 8 Aug 2005 03:05:34 +0800 (CST) Received: by beastie.frontfree.net (Postfix, from userid 1001) id C14FD134E7F; Mon, 8 Aug 2005 03:05:33 +0800 (CST) Date: Mon, 8 Aug 2005 03:05:33 +0800 From: Xin LI To: Chuck Swiger Message-ID: <20050807190533.GA61954@frontfree.net> References: <20050807184707.GA61714@frontfree.net> <42F659AE.80905@mac.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline In-Reply-To: <42F659AE.80905@mac.com> User-Agent: Mutt/1.4.2.1i X-GPG-key-ID/Fingerprint: 0xCAEEB8C0 / 43B8 B703 B8DD 0231 B333 DC28 39FB 93A0 CAEE B8C0 X-GPG-Public-Key: http://www.delphij.net/delphij.asc X-Operating-System: FreeBSD beastie.frontfree.net 5.4-RELEASE-p6 FreeBSD 5.4-RELEASE-p6 #4: Thu Jul 28 10:59:26 CST 2005 delphij@beastie.frontfree.net:/usr/obj/usr/src/sys/BEASTIE i386 X-URL: http://www.delphij.net X-By: delphij@beastie.frontfree.net X-Location: Beijing, China X-Virus-Scanned: amavisd-new at frontfree.net Cc: freebsd-performance@FreeBSD.org Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 19:05:51 -0000 --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 07, 2005 at 02:57:50PM -0400, Chuck Swiger wrote: [snip] > On the other hand, I've got several firewall boxes with only 128MB, and= =20 > it's not reasonable to simply dedicate up to 64MB (half!) to dirhash=20 > without paying more attention to the amount of physical memory that is=20 > actually available. Forgot to mention that dirhash_maxmem is the upper limit, not the exact amount that dirhash must use. If your firewall does not host zillion of large directories then a bigger default dirhash_maxmem would essentially a big no-op :-) > How big should dirhash_maxmem be? 5-10% of available RAM, perhaps? Actually this depends on how many large directories that you actively access, not only the available RAM. If you access only a little of small directories, e.g. hosting a database, then dirhash won't help much. But with a lot of large directories having 1000+ files in most of them, then raising the dirhash_maxmem will be a great help for performance. I think it would be great if we can implement a automatical "suggested value" of dirhash during bootstrap stage, though, which can be overridden through sysctl.conf or subsequent sysctl. Cheers, --=20 Xin LI http://www.delphij.net/ See complete headers for GPG key and other information. --Kj7319i9nmIyA2yE Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFC9lt9/cVsHxFZiIoRAm/kAJ9UZLf+FAk55+YA5BcEs3Su5IZTAACfRwzS ydUrVCocULNSkEkdUBlk2Ps= =eame -----END PGP SIGNATURE----- --Kj7319i9nmIyA2yE-- From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 16:16:21 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 984E416A41F for ; Sun, 7 Aug 2005 16:16:21 +0000 (GMT) (envelope-from lofi@freebsd.org) Received: from mail-in-05.arcor-online.net (mail-in-05.arcor-online.net [151.189.21.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1EAD343D45 for ; Sun, 7 Aug 2005 16:16:21 +0000 (GMT) (envelope-from lofi@freebsd.org) Received: from mail-in-03-z2.arcor-online.net (mail-in-03-z2.arcor-online.net [151.189.8.15]) by mail-in-05.arcor-online.net (Postfix) with ESMTP id CE4C960B12; Sun, 7 Aug 2005 18:16:19 +0200 (CEST) Received: from mail-in-03.arcor-online.net (mail-in-06.arcor-online.net [151.189.21.46]) by mail-in-03-z2.arcor-online.net (Postfix) with ESMTP id BF30C17E354; Sun, 7 Aug 2005 18:16:19 +0200 (CEST) Received: from lofi.dyndns.org (dsl-213-023-198-171.arcor-ip.net [213.23.198.171]) by mail-in-03.arcor-online.net (Postfix) with ESMTP id 71A62347EF; Sun, 7 Aug 2005 18:16:19 +0200 (CEST) Received: from [192.168.8.4] (kiste.my.domain [192.168.8.4]) (authenticated bits=0) by lofi.dyndns.org (8.13.3/8.13.3) with ESMTP id j77GGGh7046229 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 7 Aug 2005 18:16:17 +0200 (CEST) (envelope-from lofi@freebsd.org) Message-ID: <42F633D0.6090407@freebsd.org> Date: Sun, 07 Aug 2005 18:16:16 +0200 From: Michael Nottebrock User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716) X-Accept-Language: de-DE, de, en-us, en MIME-Version: 1.0 To: pfgshield-freebsd@yahoo.com References: <20050807011849.77470.qmail@web51610.mail.yahoo.com> In-Reply-To: <20050807011849.77470.qmail@web51610.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new X-Mailman-Approved-At: Sun, 07 Aug 2005 20:13:27 +0000 Cc: freebsd-performance@freebsd.org Subject: Re: LibMicro: Portable Microbenchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 16:16:21 -0000 pfgshield-freebsd@yahoo.com schrieb: > Hi guys; > > This looks pretty interesting... > > "LibMicro is a portable set of microbenchmarks that many Solaris engineers used > [...] You mean like the libmicro mentioned in http://lists.freebsd.org/pipermail/freebsd-performance/2005-August/001445.html ? :-) -- ,_, | Michael Nottebrock | lofi@freebsd.org (/^ ^\) | FreeBSD - The Power to Serve | http://www.freebsd.org \u/ | K Desktop Environment on FreeBSD | http://freebsd.kde.org From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 20:24:48 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 779DE16A423 for ; Sun, 7 Aug 2005 20:24:48 +0000 (GMT) (envelope-from dwmalone@maths.tcd.ie) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by mx1.FreeBSD.org (Postfix) with SMTP id C5E6343E30 for ; Sun, 7 Aug 2005 19:24:21 +0000 (GMT) (envelope-from dwmalone@maths.tcd.ie) Received: from walton.maths.tcd.ie ([134.226.81.10] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 7 Aug 2005 20:24:20 +0100 (BST) Date: Sun, 7 Aug 2005 20:24:20 +0100 From: David Malone To: freebsd-performance@FreeBSD.org Message-ID: <20050807192420.GA83026@walton.maths.tcd.ie> References: <20050807185129.GA61807@frontfree.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050807185129.GA61807@frontfree.net> User-Agent: Mutt/1.5.6i Sender: dwmalone@maths.tcd.ie Cc: Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 20:24:48 -0000 On Mon, Aug 08, 2005 at 02:51:29AM +0800, Xin LI wrote: > My proposal is to increase the default dirhash_maxmem value to at least > 32MB or 64MB. Any objections? I think autotuning the value on boot might be a good idea, providing that there's reasonable evidence that the existing value is too small. Maybe setting it to some fraction of the available ram might be a reasonable plan. I set it to 20MB on our NFS server which has 1GB of ram. That's more than enough enough to quickly clean up when a user accidently puts a million files in one directory. > Cons for this, discussed in -developer: > - dirhash does not implements automatical mechanism to reduce memory > usage in response to system memory pressure, and benefits mainly to > large directories, e.g. Maildirs. Note that it benefits *only* large directories. Making dirhash respond to demands on memory would be a good plan, certainly on -current where we can in principle get feedback from UMA about memory shortages. This would eliminate most of the disadvantages of increasing the limit. In fact, you might be able to remove the limit all together. David. From owner-freebsd-performance@FreeBSD.ORG Sun Aug 7 21:43:38 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: by hub.freebsd.org (Postfix, from userid 680) id 5C12716A420; Sun, 7 Aug 2005 21:43:38 +0000 (GMT) Date: Sun, 7 Aug 2005 21:43:38 +0000 From: Darren Reed To: freebsd-performance@FreeBSD.org Message-ID: <20050807214338.GA34438@hub.freebsd.org> References: <20050807185129.GA61807@frontfree.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050807185129.GA61807@frontfree.net> User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Mon, 08 Aug 2005 00:54:30 +0000 Cc: Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2005 21:43:38 -0000 On Mon, Aug 08, 2005 at 02:51:29AM +0800, Xin LI wrote: ... > My proposal is to increase the default dirhash_maxmem value to at least > 32MB or 64MB. Any objections? > > Cons for this, discussed in -developer: > - dirhash does not implements automatical mechanism to reduce memory > usage in response to system memory pressure, and benefits mainly to > large directories, e.g. Maildirs. So why not make the determination of "dirhash_maxmem" the result of some calculation(s) that takes into account RAM size, etc ? The obvious lesson here is that picking a number to be a limit based on the current size of machines fails the test of time. Darren From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 01:16:18 2005 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8434F16A41F; Mon, 8 Aug 2005 01:16:18 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from efnet-math.org (efnet-math.org [69.60.109.125]) by mx1.FreeBSD.org (Postfix) with ESMTP id C629C43DA0; Mon, 8 Aug 2005 01:16:14 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from [151.24.152.46] (ppp-46-152.24-151.libero.it [151.24.152.46]) (authenticated bits=0) by efnet-math.org (8.13.1/8.13.1) with ESMTP id j781G9Bk025826 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO); Sun, 7 Aug 2005 21:16:12 -0400 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v733) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <01F3BA1C-C7C6-41C7-AFE8-675FA972D1A3@FreeBSD.org> Content-Transfer-Encoding: 7bit From: Suleiman Souhlal Date: Mon, 8 Aug 2005 03:15:59 +0200 To: gnn@FreeBSD.org X-Mailer: Apple Mail (2.733) Cc: performance@FreeBSD.org Subject: Re: Tarball of ported libmicro 0.3 available for testing... X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 01:16:18 -0000 Hello, On Aug 6, 2005, at 3:25 PM, gnn@freebsd.org wrote: > I plan to make a port of this this weekend, but would like some > feedback on this set of benchmarks. If they're useful I think we > should make them part of a nightly benchmarking strategy. In case you're interested, I ran it on a dual p4 xeon (without HyperThreading) from the netperf cluster, to compare the performance of RELENG_5, RELENG_6 and HEAD. You can find the results at http://people.freebsd.org/~ssouhlal/stuff/ compare_tiger-3.html . It shows that RELENG_6 and HEAD are (in these tests) almost never slower than RELENG_5, and often more than 20% faster. -- Suleiman Souhlal | ssouhlal@vt.edu The FreeBSD Project | ssouhlal@FreeBSD.org From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 01:26:40 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C9E716A421 for ; Mon, 8 Aug 2005 01:26:40 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (ppp166-27.static.internode.on.net [150.101.166.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 749DF43D48 for ; Mon, 8 Aug 2005 01:26:39 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (localhost [127.0.0.1]) by vault.mel.jumbuck.com (Postfix) with ESMTP id 9C72A8A02D; Mon, 8 Aug 2005 11:26:03 +1000 (EST) Received: from [192.168.46.52] (unknown [192.168.46.52]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by vault.mel.jumbuck.com (Postfix) with ESMTP id 8156F8A028; Mon, 8 Aug 2005 11:26:03 +1000 (EST) Message-ID: <42F6B4CD.5030703@roq.com> Date: Mon, 08 Aug 2005 11:26:37 +1000 From: Michael VInce User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.10) Gecko/20050802 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Claus Guttesen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-performance@freebsd.org Subject: Re: changing max_connections in postgresql on FreeBSD 5.4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 01:26:40 -0000 Hi, I have done all my Postgres optimization configuration via sysctl or the postgresql.conf no kernel recompilation was performed. I did benchmarks at complete default FreeBSD / Postgres configuration and benchmarks after. I found raising the values to probably not much more then 1/4 of what they are now list below gave around 30% increase in performance. But increase the numbers after that did nothing. I still left them way past the mark anyway because this server has 4gigs of ram that would other wise go to waste. The Postgres DB setup here is only meant to handle a few connections at a time unlike yours. sysctl.conf kern.ipc.shmmax=1073741824 # x32 the default ( 8192 ) kern.ipc.shmall=262144 postgresql.conf # shared_buffers = 108000 32 x default = 884,736,000 bytes shared_buffers = 108000 # min 16, at least max_connections*2, 8KB each work_mem = 65536 # min 64, size in KB max_stack_depth = 4096 # min 100, size in KB max_fsm_pages = 40000 # min max_fsm_relations*16, 6 bytes each max_fsm_relations = 2000 # min 100, ~50 bytes each Also note that I have a MySQL server with a similar table setup and it performs a lot better. Claus Guttesen wrote: >Hi. > >I recently lowered max_connections from 1024 to 384 in >/usr/local/pgsql/data/postgresql.conf. The server is a quad opteron @ >2 GHz and 4 GB of RAM. > >This decreased the SIZE and RES values in top and it seems that the >current max_connections is more than adequate. To see how many >concurrent connections the db-server had I did a 'netstat -na|grep -i >establi|wc -l' which showed some 230 established connections during >peak. > >In order to use max_connections set to 1024 I had to raise the >following values in the kernel: > >options SHMMAXPGS=327680 >options SEMMNI=200 >options SEMMNS=1200 >options SEMUME=200 >options SEMMNU=600 > >The values were taken from the postgresql-ports-installation-notes and >were simply increased to fit the max_connections-parameter in >postgresql.conf. > >Will more RAM become available to postgresql if I change the >kernel-values mentioned above to half the size? > >The server does occasionally become loaded with load-averages around 6 >or 7 but usually stays below 4. Disk-io is not a problem, usually >remains below 1 MB/s, cpu-idle is approx. 5-10 % during peak and >around 80 % idle otherwise. > >regards >Claus >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 02:30:38 2005 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3CDD116A431; Mon, 8 Aug 2005 02:30:38 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BDEC43E0E; Mon, 8 Aug 2005 02:01:37 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: from beastie.frontfree.net (unknown [219.239.99.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 389F3EB4F35; Mon, 8 Aug 2005 10:01:34 +0800 (CST) Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id C51021391FD; Mon, 8 Aug 2005 10:01:32 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 70526-12; Mon, 8 Aug 2005 10:01:24 +0800 (CST) Received: from [10.217.12.83] (unknown [61.135.152.194]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by beastie.frontfree.net (Postfix) with ESMTP id 7F891139173; Mon, 8 Aug 2005 10:01:23 +0800 (CST) From: Xin LI To: Suleiman Souhlal In-Reply-To: <01F3BA1C-C7C6-41C7-AFE8-675FA972D1A3@FreeBSD.org> References: <01F3BA1C-C7C6-41C7-AFE8-675FA972D1A3@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-K19JSnonNJgCgtZ+X8tQ" Organization: The FreeBSD Simplified Chinese Project Date: Mon, 08 Aug 2005 10:01:17 +0800 Message-Id: <1123466477.767.2.camel@spirit> Mime-Version: 1.0 X-Mailer: Evolution 2.2.3 FreeBSD GNOME Team Port X-Virus-Scanned: amavisd-new at frontfree.net Cc: gnn@FreeBSD.org, performance@FreeBSD.org Subject: Re: Tarball of ported libmicro 0.3 available for testing... X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 02:30:40 -0000 --=-K19JSnonNJgCgtZ+X8tQ Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =E5=9C=A8 2005-08-08=E4=B8=80=E7=9A=84 03:15 +0200=EF=BC=8CSuleiman Souhlal= =E5=86=99=E9=81=93=EF=BC=9A > Hello, >=20 > On Aug 6, 2005, at 3:25 PM, gnn@freebsd.org wrote: >=20 > > I plan to make a port of this this weekend, but would like some > > feedback on this set of benchmarks. If they're useful I think we > > should make them part of a nightly benchmarking strategy. >=20 > In case you're interested, I ran it on a dual p4 xeon (without =20 > HyperThreading) from the netperf cluster, to compare the performance =20 > of RELENG_5, RELENG_6 and HEAD. > You can find the results at http://people.freebsd.org/~ssouhlal/stuff/=20 > compare_tiger-3.html . > It shows that RELENG_6 and HEAD are (in these tests) almost never =20 > slower than RELENG_5, and often more than 20% faster. Great work! BTW. Is there any clue about why pthread_128 looks slower than RELENG_5 and then recovered in HEAD? Cheers, --=20 Xin LI http://www.delphij.net/ --=-K19JSnonNJgCgtZ+X8tQ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQBC9rzt/cVsHxFZiIoRAjpIAJ0Tzvs+ajWMOL94JnvQWDPUkq+lKwCdGXG+ b5EGNy0VTm+5e2ePUwJE2AY= =Psra -----END PGP SIGNATURE----- --=-K19JSnonNJgCgtZ+X8tQ-- From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 04:21:31 2005 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 84F3216A41F for ; Mon, 8 Aug 2005 04:21:31 +0000 (GMT) (envelope-from silby@silby.com) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.FreeBSD.org (Postfix) with SMTP id 0B16F43D46 for ; Mon, 8 Aug 2005 04:21:30 +0000 (GMT) (envelope-from silby@silby.com) Received: (qmail 24036 invoked from network); 8 Aug 2005 04:21:29 -0000 Received: from unknown (HELO localhost) (unknown) by unknown with SMTP; 8 Aug 2005 04:21:29 -0000 X-pair-Authenticated: 209.68.2.70 Date: Sun, 7 Aug 2005 23:21:28 -0500 (CDT) From: Mike Silbersack To: Xin LI In-Reply-To: <20050807185129.GA61807@frontfree.net> Message-ID: <20050807232016.W9586@odysseus.silby.com> References: <20050807185129.GA61807@frontfree.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@FreeBSD.org Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 04:21:31 -0000 On Mon, 8 Aug 2005, Xin LI wrote: > [Bcc'ed to -developers@, so this can be discussed in a public list] > > Hi, > > It seems that vfs.ufs.dirhash_maxmem is set to 2MB. I think this value > is slightly too small for modern machines: > > My proposal is to increase the default dirhash_maxmem value to at least > 32MB or 64MB. Any objections? Scale it to the kernel map size by throwing something in the init_param3 function inside subr_param.c. That way all you have to do is put a floor on it, no need to worry about putting a ceiling on it. Mike "Silby" Silbersack From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 06:59:09 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 89E8416A41F for ; Mon, 8 Aug 2005 06:59:09 +0000 (GMT) (envelope-from kometen@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id A72A143D53 for ; Mon, 8 Aug 2005 06:59:08 +0000 (GMT) (envelope-from kometen@gmail.com) Received: by wproxy.gmail.com with SMTP id i22so961007wra for ; Sun, 07 Aug 2005 23:59:08 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=o2YMWEf0TBMr2l4Qi5+apneCbI9U+MIPNnyPlnBDdkw1+iQvlqhYh9QUxB9S2h7e4b/oGu1AsX/FQ3QTwBNMGtfSRi6eP1RFCNC49FyV5Ann2vTRGEL5zu27ynSzokIyVI0jFhYUWf3/f+3ZMHs6X4TDFvoWgLNwP1bz8F2hJng= Received: by 10.54.111.11 with SMTP id j11mr4069064wrc; Sun, 07 Aug 2005 23:59:08 -0700 (PDT) Received: by 10.54.144.11 with HTTP; Sun, 7 Aug 2005 23:59:06 -0700 (PDT) Message-ID: Date: Mon, 8 Aug 2005 08:59:06 +0200 From: Claus Guttesen To: performance@freebsd.org In-Reply-To: <1123466477.767.2.camel@spirit> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <01F3BA1C-C7C6-41C7-AFE8-675FA972D1A3@FreeBSD.org> <1123466477.767.2.camel@spirit> Cc: gnn@freebsd.org Subject: Re: Tarball of ported libmicro 0.3 available for testing... X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 06:59:09 -0000 > > > I plan to make a port of this this weekend, but would like some > > > feedback on this set of benchmarks. If they're useful I think we > > > should make them part of a nightly benchmarking strategy. > > I ran them on my dual Xeon @ 2.4 GHz, but it appears that rather than doing it's calculations in a timely manner it's been inspired by the world championship in athletics in Helsingfors. It's been longjumping since last night: last pid: 20440; load averages: 1.09, 1.13, 1.09 =20 up 12+21:24:00 08:57:17 140 processes: 2 running, 138 sleeping CPU states: 19.1% user, 0.0% nice, 33.5% system, 0.0% interrupt, 47.4% id= le Mem: 463M Active, 1213M Inact, 220M Wired, 104M Cache, 112M Buf, 7320K Free Swap: 4096M Total, 992K Used, 4095M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 18858 claus 1 115 0 2368K 1056K CPU1 1 631:52 99.02% longjmp 57798 claus 1 96 0 150M 148M select 0 48:48 0.54% Xorg Did a gmake and started the benchmark without any parameters. Nice work moving it to FreeBSD. regards Claus From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 11:55:22 2005 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D475A16A41F; Mon, 8 Aug 2005 11:55:22 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from efnet-math.org (efnet-math.org [69.60.109.125]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5033C43D46; Mon, 8 Aug 2005 11:55:22 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from [151.28.120.133] (ppp-133-120.28-151.libero.it [151.28.120.133]) (authenticated bits=0) by efnet-math.org (8.13.1/8.13.1) with ESMTP id j78BtG2n029278 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO); Mon, 8 Aug 2005 07:55:19 -0400 In-Reply-To: <1123466477.767.2.camel@spirit> References: <01F3BA1C-C7C6-41C7-AFE8-675FA972D1A3@FreeBSD.org> <1123466477.767.2.camel@spirit> Mime-Version: 1.0 (Apple Message framework v733) Content-Type: text/plain; charset=UTF-8; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable From: Suleiman Souhlal Date: Mon, 8 Aug 2005 13:55:05 +0200 To: Xin LI X-Mailer: Apple Mail (2.733) Cc: gnn@FreeBSD.org, performance@FreeBSD.org Subject: Re: Tarball of ported libmicro 0.3 available for testing... X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 11:55:22 -0000 Hello, On Aug 8, 2005, at 4:01 AM, Xin LI wrote: > =E5=9C=A8 2005-08-08=E4=B8=80=E7=9A=84 03:15 +0200=EF=BC=8CSuleiman = Souhlal=E5=86=99=E9=81=93=EF=BC=9A > >> Hello, >> >> On Aug 6, 2005, at 3:25 PM, gnn@freebsd.org wrote: >> >> >>> I plan to make a port of this this weekend, but would like some >>> feedback on this set of benchmarks. If they're useful I think we >>> should make them part of a nightly benchmarking strategy. >>> >> >> In case you're interested, I ran it on a dual p4 xeon (without >> HyperThreading) from the netperf cluster, to compare the performance >> of RELENG_5, RELENG_6 and HEAD. >> You can find the results at http://people.freebsd.org/~ssouhlal/=20 >> stuff/ >> compare_tiger-3.html . >> It shows that RELENG_6 and HEAD are (in these tests) almost never >> slower than RELENG_5, and often more than 20% faster. >> > > Great work! BTW. Is there any clue about why pthread_128 looks =20 > slower > than RELENG_5 and then recovered in HEAD? I'm not sure, but I ran the benchmark on the same kernel several =20 times (after rebooting), and as gnn noticed, it seems like there is a =20= high variance for the tests that have 128k size things. Something =20 must be going on at those sizes. The results are at http://people.freebsd.org/~ssouhlal/stuff/=20 compare_HEAD.html -- Suleiman Souhlal | ssouhlal@vt.edu The FreeBSD Project | ssouhlal@FreeBSD.org From owner-freebsd-performance@FreeBSD.ORG Mon Aug 8 21:42:41 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC16616A41F for ; Mon, 8 Aug 2005 21:42:41 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from mail.pgt.mpt.gov.br (mail.pgt.mpt.gov.br [200.157.62.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE95143D49 for ; Mon, 8 Aug 2005 21:42:37 +0000 (GMT) (envelope-from tuliogs@pgt.mpt.gov.br) Received: from [10.0.0.136] (516e.pgt.mpt.gov.br [10.0.0.136]) by mail.pgt.mpt.gov.br (8.13.1/8.13.1) with ESMTP id j78Lgaj3033830 for ; Mon, 8 Aug 2005 18:42:36 -0300 (BRST) (envelope-from tuliogs@pgt.mpt.gov.br) Message-ID: <42F7D241.7060201@pgt.mpt.gov.br> Date: Mon, 08 Aug 2005 18:44:33 -0300 From: =?ISO-8859-1?Q?Tulio_Guimar=E3es_da_Silva?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: multipart/mixed; boundary="------------010806040704080203020408" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [Fwd: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value?] X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 21:42:41 -0000 This is a multi-part message in MIME format. --------------010806040704080203020408 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Dumb me, forgot do redirect to the list. Sorry for that. Tulio G. Silva -------- Original Message -------- Subject: Re: [RFC] Bumping ufs.dirhash_maxmem to a larger value? Date: Mon, 08 Aug 2005 11:08:49 -0300 From: Tulio Guimarães da Silva To: Xin LI References: <20050807184707.GA61714@frontfree.net> <42F659AE.80905@mac.com> <20050807190533.GA61954@frontfree.net> I´m suggesting this without any clue of the amount of work required, but it´s a suggestion, anyway: maybe it could be dinamically calculated based on the growth of dirhash usage AND available RAM... I guess 10% is a reasonable *max* value, but this could also be disabled with a sysctl or compile time variable, à lá MAX_USERS. Say, if not set, or set to zero, then it´ll be dynamically calculated. For an initial value, maybe 2% is enough (the same 2MB for a 128MB machine). Since customized kernels are very frequent, this should not be an issue. Of course, the ideal "max percentage" could be obtained from beta-testing, since it should be by default set for a generic setup. But, again, I´m just guessing. ;) Tulio G. Silva Xin LI wrote: >On Sun, Aug 07, 2005 at 02:57:50PM -0400, Chuck Swiger wrote: >[snip] > > >>On the other hand, I've got several firewall boxes with only 128MB, and >>it's not reasonable to simply dedicate up to 64MB (half!) to dirhash >>without paying more attention to the amount of physical memory that is >>actually available. >> >> > >Forgot to mention that dirhash_maxmem is the upper limit, not the exact >amount that dirhash must use. If your firewall does not host zillion >of large directories then a bigger default dirhash_maxmem would >essentially a big no-op :-) > > > >>How big should dirhash_maxmem be? 5-10% of available RAM, perhaps? >> >> > >Actually this depends on how many large directories that you actively >access, not only the available RAM. If you access only a little of >small directories, e.g. hosting a database, then dirhash won't help >much. But with a lot of large directories having 1000+ files in most >of them, then raising the dirhash_maxmem will be a great help for >performance. > >I think it would be great if we can implement a automatical "suggested >value" of dirhash during bootstrap stage, though, which can be overridden >through sysctl.conf or subsequent sysctl. > >Cheers, > > --------------010806040704080203020408-- From owner-freebsd-performance@FreeBSD.ORG Sat Aug 13 02:19:55 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D860716A41F; Sat, 13 Aug 2005 02:19:55 +0000 (GMT) (envelope-from akhthar@carmatec.com) Received: from server1.carmatec.com (server1.carmatec.com [66.45.229.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 69A4943D48; Sat, 13 Aug 2005 02:19:55 +0000 (GMT) (envelope-from akhthar@carmatec.com) Received: from [202.56.253.42] (helo=192.168.0.87) by server1.carmatec.com with esmtpa (Exim 4.43) id 1E3lck-0005Di-SL; Fri, 12 Aug 2005 21:19:29 -0500 From: "Akhthar Parvez. K" Organization: Carmatec IT Solutions To: freebsd-questions@freebsd.org Date: Sat, 13 Aug 2005 07:50:14 +0530 User-Agent: KMail/1.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200508130750.14779.akhthar@carmatec.com> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server1.carmatec.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [47 12] X-AntiAbuse: Sender Address Domain - carmatec.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-performance@freebsd.org Subject: resource deadlock avoided?? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: akhthar@carmatec.com List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Aug 2005 02:19:56 -0000 Hi all, I am gettting the following error message while accessing the interchange admin panel. (perl), uid 1004: exited on signal 11 (core dumped) I have no problem in accessing the main page. Does anyone know what could be the problem? I had a look at sysctl directives, but did not notice anything particular. I would also like to tune my FreeBSD server. The server has 2 cpus and RAID5. There are around 1000 accounts in it. Can anyone suggest the ideal values for the sysctl variables. -- With Regards, Akhthar Parvez.K SA From owner-freebsd-performance@FreeBSD.ORG Sat Aug 13 06:35:45 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AAE1F16A41F for ; Sat, 13 Aug 2005 06:35:45 +0000 (GMT) (envelope-from chrcoluk@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B53643D45 for ; Sat, 13 Aug 2005 06:35:45 +0000 (GMT) (envelope-from chrcoluk@gmail.com) Received: by wproxy.gmail.com with SMTP id i22so715825wra for ; Fri, 12 Aug 2005 23:35:44 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=FssHBm1ZBUZPTtQHxsorRvQ3h8MOJqFdsW41MAyz8HTKDE+dfKu9IpM915plQdbJA9odd7a12QlzW/8q1G2DGdZxomm2Fkkd/I98Z4DIiZkuO0EDx7QlQnJfKnRqcHS7fjs8Oi5ePWxHB2i/UbxQ8HWS/Tw4IhQqs0UBH9446c4= Received: by 10.54.143.7 with SMTP id q7mr2405428wrd; Fri, 12 Aug 2005 23:35:44 -0700 (PDT) Received: by 10.54.113.3 with HTTP; Fri, 12 Aug 2005 23:35:44 -0700 (PDT) Message-ID: <3aaaa3a05081223352f69a959@mail.gmail.com> Date: Sat, 13 Aug 2005 07:35:44 +0100 From: Chris To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: shell speed/priority X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Aug 2005 06:35:45 -0000 Hi I am looking at a way so that any fg apps I run in the shell are higher priority then normal and also to make the shell more responsive during a denial of service attack that saturates cpu. I would guess renice on sshd/bash be appropriate but there are numerous processes of each and if so which ones should I renice. 93898 p2 S 0:00.20 -su (bash) 25726 ?? Ss 0:03.04 /usr/local/sbin/sshd -u0 91543 ?? Ss 0:00.05 sshd: admin [priv] (sshd) Chris From owner-freebsd-performance@FreeBSD.ORG Sat Aug 13 07:10:03 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9CAC216A41F for ; Sat, 13 Aug 2005 07:10:03 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: from web41207.mail.yahoo.com (web41207.mail.yahoo.com [66.218.93.40]) by mx1.FreeBSD.org (Postfix) with SMTP id 4AB2743D45 for ; Sat, 13 Aug 2005 07:10:03 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 84470 invoked by uid 60001); 13 Aug 2005 07:10:03 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=LuGZzQqZ2ztlwcmlS9095KZK7xezXIoVZU64HpYedY+IuLixfCPMVEfo7/yyqKjv63KjGJntn8Z7MGDoNK/3mDSnPXfdkq5nPRTs1KoKsUDXnsVJtaE1lP1pFGI7JIWW/4Zpq6EVn4Is1/uXWcZrtXK9Ya+ITSqaja0gCyr8sC8= ; Message-ID: <20050813071003.84468.qmail@web41207.mail.yahoo.com> Received: from [213.54.67.76] by web41207.mail.yahoo.com via HTTP; Sat, 13 Aug 2005 00:10:03 PDT Date: Sat, 13 Aug 2005 00:10:03 -0700 (PDT) From: Arne "Wörner" To: Chris , freebsd-performance@freebsd.org In-Reply-To: <3aaaa3a05081223352f69a959@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: shell speed/priority X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Aug 2005 07:10:03 -0000 --- Chris wrote: > [...] I would guess renice on sshd/bash be appropriate > but there are numerous processes of each and if so which > ones should I renice. > Just try it... :-)) You could try renice -10 $$ after you managed to get root priviliges (that command would give your current root shell the nice value -10 instead of 0, so that all the child processes of that shell would have the same nice level... possibly... including the background processes...). :-) -Arne ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs