From owner-freebsd-performance@FreeBSD.ORG Thu May 15 05:26:54 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91D7437B404; Thu, 15 May 2003 05:26:54 -0700 (PDT) Received: from milla.ask33.net (milla.ask33.net [217.197.166.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8765243FA3; Thu, 15 May 2003 05:26:52 -0700 (PDT) (envelope-from nick@milla.ask33.net) Received: by milla.ask33.net (Postfix, from userid 1001) id A6B323ABB4D; Thu, 15 May 2003 14:30:33 +0200 (CEST) Date: Thu, 15 May 2003 14:30:33 +0200 From: Pawel Jakub Dawidek To: freebsd-hackers@freebsd.org Message-ID: <20030515123033.GP45118@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="6+tq7hoORRLxZXV8" Content-Disposition: inline X-PGP-Key-URL: http://garage.freebsd.pl/jules.asc X-OS: FreeBSD 4.8-RELEASE i386 X-URL: http://garage.freebsd.pl User-Agent: Mutt/1.5.1i cc: freebsd-performance@freebsd.org Subject: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: freebsd-performance@freebsd.org List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 12:26:55 -0000 --6+tq7hoORRLxZXV8 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello hackers... IMHO optimization in FreeBSD's code has too low priority. Maybe is time to think about some 'optimiztion team' creation? It will be a group of hackers that are skilled in using gcc(1) optimiztion flags, when those flags can cause problems and how to slove them (we got -O for years) - optimization flags should be increased just like WARNS value is. They should have experience in old-school optimization (there are many places where '* (2^n)' could be changed to '<< n' and many places where compilator don't know how to or is just unable to optimize, etc.). They could made regular code profiling and present results somewhere (and of course help authors to get better performance in critical places). Bla, bla, bla... Just let's speed-up FreeBSD!. Any opinions/ideas? --=20 Pawel Jakub Dawidek pawel@dawidek.net UNIX Systems Programmer/Administrator http://garage.freebsd.pl Am I Evil? Yes, I Am! http://cerber.sourceforge.net --6+tq7hoORRLxZXV8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iQCVAwUBPsOIaT/PhmMH/Mf1AQHFNwP/bMDIZZf0lFJ2DPf/HMxsB4vDXn1nt03k m/cBZs+dydLdpKwaFz0N0Zua9fmiERQADyBcmiATwm5Ewzoi3kX2M5guZ6K4JFd/ bzquAhQO/6TYunfoqXp8IE5WM4SEpMr6mv3ZSfQW2dy/bmXzueDniobjBFAieGPv d1tPinXYYMc= =6u+G -----END PGP SIGNATURE----- --6+tq7hoORRLxZXV8-- From owner-freebsd-performance@FreeBSD.ORG Thu May 15 08:16:35 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F54A37B401; Thu, 15 May 2003 08:16:35 -0700 (PDT) Received: from milla.ask33.net (milla.ask33.net [217.197.166.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F37943FBD; Thu, 15 May 2003 08:16:34 -0700 (PDT) (envelope-from nick@milla.ask33.net) Received: by milla.ask33.net (Postfix, from userid 1001) id A94183ABB4D; Thu, 15 May 2003 17:19:45 +0200 (CEST) Date: Thu, 15 May 2003 17:19:45 +0200 From: Pawel Jakub Dawidek To: Mooneer Salem Message-ID: <20030515151945.GQ45118@garage.freebsd.pl> References: <20030515123033.GP45118@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="ovQKbPq78H0oOD2w" Content-Disposition: inline In-Reply-To: X-PGP-Key-URL: http://garage.freebsd.pl/jules.asc X-OS: FreeBSD 4.8-RELEASE i386 X-URL: http://garage.freebsd.pl User-Agent: Mutt/1.5.1i cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 15:16:35 -0000 --ovQKbPq78H0oOD2w Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, May 15, 2003 at 07:01:44AM -0700, Mooneer Salem wrote: +> Interesting. Wouldn't some optimizations only produce negliable results +> though (in relation to the +> amount of mess created in the code), especially since PC technology has = come +> a long way? Exactly! This is todays point of view. No! Not everything could be optimize by compiler and I'm not saying that algorythms optimization isn't important - it is of course much more important than code optimization and I'm talking about it to, but micro optimization is needed as well. Messy code? Why? People skilled in optimization know which parts of code should be really optimized and which aren't important from performance point of view at all (code profiling?). Good optimized code shouldn't be ugly, ehh. IMHO we should really, seriously think over this problems, because even if we are able to speed-up some stuff (like IPFW, UFS, VM, internet protoco= ls) even 5% it is worth it. --=20 Pawel Jakub Dawidek pawel@dawidek.net UNIX Systems Programmer/Administrator http://garage.freebsd.pl Am I Evil? Yes, I Am! http://cerber.sourceforge.net --ovQKbPq78H0oOD2w Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iQCVAwUBPsOwET/PhmMH/Mf1AQGwYAP+KeQBZRySe6AuM9tIcSjrRgsvXckzExH1 osHGjabbQRY7HBXfal6lcoswBuGUgamTi2jHVCW00EvhlPjXzxdpsPd/jvWioJPz 3Q/WTYiUd2U6S00q/STOkb/1Wm9OI1PpaW2J1d7nLgAfyZ32Y0J083Hjfbbrv6cB 7sV1+WEbnB4= =mhBm -----END PGP SIGNATURE----- --ovQKbPq78H0oOD2w-- From owner-freebsd-performance@FreeBSD.ORG Thu May 15 06:17:30 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD7C237B401 for ; Thu, 15 May 2003 06:17:30 -0700 (PDT) Received: from mail.cvt.dk (mail.cvt.dk [130.225.95.186]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D55F43F85 for ; Thu, 15 May 2003 06:17:28 -0700 (PDT) (envelope-from cp@cvt.dk) Received: from cvt.dk (cp.cvt.dk [130.225.95.152]) by mail.cvt.dk (Postfix) with ESMTP id C0AE8F3937 for ; Thu, 15 May 2003 15:17:25 +0200 (CEST) Message-ID: <3EC38FC0.E628A57F@cvt.dk> Date: Thu, 15 May 2003 15:01:52 +0200 From: Christoffer Pio Organization: Center for Videnteknologi X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 15 May 2003 13:47:11 -0700 Subject: GBIT router/switch/nic/driver questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 13:17:31 -0000 > Hello, > > Can anyone tell me about performance issues re. freebsd > in filtering bridge configuration, compared to router > configuration? The filtering will take place between > two ethernet gigabit interfaces. Also, what kind of ethernet > fiber SX gigabit interface is recommended for production > use? I need to continue maintaing high uptime and stability > is primary concern, also it need to perform well and of course > the driver must work well. > > How many NMBCLUSTERS do I need for a bussy 1 gbit ethernet > filtering bridge or router? Other kernel issues to keep in mind? > > I use ipf for ip filtering. I was also wondering if there exist > any dual fiber gigabit ethernet cards (two fiber-pairs on one > card) in order to save pci slot space. Also, any > considerations on pci bus bandwidth must be done? > > I will be grateful for any hints or comments. > > Please direct copy of response to cp@cvt.dk since I do > not subscribe to any mailing lists. > > Thanks > Christoffer Pio From owner-freebsd-performance@FreeBSD.ORG Thu May 15 07:01:57 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5A36C37B411; Thu, 15 May 2003 07:01:57 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E5C443F75; Thu, 15 May 2003 07:01:56 -0700 (PDT) (envelope-from mooneer@translator.cx) Received: from pool0005.cvx18-bradley.dialup.earthlink.net ([209.179.238.5] helo=morpheus) by falcon.mail.pas.earthlink.net with smtp (Exim 3.33 #1) id 19GJIx-00043r-00; Thu, 15 May 2003 07:01:56 -0700 From: "Mooneer Salem" To: , Date: Thu, 15 May 2003 07:01:44 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <20030515123033.GP45118@garage.freebsd.pl> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Mailman-Approved-At: Thu, 15 May 2003 13:47:11 -0700 Subject: RE: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 14:01:57 -0000 Hello, Interesting. Wouldn't some optimizations only produce negliable results though (in relation to the amount of mess created in the code), especially since PC technology has come a long way? Thanks, -- Mooneer Salem GPLTrans: http://www.translator.cx/ lifeafterking.org: http://www.lifeafterking.org/ -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freebsd.org]On Behalf Of Pawel Jakub Dawidek Sent: Thursday, May 15, 2003 5:31 AM To: freebsd-hackers@freebsd.org Cc: freebsd-performance@freebsd.org Subject: Optimizations. Hello hackers... IMHO optimization in FreeBSD's code has too low priority. Maybe is time to think about some 'optimiztion team' creation? It will be a group of hackers that are skilled in using gcc(1) optimiztion flags, when those flags can cause problems and how to slove them (we got -O for years) - optimization flags should be increased just like WARNS value is. They should have experience in old-school optimization (there are many places where '* (2^n)' could be changed to '<< n' and many places where compilator don't know how to or is just unable to optimize, etc.). They could made regular code profiling and present results somewhere (and of course help authors to get better performance in critical places). Bla, bla, bla... Just let's speed-up FreeBSD!. Any opinions/ideas? -- Pawel Jakub Dawidek pawel@dawidek.net UNIX Systems Programmer/Administrator http://garage.freebsd.pl Am I Evil? Yes, I Am! http://cerber.sourceforge.net From owner-freebsd-performance@FreeBSD.ORG Thu May 15 07:46:11 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 579CE37B404; Thu, 15 May 2003 07:46:11 -0700 (PDT) Received: from mailout04.sul.t-online.com (mailout04.sul.t-online.com [194.25.134.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3EF343F85; Thu, 15 May 2003 07:46:09 -0700 (PDT) (envelope-from Alexander@Leidinger.net) Received: from fwd03.sul.t-online.de by mailout04.sul.t-online.com with smtp id 19GJzh-00009p-03; Thu, 15 May 2003 16:46:05 +0200 Received: from Andro-Beta.Leidinger.net (520065502893-0001@[217.229.217.210]) by fmrl03.sul.t-online.com with esmtp id 19GJzQ-0OikboC; Thu, 15 May 2003 16:45:48 +0200 Received: from Magelan.Leidinger.net (Magelan [192.168.1.1]) h4FEji3G015833; Thu, 15 May 2003 16:45:44 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from Magelan.Leidinger.net (netchild@localhost [127.0.0.1]) by Magelan.Leidinger.net (8.12.9/8.12.9) with SMTP id h4FEjiiC002220; Thu, 15 May 2003 16:45:44 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Date: Thu, 15 May 2003 16:45:44 +0200 From: Alexander Leidinger To: freebsd-performance@freebsd.org Message-Id: <20030515164544.6ade3b2e.Alexander@Leidinger.net> In-Reply-To: <20030515123033.GP45118@garage.freebsd.pl> References: <20030515123033.GP45118@garage.freebsd.pl> X-Mailer: Sylpheed version 0.8.10claws (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Sender: 520065502893-0001@t-dialin.net X-Mailman-Approved-At: Thu, 15 May 2003 13:47:11 -0700 cc: nick@garage.freebsd.pl cc: freebsd-hackers@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: freebsd-performance@freebsd.org List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 14:46:11 -0000 On Thu, 15 May 2003 14:30:33 +0200 Pawel Jakub Dawidek wrote: > IMHO optimization in FreeBSD's code has too low priority. > Maybe is time to think about some 'optimiztion team' creation? The actual goal is to finish the "SMPng"-feature (and other new 5.x technologie). After that the "roadmap" (there's no official one, but if you read -current you read sometimes about what is written on the TODO lists) says something about optimizing those features. You know, premature optimization is evil, do you? > value is. They should have experience in old-school optimization (there are > many places where '* (2^n)' could be changed to '<< n' and many places where Are you sure? Compare the produced code, if the value is known to be a power of 2 at compile time, this optimization gets already performed by the compiler. Regardless of the fact that todays compilers already do a good job with such easy optimizations, not every place where you could improve the produced code is worth to improve in such a way (if you improve the code by 1ms but the complete "action" takes 1000ms it isn't worth... and optimizing algorithms gives you most of the time much more bang for the bucks...). Personally I prefer readability of the algorithm instead of speed in such situations. The actual performance difference between gcc and icc doesn't comes from such optimizations, its from using SIMD code and from scheduling the instructions in a different way (on a P4 you get huge benefits from this), so the time is better spend on improving those aspects of gcc instead. Bye, Alexander. -- The three Rs of Microsoft support: Retry, Reboot, Reinstall. http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7 From owner-freebsd-performance@FreeBSD.ORG Thu May 15 13:30:19 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 347B437B401; Thu, 15 May 2003 13:30:19 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 22F0943FA3; Thu, 15 May 2003 13:30:17 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h4FKUGwk068609; Thu, 15 May 2003 13:30:16 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h4FKUGml000780; Thu, 15 May 2003 13:30:16 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h4FKUGnt000779; Thu, 15 May 2003 13:30:16 -0700 (PDT) Date: Thu, 15 May 2003 13:30:16 -0700 From: Marcel Moolenaar To: freebsd-performance@freebsd.org Message-ID: <20030515203016.GB543@athlon.pn.xcllnt.net> References: <20030515123033.GP45118@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030515123033.GP45118@garage.freebsd.pl> User-Agent: Mutt/1.5.3i X-Mailman-Approved-At: Thu, 15 May 2003 13:47:11 -0700 cc: freebsd-hackers@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 20:30:19 -0000 On Thu, May 15, 2003 at 02:30:33PM +0200, Pawel Jakub Dawidek wrote: > Hello hackers... > > IMHO optimization in FreeBSD's code has too low priority. > Maybe is time to think about some 'optimiztion team' creation? I think I don't want to see this happen based on professional experience. My biggest concern is that it becomes a crusade. For if your focus is performance, you pretty much loose track of everything else. If not, then obviously you haven't been focussing. Don't get me wrong. Optimizing is a good thing, when done at the right time for the right reasons and in the right way. Having an optimization team created out of thin air can only lead to round-table discussions that boil down to "Hi, I'm Marcel. I like to optimize small objects" and before you know it all hell breaks loose for a barely measurable performance change. If you set yourself some simple goals and keep it high-level, then we can all get used to the idea and we will probably find other opportunities while we go. The end result can be much the same as you try to achieve now, except that it has a bigger chance to be integrated rather than some weird bunch on the side that plays with compiler options "and shit". Just a thought, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-performance@FreeBSD.ORG Thu May 15 14:26:24 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 61FDA37B401; Thu, 15 May 2003 14:26:24 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A06F43F93; Thu, 15 May 2003 14:26:21 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h4FLQJ6U093669; Fri, 16 May 2003 00:26:19 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h4FLQJcr093666; Fri, 16 May 2003 00:26:19 +0300 (EEST) Date: Fri, 16 May 2003 00:26:19 +0300 (EEST) From: Narvi To: Marcel Moolenaar In-Reply-To: <20030515203016.GB543@athlon.pn.xcllnt.net> Message-ID: <20030516002105.K40030-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 21:26:24 -0000 On Thu, 15 May 2003, Marcel Moolenaar wrote: > On Thu, May 15, 2003 at 02:30:33PM +0200, Pawel Jakub Dawidek wrote: > > Hello hackers... > > > > IMHO optimization in FreeBSD's code has too low priority. > > Maybe is time to think about some 'optimiztion team' creation? > > I think I don't want to see this happen based on professional > experience. My biggest concern is that it becomes a crusade. > For if your focus is performance, you pretty much loose track > of everything else. If not, then obviously you haven't been > focussing. > Well, supposedly any such team would need to start by creating a set of tools and benchmarks that could be used to quantify perfomance, get buy-in from the project that these give good measures (iterate until done) and then set up a bunch of machines to track there are no regressions? Followed by then starting to work towards eliminating problems 8-) > Don't get me wrong. Optimizing is a good thing, when done at the > right time for the right reasons and in the right way. Having > an optimization team created out of thin air can only lead to > round-table discussions that boil down to "Hi, I'm Marcel. I like > to optimize small objects" and before you know it all hell breaks > loose for a barely measurable performance change. > > If you set yourself some simple goals and keep it high-level, then > we can all get used to the idea and we will probably find other > opportunities while we go. The end result can be much the same as > you try to achieve now, except that it has a bigger chance to be > integrated rather than some weird bunch on the side that plays > with compiler options "and shit". nah, don't be too hard on them. they are volunteering for a large amount of hard work 8-) > > Just a thought, > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net > From owner-freebsd-performance@FreeBSD.ORG Thu May 15 14:38:05 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E886537B401; Thu, 15 May 2003 14:38:05 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id F229E43FA3; Thu, 15 May 2003 14:38:04 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h4FLc2xm003642; Thu, 15 May 2003 23:38:03 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Narvi From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 16 May 2003 00:26:19 +0300." <20030516002105.K40030-100000@haldjas.folklore.ee> Date: Thu, 15 May 2003 23:38:02 +0200 Message-ID: <3641.1053034682@critter.freebsd.dk> X-Mailman-Approved-At: Thu, 15 May 2003 14:40:34 -0700 cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org cc: Marcel Moolenaar Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 21:38:07 -0000 In message <20030516002105.K40030-100000@haldjas.folklore.ee>, Narvi writes: >On Thu, 15 May 2003, Marcel Moolenaar wrote: >> On Thu, May 15, 2003 at 02:30:33PM +0200, Pawel Jakub Dawidek wrote: >> > Maybe is time to think about some 'optimiztion team' creation? >> I think I don't want to see this happen based on professional >> experience. >Well, supposedly any such team would need to start by creating a set of >tools and benchmarks [...] If I have to be honest, I think this is the wrong way to approach the subject, if on no other ground than on the 3. rule of optimizations ("Don't do it yet"). While it would be nice to have a set of "blessed benchmarks" canned and ready to run, we should learn from the lmbench fiasco in Linux that such benchmarks can easier mislead than lead. My personal professional experience with optimizations or "Performance management" as it was called, is that you want some very rigid _functional_ testcases, which must pass at any one time so you don't unnoticed loose functionality to optimizations. We also know that the main performance issue is Giant, Giant and Giant. So I really think the band of merry men we are talking about, if they can be interested, would do much more good if they would start out building functional and regression tests for our most critical facilities. I can't speak for the other heavy-duty guys in the project, but I would personally be _really_ _REALLY_ grateful if I could "cd /usr/src ; make test" and know that a significant fraction of our functionality worked if it returned a zero exit code. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-performance@FreeBSD.ORG Thu May 15 15:11:18 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F21137B401 for ; Thu, 15 May 2003 15:11:18 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id C622F43F85 for ; Thu, 15 May 2003 15:11:17 -0700 (PDT) (envelope-from sean@perrin.int.nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id CE4642105D; Thu, 15 May 2003 15:11:16 -0700 (PDT) Date: Thu, 15 May 2003 15:11:16 -0700 From: Sean Chittenden To: Eric Anderson Message-ID: <20030515221116.GE42125@perrin.int.nxad.com> References: <001901c3137b$cc8d66f0$044ec7c8@AJMNOTEBOOK> <20030506065837.GT94932@perrin.int.nxad.com> <3EB7B1A4.9090500@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EB7B1A4.9090500@centtech.com> User-Agent: Mutt/1.4i X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ cc: freebsd-performance cc: Alan Jorge Markus Subject: Re: FReebsd 5.0 network performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 22:11:18 -0000 > [..snip..] > >Don't use 5.0 for production. If you are going to put anything in > >production, use a _very_ recent -CURRENT. Even then, 5.0 isn't as > >fast as the -STABLE series at the moment because it's still in an > >awkward transition between locking via giant and fine grain locking. > >If you'd like to help figure out performance problems for -CURRENT > >however, have you tried running any kernel profiling to see where the > >kernel is spending it's cycles? -sc > > What's the appropraite way to run kernel profiling, and how do you > read/pull out the results? Kernel Profiling in a nut shell: 1) Add "makeoptions DEBUG=-g" and "makeoptions PROFLEVEL=2" to your kernel config, recompile your kernel, install, and reboot. Use PROFLEVEL=1 for non-high resolution profiling. 2) Read kgmon(8) and gprof(1) 3) Repeat step #2 as many times as necessary. 4) Post any interesting results to this list with a detailed summary of what you were doing, what you think was interesting about the results, what version OS, what you'd expect the results to be like, and any hunches you may have about the results or why the results are what they are. Any relevant kernel config or sysctl tweaks should be included in the report too. 5) Remember at all times to keep in mind that SMPng isn't done and KSE isn't finished either and once those two projects are finished, the results may look radically different. Failure to comply with rules 1-5 will result in pointy hats, flamage, moderation, or filtering. :) Let's please try and keep this as objective as possible. -sc performance@ != chat@ != questions@ != high signal to noise ratio -- Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Thu May 15 15:15:05 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40AE137B401; Thu, 15 May 2003 15:15:05 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 22CA243FB1; Thu, 15 May 2003 15:15:04 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h4FMF3wk069151; Thu, 15 May 2003 15:15:03 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h4FMF3ml001045; Thu, 15 May 2003 15:15:03 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h4FMF3qd001044; Thu, 15 May 2003 15:15:03 -0700 (PDT) Date: Thu, 15 May 2003 15:15:03 -0700 From: Marcel Moolenaar To: Narvi Message-ID: <20030515221503.GC821@athlon.pn.xcllnt.net> References: <20030515203016.GB543@athlon.pn.xcllnt.net> <20030516002105.K40030-100000@haldjas.folklore.ee> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030516002105.K40030-100000@haldjas.folklore.ee> User-Agent: Mutt/1.5.3i cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 22:15:05 -0000 On Fri, May 16, 2003 at 12:26:19AM +0300, Narvi wrote: > > > If you set yourself some simple goals and keep it high-level, then > > we can all get used to the idea and we will probably find other > > opportunities while we go. The end result can be much the same as > > you try to achieve now, except that it has a bigger chance to be > > integrated rather than some weird bunch on the side that plays > > with compiler options "and shit". > > nah, don't be too hard on them. they are volunteering for a large amount > of hard work 8-) Hard work it is, but the question of timing pops up: when there's so much that needs to be done functionality-wise, how does the hard work relate to progress? Of course, this being a voluntary project, people are free to spend their time on things they like to do. But with freedom comes opposition -- or something along those lines :-) If I would be asked to sacrifice implementation flexibility at a time when things may need to be rewritten a couple of times to get it right (think new platforms), I will be mostly insensitive to arguments that limit that flexibility in favor of performance. This is the root of my concern. Any team that seperates itself as one that focusses on performance, is one that finds opposition and the opposition may be too large to be able to do any performance related work. It has to come natural, like enforcing style(9), or security. Not that enforcing style(9) is without friction. But it's a well-known and (almost :-) acceptable part of development. I think performance has to be that too if we don't want it to be a constant source of conflicts. If a performance team could achieve that, then we're in for a winner. /me thinks, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-performance@FreeBSD.ORG Thu May 15 15:23:04 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D99137B401; Thu, 15 May 2003 15:23:04 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id D8F7543F93; Thu, 15 May 2003 15:23:02 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h4FMMv6U094016; Fri, 16 May 2003 01:22:58 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h4FMMv5Q094013; Fri, 16 May 2003 01:22:57 +0300 (EEST) Date: Fri, 16 May 2003 01:22:57 +0300 (EEST) From: Narvi To: Poul-Henning Kamp In-Reply-To: <3641.1053034682@critter.freebsd.dk> Message-ID: <20030516004006.S40030-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org cc: Marcel Moolenaar Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 22:23:04 -0000 On Thu, 15 May 2003, Poul-Henning Kamp wrote: > > In message <20030516002105.K40030-100000@haldjas.folklore.ee>, Narvi writes: > >On Thu, 15 May 2003, Marcel Moolenaar wrote: > >> On Thu, May 15, 2003 at 02:30:33PM +0200, Pawel Jakub Dawidek wrote: > >> > Maybe is time to think about some 'optimiztion team' creation? > >> I think I don't want to see this happen based on professional > >> experience. > >Well, supposedly any such team would need to start by creating a set of > >tools and benchmarks [...] > > If I have to be honest, I think this is the wrong way to approach the > subject, if on no other ground than on the 3. rule of optimizations > ("Don't do it yet"). > > While it would be nice to have a set of "blessed benchmarks" canned > and ready to run, we should learn from the lmbench fiasco in Linux > that such benchmarks can easier mislead than lead. > Microbenchmarks can never lead you - at most they can alert you if something went wrong. Even more comprehensive ones just give you feedback unless what you are doing really is trying to find that one parameter / code sequence you can tune that would result in speedup in a large variety of relevant to the excerice apps (yes, it might be for as boring reason as 'ship criteria' or 'speed up by 10%'). > My personal professional experience with optimizations or "Performance > management" as it was called, is that you want some very rigid > _functional_ testcases, which must pass at any one time so you don't > unnoticed loose functionality to optimizations. > > We also know that the main performance issue is Giant, Giant and Giant. > Ah, yes - but I was thinking more in the 1 - 1.5 year from now timeframe, which is a sort of reasonable timeframe if you don't expect miracles or a largish corporate backer employing several people to do this in addition to volunteers and do expect people getting bored, wandering off, having real life issues and work, etcetera to deal with aswell. > So I really think the band of merry men we are talking about, if they > can be interested, would do much more good if they would start out > building functional and regression tests for our most critical > facilities. > > I can't speak for the other heavy-duty guys in the project, but I > would personally be _really_ _REALLY_ grateful if I could "cd > /usr/src ; make test" and know that a significant fraction of our > functionality worked if it returned a zero exit code. > Whichever framework they come up with to produce reliable, repeatable and fair reasults will very probably allow you to plug in regression tests aswell or be eaily modifiable for such. If it ends up producing additions to the presently pretty small set of profiling tools, so much the better - IMHO. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > From owner-freebsd-performance@FreeBSD.ORG Thu May 15 15:56:26 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C67F437B401; Thu, 15 May 2003 15:56:26 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id C04C343F75; Thu, 15 May 2003 15:56:25 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h4FMuO6U094178; Fri, 16 May 2003 01:56:24 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h4FMuOxd094175; Fri, 16 May 2003 01:56:24 +0300 (EEST) Date: Fri, 16 May 2003 01:56:24 +0300 (EEST) From: Narvi To: Marcel Moolenaar In-Reply-To: <20030515221503.GC821@athlon.pn.xcllnt.net> Message-ID: <20030516012353.U40030-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: Optimizations. X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 22:56:27 -0000 On Thu, 15 May 2003, Marcel Moolenaar wrote: > On Fri, May 16, 2003 at 12:26:19AM +0300, Narvi wrote: > > > > > If you set yourself some simple goals and keep it high-level, then > > > we can all get used to the idea and we will probably find other > > > opportunities while we go. The end result can be much the same as > > > you try to achieve now, except that it has a bigger chance to be > > > integrated rather than some weird bunch on the side that plays > > > with compiler options "and shit". > > > > nah, don't be too hard on them. they are volunteering for a large amount > > of hard work 8-) > > Hard work it is, but the question of timing pops up: when there's > so much that needs to be done functionality-wise, how does the > hard work relate to progress? Of course, this being a voluntary > project, people are free to spend their time on things they like > to do. But with freedom comes opposition -- or something along > those lines :-) Ultimately, they have to prove themselves and make sure what they produce gets used - otherwise they fail. > > If I would be asked to sacrifice implementation flexibility at a > time when things may need to be rewritten a couple of times to get > it right (think new platforms), I will be mostly insensitive to > arguments that limit that flexibility in favor of performance. > But if they are going to do this they don't understand how the project works, and will probably fail at any rate. > This is the root of my concern. Any team that seperates itself as > one that focusses on performance, is one that finds opposition and > the opposition may be too large to be able to do any performance > related work. It has to come natural, like enforcing style(9), or > security. Not that enforcing style(9) is without friction. But it's > a well-known and (almost :-) acceptable part of development. I > think performance has to be that too if we don't want it to be a > constant source of conflicts. > > If a performance team could achieve that, then we're in for a > winner. /me thinks, Yes, true. But its not a simple target 8-) > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net >