From owner-svn-src-all@FreeBSD.ORG Tue Dec 27 00:09:23 2011 Return-Path: Delivered-To: svn-src-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B76D9106564A; Tue, 27 Dec 2011 00:09:23 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 888E38FC0C; Tue, 27 Dec 2011 00:09:23 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id pBR09FRk080225; Mon, 26 Dec 2011 16:09:15 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id pBR09FLU080224; Mon, 26 Dec 2011 16:09:15 -0800 (PST) (envelope-from sgk) Date: Mon, 26 Dec 2011 16:09:15 -0800 From: Steve Kargl To: Doug Barton Message-ID: <20111227000915.GA80162@troutmask.apl.washington.edu> References: <201112241216.pBOCGd1H012696@svn.freebsd.org> <4EF645D2.8080407@FreeBSD.org> <20111226102820.GT90831@alchemy.franken.de> <4EF8DC5B.9070404@FreeBSD.org> <20111226225136.GA79882@troutmask.apl.washington.edu> <4EF902EB.5050009@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EF902EB.5050009@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Marius Strobl Subject: Re: svn commit: r228857 - in head/usr.bin: . csup X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 00:09:23 -0000 On Mon, Dec 26, 2011 at 03:27:39PM -0800, Doug Barton wrote: > On 12/26/2011 14:51, Steve Kargl wrote: > > On Mon, Dec 26, 2011 at 12:43:07PM -0800, Doug Barton wrote: > >> On 12/26/2011 02:28, Marius Strobl wrote: > >>> On Sat, Dec 24, 2011 at 01:36:18PM -0800, Doug Barton wrote: > >>>> On 12/24/2011 04:16, Marius Strobl wrote: > >>>>> On FreeBSD just use the MD5 implementation of libmd rather than that of > >>>>> libcrypto so we don't need to relinquish csup when world is built without > >>>>> OpenSSL. > >>>> > >>>> Did you benchmark this at all? I agree that keeping csup available > >>>> absent openssl is a good goal, but csup is a prototypical "tool that > >>>> does the same thing many thousands of times" so even tiny regressions > >>>> could add up to a large cost in wall clock time. > >>> > >>> Well, in a real world test updating the same base on an amd64 machine > >>> connected to the Internet > >> > >> Adding a network connection to the test is almost certainly going to > >> obscure the results beyond utility. > > > > Given that the majority of FreeBSD users will be pulling code > > from the internet, this seems to be the most relevant test. > > Sorry if I wasn't clear. The change was to how the md5 portion of csup > is linked. In order to isolate the effects of that change you have to > remove everything that isn't related to that change. > > But this is regression testing 101, so I'm sure that you know that already. If 99% of the usage of csup goes over the internet and 95% of the execution time (as determined by a profiler) involves dealing with network, then worrying about libmd vs libcrypto is a waste of time. If you're concerned about performance, then find the bottlenecks for that most common usage pattern. Micro-optimizing a synthetic usage case is a waste of time. > >> The appropriate way to test this > >> would be to create a binary out of the md5 routine in csup, and link it > >> alternately with libcrypto and libmd. Then for each version run it > >> against the src tree (or ports, either way) 10 times. Discard the first > >> and last, and then plot the results with ministat. > > > > The proper way to test the libmd vs libcrypto versions of > > the md5 routines is to use a profiler. > > That'll give you a good view of where the performance bottlenecks are if > it turns out that libmd is actually slower, sure. But the interesting > question in terms of this change is the effect on wall clock time, since > that's what users are going to see. > > > Of course, one might ask the question on how the use of > > libmd effects the majority of FreeBSD users (ie., not FreeBSD > > developers). Does the majority run csup hourly? Daily? > > Weekly? > > For those that use csup, I imagine that they use it at least daily. But > that's not the point. > > > For a utility seldomly run be the majority of FreeBSD > > users, Doug, you seem to be wasting Marius's time. > > How often it's used isn't really relevant to whether or not introducing > a pessimization is worth it. In any case I didn't ask him to back it > out, I only asked to have it be an option if it turns out that libmd is > slower. Yes, I know you did not ask him to back out his change. You asked him if he measured the impact on performance with that implication that he should run some performance test. Marius ran additional tests (wasting his time) to answer your question. Your response was essentially, "well, your really need to do the test this way (ie., no internet)", with the obvious implication of "please go do your test again." > I understand that what you're really trying to do here is to take a shot > at me relative to my assertion that profiled libs should be off by > default. If you're going to respond in kind to every message I send it's > going to get boring really quick. Nope. I'm concerned that your wasting valuable developer time. If you were really concerned with the performance, I suspect that you know how to do the tests you have asked of Marius. Now, let's read his commit message: On FreeBSD just use the MD5 implementation of libmd rather than that of libcrypto so we don't need to relinquish csup when world is built without OpenSSL. His change actually allows FreeBSD users to use csup if they are in a situation where WITHOUT_CRYPTO and/or WITHOUT_OPENSSL is required. Yes, I know you want him to waste his time to come up with the perfect patch with Makefile magic to find libcrypto and fall back to libmd. If you're really concerned about performance I'm fairly certain that Marius would be willing to review your Makefile magic patch. -- Steve