From owner-freebsd-arch@FreeBSD.ORG Thu Dec 30 06:45:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB880106564A for ; Thu, 30 Dec 2010 06:45:19 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4F1D98FC1F for ; Thu, 30 Dec 2010 06:45:19 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3ACFB.dip.t-dialin.net [87.179.172.251]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id F2B69844027; Thu, 30 Dec 2010 07:45:07 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id 42D651418; Thu, 30 Dec 2010 07:45:03 +0100 (CET) Date: Thu, 30 Dec 2010 07:45:00 +0100 From: Alexander Leidinger To: Garrett Cooper Message-ID: <20101230074500.000040d5@unknown> In-Reply-To: References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> <20101229000130.000052f4@unknown> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: F2B69844027.A9086 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1294296308.57989@b+VosxQmolVXikD0H3pJdg X-EBL-Spam-Status: No Cc: "Bjoern A. Zeeb" , freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2010 06:45:19 -0000 On Wed, 29 Dec 2010 03:17:41 -0800 Garrett Cooper wrote: > On Tue, Dec 28, 2010 at 3:01 PM, Alexander Leidinger > wrote: > >> We still lack the parts that would tell us something in the last > >> week or last 24 hours caused a regression that made my > >> TCP/NFS/ZFS/UFS/ n% slower. =A0Kris had been doing a > >> good job in the past but as time shows we need more people, > >> different setups, ... > > > > We do not lack the parts, we lack someone to take the parts and get > > them up and running. See: > > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02819= .html > > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02821= .html >=20 > No: according to Erik in the first link, the parts were never fully > integrated and the project as a whole was never completed due to lack > of hardware and time. Some bits are also just directions as well as I understand this link (and my maybe flaky memory when he presented his work the first time) as he didn't had the time and hardware to set up the automatic benchmarking suite to run periodically "forever". And what you call directions looks to me like documentation. As Erik is participating in this thread, I let the last word for him. :) > Erik says in his reply to the thread. After seeing enough folks > prematurely claim victory within my own company, I know enough that > nothing is complete unless it works (to some basic degree, i.e. it can > be replicated by someone other than the original author of the > infrastructure or code using the directions provided from scratch). I agree. Erik can you make your work available for download? If not, I offer webspace (either on my private site or in my area of the FreeBSD owned webspace) to make it possible to donwload it. > >> It's not only "compiles", "boots", but also the formerly in this > >> thread mentioned "works correctly" and in addition to that the > >> "works well as expected" or "works better than before" - > >> hopefully;). > > > > Maybe this could also be used to run the regression tests as one of > > the benchmarks. If yes: As Robert mentioned, we can not go and tell > > to run them all in one command (ATM), but we could have each of > > them as a different benchmark. >=20 > Regression tests should be stable. Regression tests shouldn't crash > targets (in general). Benchmarks and stress tests can do that. We need I was referring to his explanation that there is not central place to kick off all of them. I agree that the should be stable, but in case of a regression a crash may be a possible failure scenario (which can then be fixed by fixing the regression). > IMO we should have something like: >=20 > make checkworld > make checkkernel I agree that this would be nice. I think this is one possible implemenation of what Robert whas referring to when he told that there is no central place to start all the regression tests. > that would go off and run functional regression tests, and something > like: >=20 > make stressworld > make stresskernel >=20 > that would to similar for benchmarks as well. Having stuff tied Depending on the benchmark you need some kind of setup which can not be done by such targets (if we assume that this shall be self-contained and possible just by running those commands instead of doing some setup of 3rd part software like databases or webservers before). Bye, Alexander.