From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 16:38:14 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69A208B4 for ; Wed, 8 Jan 2014 16:38:14 +0000 (UTC) Received: from mail-we0-x234.google.com (mail-we0-x234.google.com [IPv6:2a00:1450:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 022DD1A4F for ; Wed, 8 Jan 2014 16:38:13 +0000 (UTC) Received: by mail-we0-f180.google.com with SMTP id t61so1685742wes.11 for ; Wed, 08 Jan 2014 08:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=/5ia9gUsniA7w3pvtJ5LJz6EXmzjUQudptsqWYdO/OA=; b=W+LxwOlkUkLReUoQ48U3zlCX/QfLYYygpzRhmn/qR4dFoE/CaNvdeDEZZv93+fW37R 1vDgUjY11+AiX2hjGhl+5HCTcV/wcu8vMwjmWlGBkv43qNNEmmKuy7DIyK8789Ho9I1t 5fGBCi+OnUv2Jqfn7pEVZkXM8B75MAiMthOd4OMO5tYzYt58AFWzh+bZXSzBKjvbH7NM bpHPuxYSTIsgxCC8KSpPDc/GCFMaPwh7sSEX1hvSn3o0x05tsHIhVQ2hB95AsJ3ipe9L LdFacrnI2uMopQGMOAy2x5MpXa0jSiBZDV1L32c2yk5ZYUhKH5kcrdXRD9tJGqaM85kG xQCw== MIME-Version: 1.0 X-Received: by 10.180.11.34 with SMTP id n2mr22519755wib.40.1389199092233; Wed, 08 Jan 2014 08:38:12 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Wed, 8 Jan 2014 08:38:12 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 09:38:12 -0700 X-Google-Sender-Auth: nVyig8nUg5xod03PlC5vF8ibeT4 Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Erik Cederstrand Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 16:38:14 -0000 On Wed, Jan 8, 2014 at 1:28 AM, Erik Cederstrand wrote: > Den 07/01/2014 kl. 17.09 skrev Ivan Voras : > >> Hello, >> >> Is someone working on a contitual benchmarking / regression testing >> project for FreeBSD? I seem to recall there was a post several months >> ago but I can't find it. > > I have an old project lying with a build/PXE server, a client installer a= nd benchmark runner and a website presenting benchmark reports. It=92s a bi= t outdated, but maybe it can be a starting point? It=92s a mix of shell and= Python. See https://github.com/ecederstrand/perfmon. I like that you stored test results in a SQL database. My SQL-foo is poor, so ATM my framework is using CSV. It also looks like you've got code to generate a website. Do you have any example output? I never got to that part. The PXE stuff, however, does not belong in the benchmark framework, IMHO. I think that the benchmark framework should just include the benchmarking and system profiling aspect, not system configuration. Provisioning and configuring systems can be done in a separate utility, one that can be shared, for example, with the continuous Kyua tester. > > I=92d like to work with you on this, if you want. I'll talk to the boss today to see if I can get any time for benchmarking, or at least to open our proprietary code. -Alan > > Erik > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= "