From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 13:29:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99E3A93B for ; Wed, 8 Jan 2014 13:29:52 +0000 (UTC) Received: from mail-vb0-x22a.google.com (mail-vb0-x22a.google.com [IPv6:2607:f8b0:400c:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5487618CF for ; Wed, 8 Jan 2014 13:29:52 +0000 (UTC) Received: by mail-vb0-f42.google.com with SMTP id w5so1139291vbf.29 for ; Wed, 08 Jan 2014 05:29:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K9PD6bz+Cr5yyhMVHwSa60bFUmyUtZDhRLxiitTws+g=; b=LYACJOQmIQTKVhAH3M1Hxl9S7o6x3pEN76NdYz8pA+yRSkxrID2CEo2GcNtgpibwPp 73S3No1QaFStwZC57sgqJPakkwRiC4mVoVCDsf//HThoU2OBnXnMM5iGYyAA/EuYR++w jo4TWxzsXO//Nfc6pQC455/LAOx4YYkGslf2bAzzHNpG1lJkip08aHj1edRWfXks8bxH Uaol4svWE7cFdkmzEBXmQznN6E/nn2JCLb6BSEZqDyiZ8QuKndbuF/1Xxaum6PGsfWLS k4blmXmC8OHtyShP+SjsxUKSPN2bldxnuGTdVoqa82dbOSHlEtLmM36hbLTcYGKMO83k qnug== MIME-Version: 1.0 X-Received: by 10.52.31.227 with SMTP id d3mr2606248vdi.38.1389187790446; Wed, 08 Jan 2014 05:29:50 -0800 (PST) Received: by 10.52.94.51 with HTTP; Wed, 8 Jan 2014 05:29:50 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 15:29:50 +0200 Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alexander Yerenkow To: Erik Cederstrand Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 13:29:52 -0000 Hello all. I'm working on something which could be used as CI / test tool. I called this FreeBSD as Firmware, or FreeBSD-RO. https://github.com/yerenkow/freebsd-vm-image/tree/master/freebsd-firmware The main idea is that from source tree producing VMDK image, which can be used in VirtualBox / Esxi / * , and to get this image working in readonly via separating configs/data/packages into different disks. I'm planning to extend build system to have snapshotted /usr/obj dirs and make rebuilding even faster. How I see typical workflow: Prepare (or 'cleanup' ) stage: * /zfs/src/src-stable-10 - updated to latest source * /zfs/src/src-stable-10@R300000 - snapshot of revision created * clone created /zfs/src/src-stable-10-clone from snapshot * kernel + world built * /usr/obj snapshot taken; * clone recreated /zfs/src/src-stable-10-clone from latest revision snapshot * /usr/obj cloned from snapshot * optional patches applied to cloned dir * image build (with no cleaning at all, or cleaned only parts touched by patches) * image booted via VBOX * image-OS run tests, upload results/logs * image-OS shut down That use case can be used both to regression testing and to new patches test runs. But, to make that happened, FreeBSD itself should provide some tests to be run. If someone would like to join in developing this - you are very welcome :) -- Regards, Alexander Yerenkow