From owner-freebsd-testing@FreeBSD.ORG  Sat Apr 18 19:13:25 2015
Return-Path: <owner-freebsd-testing@FreeBSD.ORG>
Delivered-To: freebsd-testing@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id BECDBEB9;
 Sat, 18 Apr 2015 19:13:25 +0000 (UTC)
Received: from FreeBSD.cs.nctu.edu.tw (FreeBSD.cs.nctu.edu.tw [140.113.17.209])
 by mx1.freebsd.org (Postfix) with ESMTP id 8779D63E;
 Sat, 18 Apr 2015 19:13:25 +0000 (UTC)
Received: by FreeBSD.cs.nctu.edu.tw (Postfix, from userid 1058)
 id 102A52A1C; Sun, 19 Apr 2015 03:13:17 +0800 (CST)
Date: Sun, 19 Apr 2015 03:13:16 +0800
From: Li-Wen Hsu <lwhsu@FreeBSD.org>
To: Ed Maste <emaste@freebsd.org>
Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org>
Subject: Re: Adding FreeBSD/arm64 Jenkins build / QEMU job
Message-ID: <20150418191316.GA90740@FreeBSD.cs.nctu.edu.tw>
References: <CAPyFy2AQwk=oSjkx8w2PqwZsdk+SXOGkrsXxBeojiFVrn-1s0Q@mail.gmail.com>
 <CAKBkRUxZ1ebm_+FoDPDp1ivWLzH8G3eUALwREU=Wr5gOrto1NQ@mail.gmail.com>
 <CAPyFy2AVYmXwRzG0HmkjZ1LpczU9bAjzEoMm9EuGPz9uYSXc-A@mail.gmail.com>
 <20150417152715.GA511@FreeBSD.cs.nctu.edu.tw>
 <CAPyFy2Ajdgyry3+_NgqpOVSWmvYm7iD0w0xc=GZYmCn_HZAk2w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAPyFy2Ajdgyry3+_NgqpOVSWmvYm7iD0w0xc=GZYmCn_HZAk2w@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-BeenThere: freebsd-testing@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Testing on FreeBSD <freebsd-testing.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-testing/>
List-Post: <mailto:freebsd-testing@freebsd.org>
List-Help: <mailto:freebsd-testing-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-testing>, 
 <mailto:freebsd-testing-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 18 Apr 2015 19:13:25 -0000

On Fri, Apr 17, 2015 at 16:47:45 -0400, Ed Maste wrote:
> On 17 April 2015 at 11:27, Li-Wen Hsu <lwhsu@freebsd.org> wrote:
> >
> > That will be great.  And I am interested to know what do you plan for
> > booting QEMU machine, running test inside VM then getting results
> > archived to outside.  There are many approaches, and I wanted to know
> > the most suitable one for us.
> 
> Good question. Are we doing this for other targets today (QEMU, bhyve,
> or metal)?

Yes.

For kuya/QEMU, it builds a image which executes tests upon launch:

    https://svnweb.freebsd.org/base/user/jmmv/autotest/node/iterate.sh?view=markup&pathrev=271564#l154

and mounts the image to extract results:

   https://svnweb.freebsd.org/base/user/jmmv/autotest/node/iterate.sh?view=markup&pathrev=271564#l284

For bhyve, it uses expect-like library to login and executes tests

    https://github.com/freebsd/freebsd-ci/blob/master/scripts/test/run-tests.py

it also extracts log from image by mounting VM image:

    https://github.com/freebsd/freebsd-ci/blob/master/scripts/test/extract-test-logs.py

It would be more awesome if we can have everything automate to create
test VM from an unmodifed image, just like what we put to
release/snapshot, setup test env, and run tests on them, then extracting
results.  I think expect solution is good for this.  And I am interested
to know if there is any other approache.

Li-Wen

-- 
Li-Wen Hsu <lwhsu@FreeBSD.org>
http://lwhsu.org