From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 22 09:11:00 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00723106567E for ; Tue, 22 Jul 2008 09:11:00 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id AF7A68FC33 for ; Tue, 22 Jul 2008 09:10:59 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A56988.dip.t-dialin.net [84.165.105.136]) by redbull.bpaserver.net (Postfix) with ESMTP id C365F2E0F8; Tue, 22 Jul 2008 11:10:53 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 5136370B53; Tue, 22 Jul 2008 11:10:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1216717839; bh=2s3yrYlf3G8hpRXwcfXXTG2k9mZpy7xW4 47JfItQi40=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Oi0rgke/lZPdQlx8N3imi/lduN6JJTbpwbk1I1EBWcR0ZIAVv8GZYq+0I4bU+LEYU 23Gv6CH3QYVXPY0lEdZQO2ncQY+PtBs7Xz25alscER5N94W3gjjk6eloJQ8iMxlO8rL bY+Fc8VFuggU8C/2gmEgXk8cw1pOdALV/l4lNq72wKZ+4gTd26TwliFW7myZEeUNn6C nJW7Tn/L86LpvSIGC7IeQtMnuGfoV2pnGO3fbq1H7Go027eYyzHscrkcyeqef+eIbhY JrM/XIlGhKl3vIVYKqJP0QyxkHE7cFAgJVZ6bNXUeg8r5JjRmPnzshamG2/SDlf+LvA vwp5NrMPQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id m6M9Acnh017631; Tue, 22 Jul 2008 11:10:38 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 22 Jul 2008 11:10:38 +0200 Message-ID: <20080722111038.21404k434075zx8g@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Tue, 22 Jul 2008 11:10:38 +0200 From: "Alexander Leidinger" To: "Garrett Cooper" References: <7d6fde3d0807212343i69526416h35a3d961fac0c9d9@mail.gmail.com> In-Reply-To: <7d6fde3d0807212343i69526416h35a3d961fac0c9d9@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.2) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-14.9, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No X-Mailman-Approved-At: Tue, 22 Jul 2008 11:23:27 +0000 Cc: jkoshy@freebsd.org, FreeBSD Hackers Subject: Re: TET and other testing framework for FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2008 09:11:00 -0000 Quoting "Garrett Cooper" (from Mon, 21 Jul 2008 =20 23:43:11 -0700): > Hi Joseph (and hackers), > I'm contacting Joseph primarily because I saw his name listed > under the TET page (http://wiki.freebsd.org/TetIntegration) as a > contact and was wondering whether or not there was any interest to > integrate test suites outside of TET into FreeBSD, but I'm also > involving hackers@ because there might be some references that someone > can provide me for other @freebsd folks. AFAIK there's no TET stuff for FreeBSD available in public. So I would =20 say it's more having regression tests available (making them compile =20 and run on FreeBSD, having a look at the failing tests and see if it =20 is a problem with the test or with FreeBSD ... see below for my =20 experience with the LTP) at all, than integrating them. > I ask because my work with LTP (Linux Test Project : > http://ltp.sf.net) for Cisco has allowed me some insight into using > OpenPOSIX and Ballista testing frameworks, which may prove helpful in > the release testing cause, and could help in detecting faults earlier > on, thus helping expedite the release process a bit more and > increasing confidence and interest in FreeBSD. Do you have any URLs or whatever besides the LTP one (where those are =20 integrated) to get those tests from an official point instead from =20 cutting it out of the LTP? > I'm pretty sure I have the blessings of the Ballista project's > principal investigator to relicense it under the BSD / GPL license, > it's currently a 'dead project' and he gave his blessings to post the > source up on Sourceforge, but I'll double check to make sure I have > his AOK before doing so... I think it would be beneficial so the what this produces on FreeBSD =20 (the license doesn't matter here) before we go and talk about =20 integration (here the license matters). > I'm also working on getting tst_res (1.) dually licensed from > the LTP folks with a BSD / GPL license to allow for better inclusion > into FreeBSD's infrastructure, to ensure that we have a deterministic > means of dealing with testcases and reporting. > Just wondering what, if any, interest would be in adopting and > applying this work to FreeBSD for the good of the community. If this is what is used in the LTP to run some tests, I have to say it =20 does not look mature. I've run the LTP a lot in FreeBSD (native linux =20 binaries, http://wiki.FreeBSD.org/linux-kernel/ltp ... LTP not run and =20 updated the page since a long time) to test the linux compatibility =20 layer, and the are several places where tests fail and no output is =20 produced or even the summary said the test passed. I even opened bug =20 reports at the LTP page on SF, but it seems nobody was interested in =20 those reports. There are also some other ideas, like using the protocol the perl test =20 suites use, to be able to use existing perl stuff to generate reports =20 and overviews out of the generated logs (AFAIR this was the idea =20 behind some changes to the existing regression tests a long time ago). Bye, Alexander. --=20 I read Playboy for the same reason I read National Geographic. To see the sights I'm never going to visit. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137