From owner-freebsd-net@FreeBSD.ORG Thu Jul 10 15:22:00 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48D9AF83; Thu, 10 Jul 2014 15:22:00 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B2E325B2; Thu, 10 Jul 2014 15:21:59 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:52429 helo=[192.168.1.12]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X5GAW-0006uM-Ky; Thu, 10 Jul 2014 11:21:57 -0400 From: "George Neville-Neil" To: "Garrett Cooper" Subject: Re: A new way to test systems in multiple machine scenarios... Date: Thu, 10 Jul 2014 11:21:54 -0400 Message-ID: In-Reply-To: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: Craig Rodrigues , "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 15:22:00 -0000 On 8 Jul 2014, at 4:56, Garrett Cooper wrote: > On Jul 6, 2014, at 9:06 PM, Craig Rodrigues = > wrote: > >> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil = >> >> wrote: >> >>> Hi, >>> >>> I've coded up a system to allow you to control multiple other = >>> systems for >>> use in testing. >>> >>> https://github.com/gvnn3/conductor >>> >>> >> Cool! The architecture you have is similar to that of the SPECsfs >> benchmark test ( http://www.spec.org/sfs2008/ ) >> which involves a "coordinator node" and multiple "client nodes" which >> direct NFS network >> traffic towards a System Under Test (SUT). Garrett Cooper actually = >> set up >> the original testbed >> that I am using now at iXsystems. :) >> >> It would be cool to put together tools like Jenkins, Kyua, and = >> conductor to >> do more advanced testing >> of FreeBSD before the project puts out releases. > > Agreed. The only thing that I have some concern about is the = > reinventing of the wheel in python. multiprocessing Managers are one = > viable option that=E2=80=99s existed since python 2.6; there=E2=80=99s = a learning = > curve though, and you=E2=80=99ll run into problems with pickling some = > objects because the pickle protocol is far from complete (example: = > http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemet= hod-when-using-pythons-multiprocessing-pool-ma = > ); you might run into this problems regardless because you=E2=80=99re = > serializing objects using pickle instead of using dill (or using a = > simpler serialization method like JSON). Fabric has a framework = > that=E2=80=99s nice to use if you have ssh capability. There are other = > frameworks that use twisted conch I think too (another library that = > implements ssh access). Yes, I learned quite a bit about pickling in writing this. Conductor = aims to be quite simple so I am hoping to avoid any crazy corner cases to do with pickling. > > Isilon has a framework they use, but it=E2=80=99s very customized to th= eir = > infrastructure and product assumptions and it=E2=80=99s in need of an = > overhaul :(. This, actually, is the problem I found. Lots of folks have partial = solutions that are either proprietary, internal, not read for prime time, not quite what we want, etc. etc. I = did get one private response of another system to look at as well. I basically did this as a stake in the ground around which to build = something we could possibly move forwards with. It's not a 100% solution but it's 80% of the solution to the = problem I run into 80% of the time. Best, George