From owner-freebsd-performance@freebsd.org Fri Nov 20 16:22:24 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7DCFA318C9 for ; Fri, 20 Nov 2015 16:22:24 +0000 (UTC) (envelope-from dan@langille.org) Received: from clavin2.langille.org (clavin2.langille.org [199.233.228.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C28A1537; Fri, 20 Nov 2015 16:22:24 +0000 (UTC) (envelope-from dan@langille.org) Received: from (clavin2.int.langille.org (clavin2.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 56A3A444A ; Fri, 20 Nov 2015 16:22:22 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: Measuring ZFS configuration differences From: Dan Langille In-Reply-To: Date: Fri, 20 Nov 2015 11:12:33 -0500 Cc: freebsd-performance@freebsd.org, Marcelo Araujo Content-Transfer-Encoding: quoted-printable Message-Id: <81C1CC91-F9EC-43DC-A2D8-A8BDDE28E9A3@langille.org> References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> To: Tom Curry X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 16:22:24 -0000 > On Nov 18, 2015, at 5:47 PM, Tom Curry wrote: >=20 > On Wed, Nov 18, 2015 at 4:12 PM, Dan Langille = wrote: >=20 >>=20 >>> On Nov 17, 2015, at 9:31 PM, Marcelo Araujo = >> wrote: >>>=20 >>> 2015-11-18 10:23 GMT+08:00 Dan Langille : >>>=20 >>>>=20 >>>>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo = >>>> wrote: >>>>>=20 >>>>> 2015-11-18 3:14 GMT+08:00 Dan Langille : >>>>>=20 >>>>>>=20 >>>>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo = >>>>>> wrote: >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> 2015-11-12 6:34 GMT+08:00 Dan Langille : >>>>>>=20 >>>>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille = wrote: >>>>>>>>=20 >>>>>>>> Following up on the discussions during EuroBSDCon 2015 = (Stockholm) >>>>>>> during the FreeBSD Developer >>>>>>>> Summit regarding various ZFS configuration settings, I write to >> start >>>>>>> our implementation phase now that some >>>>>>>> usual suspects have joined the list. >>>>>>>>=20 >>>>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance >>>>>>>>=20 >>>>>>>> I think the first order of business is granting access rights = to the >>>>>>> server (varm) in question: >>>>>>>>=20 >>>>>>>> http://dan.langille.org/2015/07/19/varm/ >>>>>>>>=20 >>>>>>>> During the workshop, mention was made of serial access. I can >> arrange >>>>>>> that. >>>>>>>>=20 >>>>>>>> The server has IPMI, however, my first thought: >>>>>>>>=20 >>>>>>>> 1 - connect a USB-serial cable to varm & link that to another = server >>>> in >>>>>>> my rack. >>>>>>>=20 >>>>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular >>>>>>> configuration for the test machine which made >>>>>>> it easy to configure and run tests? Was it PXE booting or = something? >>>>>>>=20 >>>>>>>> 2 - create a jail in that server and give it access to that = serial >>>>>>> connection >>>>>>>> 3 - redirect incoming port XYZ to that jail via a = public-key-only >> ssh >>>>>>> connection >>>>>>>> 4 - give people access >>>>>>>>=20 >>>>>>>> Any suggestions? >>>>>>>=20 >>>>>>> =E2=80=94 >>>>>>> Dan Langille >>>>>>> http://langille.org/ >>>>>>>=20 >>>>>>>=20 >>>>>> Hello Dan, >>>>>>=20 >>>>>> Yes, was me :) >>>>>>=20 >>>>>> I mention about zopkio test framework. >>>>>> I gave a presentation last weekend at PyCon Hong Kong about it. >>>>>>=20 >>>>>> Here is my slides: >>>>>>=20 >>>>=20 >> = http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests= -using-zopkio >>>>>>=20 >>>>>> The good of Zopkio is, we can write tests at once and run it as = much >> as >>>> we >>>>>> want in different machines. Also Zopkio depends of Naarad, that = can >>>> parse a >>>>>> CSV file and create metrics and SLA over those metrics, plot = graphs >> and >>>> so >>>>>> on. Pretty nice tool!!! >>>>>>=20 >>>>>> I'm wondering if we could start to test something and maybe show = it at >>>>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think? >>>>>> What I need right now would be a list of tests that we want to = perform >>>> as >>>>>> well as what parameters we would like to take as metrics to = compare.\ >>>>>>=20 >>>>>>=20 >>>>>> For tests, we can start with this list: >>>>>> https://github.com/dlangille/zfs_benchmarks/issues >>>>>>=20 >>>>>> We can start as soon as I figure out how to provide access to the >>>>>> testers. See above re serial connection. >>>>>>=20 >>>>>> I want to provide access, but I want to keep access restricted to = only >>>>>> this box and not to the rest of my home LAN. I plan to do this = via a >>>>>> VLAN. >>>>>>=20 >>>>>> I could fire up a Rasperberry Pi and allow ssh into that. Will = that >> be >>>>>> enough >>>>>> power for what you need to do? >>>>>>=20 >>>>>>=20 >>>>> First of all, thanks to share the tests cases. >>>>>=20 >>>>> If I use zopkio, the best would be access SSH direct to the target >>>> machine >>>>> where I need to run the tests. For zopkio, I need to have my SSH = KEY on >>>> the >>>>> target machine. >>>>=20 >>>> I am OK with this. >>>>=20 >>>>> As I don't know your network, maybe what you could do is: Via >>>> RasperBerry, >>>>> forward the SSH to the target machine, I will pass-through via = your >>>>> RasperBerry where you can control the access for the rest of your = LAN. >>>>>=20 >>>>> Another approach could be, two different subnets and a firewall. = Or as >>>> you >>>>> said, VLANS. >>>>=20 >>>> I will be doing VLANS, which have yet to be set up. >>>>=20 >>>> The target system will have ZFS pools can be configured for = different >>>> tests (i.e. raidz2 vs raidz3). >>>> This will involve gpart etc because the drives & pools will need to = be >>>> 'wiped' between different test >>>> runs. >>>>=20 >>>> I seem to recall someone suggesting PXE boot and configuring the = system >>>> remotely. Does anyone >>>> recall that? That aspect of the discussion was not recorded: >>>> https://wiki.freebsd.org/201510DevSummit/Performance >>>>=20 >>>>=20 >>> Bapt@ mentioned that, this is the way how we are doing in another >> project. >>> But in my point of view, it is not a must for our case! >>>=20 >>> The PXE wold be good if we try to test different of OS flavors, or = build >>> different images. >>=20 >> OK. The only thing holding us back is: >>=20 >> - adding the air filters to the case >> - moving to the new switch with the new VLANs >>=20 >> It's now a matter of time. >>=20 >> =E2=80=94 >> Dan Langille >> http://langille.org/ >>=20 >>=20 >>=20 >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >>=20 >=20 >=20 > Dan, >=20 > I don't know if you have already cracked the serial access nut, but = every > supermicro motherboard I have owned has supported serial over lan. I > personally have never used it, I have just noticed it from time to = time. > Here is a good resource on setting it up > = http://serverfault.com/questions/574351/serial-over-lan-on-freebsd-10-0-wi= th-supermicro-x9-scm-f > (note: it's for an X9 motherboard but it should work for your X10, see = the > comment at the very end) If we go that way, I will keep that in mind, thank you. > Also, FreeBSD, ZFS and performance are three of my most favorite = words. Is > there any way I can participate in this adventure? Yes, there will be, I'm sure.=20 This comes to mind immediately: running the tests on your own server = will validate/invalidate our results. For now, we have no results, because it's all waiting on me and my = network at home. It's a time issue. This weekend is pretty much filled with other stuff. =E2=80=94=20 Dan Langille http://langille.org/