From owner-freebsd-performance@freebsd.org Tue Nov 17 19:15:05 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDBA0A31D12 for ; Tue, 17 Nov 2015 19:15:04 +0000 (UTC) (envelope-from dan@langille.org) Received: from clavin2.langille.org (clavin2.langille.org [199.233.228.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A56BF1DC2; Tue, 17 Nov 2015 19:15:04 +0000 (UTC) (envelope-from dan@langille.org) Received: from (clavin2.int.langille.org (clavin2.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 199338CC0 ; Tue, 17 Nov 2015 19:14:57 +0000 (UTC) Subject: Re: Measuring ZFS configuration differences Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Content-Type: multipart/signed; boundary="Apple-Mail=_C035783A-6424-4400-80DA-134087C0DDE4"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.6b2 From: Dan Langille In-Reply-To: Date: Tue, 17 Nov 2015 14:14:50 -0500 Cc: freebsd-performance@freebsd.org Message-Id: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> To: Marcelo Araujo X-Mailer: Apple Mail (2.3096.5) X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 19:15:05 -0000 --Apple-Mail=_C035783A-6424-4400-80DA-134087C0DDE4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 12, 2015, at 1:30 AM, Marcelo Araujo = wrote: >=20 >=20 >=20 > 2015-11-12 6:34 GMT+08:00 Dan Langille >: > On Oct 12, 2015, at 1:00 PM, Dan Langille > wrote: > > > > Following up on the discussions during EuroBSDCon 2015 (Stockholm) = during the FreeBSD Developer > > Summit regarding various ZFS configuration settings, I write to = start our implementation phase now that some > > usual suspects have joined the list. > > > > re https://wiki.freebsd.org/201510DevSummit/Performance = > > > > I think the first order of business is granting access rights to the = server (varm) in question: > > > > http://dan.langille.org/2015/07/19/varm/ = > > > > During the workshop, mention was made of serial access. I can = arrange that. > > > > The server has IPMI, however, my first thought: > > > > 1 - connect a USB-serial cable to varm & link that to another server = in my rack. >=20 > Marcelo: At EuroBSDCon, was it you who mentioned a particular = configuration for the test machine which made > it easy to configure and run tests? Was it PXE booting or something? >=20 > > 2 - create a jail in that server and give it access to that serial = connection > > 3 - redirect incoming port XYZ to that jail via a public-key-only = ssh connection > > 4 - give people access > > > > Any suggestions? >=20 > =E2=80=94 > Dan Langille > http://langille.org/ >=20 >=20 > Hello Dan, >=20 > Yes, was me :) >=20 > I mention about zopkio test framework. > I gave a presentation last weekend at PyCon Hong Kong about it. >=20 > Here is my slides: = http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests= -using-zopkio = >=20 > The good of Zopkio is, we can write tests at once and run it as much = as we want in different machines. Also Zopkio depends of Naarad, that = can parse a CSV file and create metrics and SLA over those metrics, plot = graphs and so on. Pretty nice tool!!! >=20 > I'm wondering if we could start to test something and maybe show it at = AsiaBSDCon and BSDCon(Canada) next year? What do you think? > What I need right now would be a list of tests that we want to perform = as well as what parameters we would like to take as metrics to compare.\ For tests, we can start with this list: = https://github.com/dlangille/zfs_benchmarks/issues = We can start as soon as I figure out how to provide access to the = testers. See above re serial connection. I want to provide access, but I want to keep access restricted to only this box and not to the rest of my home LAN. I plan to do this via a = VLAN. I could fire up a Rasperberry Pi and allow ssh into that. Will that be = enough power for what you need to do? =E2=80=94 Dan Langille http://langille .org/ --Apple-Mail=_C035783A-6424-4400-80DA-134087C0DDE4 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQJ8BAEBCgBmBQJWS3ywXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ1MTE2RjM0ODIzRDdERDM4OTY0OUJBNzdF QjIxNTlERUU5NzI3MzlGAAoJEOshWd7pcnOfO/4P+gOuILl+TjKn3E8fpTvF0Y/D rHT5EjCYZ3KANMXfSDCUk9kceA/0nKLcjLagVYirzl2VzgpPhfPcdb6C1AfBHCUT 7NvHUh/aMJf9qn1I9nGQflRdyJm6zHTBjM8WtsXaYj7Tqr5frOzBMLsfQl+UUTx5 HS9WumFMFY7QI55CG5kFkDuJpgd8u+hy8ytz7mZ6CDPjr9hGEDu8U35+983XqAo0 TnBXk/TbRSyGoGfTgaAY5SujJBMiu1iZkVi/zpm5qSgvVmj1OqaWoo8JpqchSCSZ gDDprvq6tskVgnVWkjeakQgm+4hAUkJRuUcRgOU/ZG0WahPsqT/77B6ftLMqK5fu 4hBZYVjq3aIteebUE2rcf9/GSnUtRAHBu+EbqmaQwgrPW1Z4V7/uWtL85W5sB7HK vP6MR2OPWizEOb95bJl85e+N46RoqXejvYNu4Co6l+9T8UldYm1FF72cVzIjMEXz xc22WQPTiNym1yOFYdn1+UUJ9w7I5O0LdufTqsnqdlaWpTNog2yC31o9/FJUDTJG /xKmpv8sUoYdDz6qI1xrgr/MdLlfsl4IwlEn0WOebAyU6yrUGFMA0MEpCWQwgXd/ aIceqrgsjDHTKPUbrJeXq505F3uUSFO76ZmJ7/sH/fcwjuFjKA85ANsP3z3R/gcH Mwe96Kbl3BDeD4hjYpZU =ohCS -----END PGP SIGNATURE----- --Apple-Mail=_C035783A-6424-4400-80DA-134087C0DDE4-- From owner-freebsd-performance@freebsd.org Wed Nov 18 01:46:44 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6634A2E6C1 for ; Wed, 18 Nov 2015 01:46:44 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AD041107 for ; Wed, 18 Nov 2015 01:46:44 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by obbww6 with SMTP id ww6so22329092obb.0 for ; Tue, 17 Nov 2015 17:46:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=O2Dy6BOQySch5uxsuZgNSzjVYv2937ZxPM80PEG7YxQ=; b=X8aAKRFX+LWdQlvUy+x8B+OCbLDsz8/yF48FHH+ddQi0iUtzggREnxFspUOGen9DB4 kKJKF5cERuibksgA3y/1XWaqNqTVBt/MUmaDmOB+lTvxDcn9rjw7VZNuhCpgNVAah7KZ ZmS1LtxAGMj6N2X3Y2UJuHTqBrnZ9nB0V/q6gkE3GCAEPX72E4x05upy/HN4Iu2iAsLt 2vRirG1kH2aMRjdDGeQHl/ztI3zvdjbH+OwqfI2R40HyGfYqhgfKlD43fviN66an6g// 6ch5f70IPjEcH6jZv/JLrNvTBq6dgDflN8Ndt7WDquD/9TQmjKoCbZhJKDxU0mHnO4tL myEA== MIME-Version: 1.0 X-Received: by 10.60.95.8 with SMTP id dg8mr27940583oeb.81.1447811203548; Tue, 17 Nov 2015 17:46:43 -0800 (PST) Received: by 10.182.174.1 with HTTP; Tue, 17 Nov 2015 17:46:43 -0800 (PST) Reply-To: araujo@FreeBSD.org In-Reply-To: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> Date: Wed, 18 Nov 2015 09:46:43 +0800 Message-ID: Subject: Re: Measuring ZFS configuration differences From: Marcelo Araujo To: Dan Langille Cc: freebsd-performance@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 01:46:44 -0000 2015-11-18 3:14 GMT+08:00 Dan Langille : > > On Nov 12, 2015, at 1:30 AM, Marcelo Araujo > wrote: > > > > 2015-11-12 6:34 GMT+08:00 Dan Langille : > >> On Oct 12, 2015, at 1:00 PM, Dan Langille wrote: >> > >> > Following up on the discussions during EuroBSDCon 2015 (Stockholm) >> during the FreeBSD Developer >> > Summit regarding various ZFS configuration settings, I write to start >> our implementation phase now that some >> > usual suspects have joined the list. >> > >> > re https://wiki.freebsd.org/201510DevSummit/Performance >> > >> > I think the first order of business is granting access rights to the >> server (varm) in question: >> > >> > http://dan.langille.org/2015/07/19/varm/ >> > >> > During the workshop, mention was made of serial access. I can arrange >> that. >> > >> > The server has IPMI, however, my first thought: >> > >> > 1 - connect a USB-serial cable to varm & link that to another server i= n >> my rack. >> >> Marcelo: At EuroBSDCon, was it you who mentioned a particular >> configuration for the test machine which made >> it easy to configure and run tests? Was it PXE booting or something? >> >> > 2 - create a jail in that server and give it access to that serial >> connection >> > 3 - redirect incoming port XYZ to that jail via a public-key-only ssh >> connection >> > 4 - give people access >> > >> > Any suggestions? >> >> =E2=80=94 >> Dan Langille >> http://langille.org/ >> >> > Hello Dan, > > Yes, was me :) > > I mention about zopkio test framework. > I gave a presentation last weekend at PyCon Hong Kong about it. > > Here is my slides: > http://www.slideshare.net/araujobsd/functional-and-scale-performance-test= s-using-zopkio > > The good of Zopkio is, we can write tests at once and run it as much as w= e > want in different machines. Also Zopkio depends of Naarad, that can parse= a > CSV file and create metrics and SLA over those metrics, plot graphs and s= o > on. Pretty nice tool!!! > > I'm wondering if we could start to test something and maybe show it at > AsiaBSDCon and BSDCon(Canada) next year? What do you think? > What I need right now would be a list of tests that we want to perform as > well as what parameters we would like to take as metrics to compare.\ > > > For tests, we can start with this list: > https://github.com/dlangille/zfs_benchmarks/issues > > We can start as soon as I figure out how to provide access to the > testers. See above re serial connection. > > I want to provide access, but I want to keep access restricted to only > this box and not to the rest of my home LAN. I plan to do this via a > VLAN. > > I could fire up a Rasperberry Pi and allow ssh into that. Will that be > enough > power for what you need to do? > > First of all, thanks to share the tests cases. If I use zopkio, the best would be access SSH direct to the target machine where I need to run the tests. For zopkio, I need to have my SSH KEY on the target machine. As I don't know your network, maybe what you could do is: Via RasperBerry, forward the SSH to the target machine, I will pass-through via your RasperBerry where you can control the access for the rest of your LAN. Another approach could be, two different subnets and a firewall. Or as you said, VLANS. Best, --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-performance@freebsd.org Wed Nov 18 02:23:07 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC3E4A2EEBD for ; Wed, 18 Nov 2015 02:23:07 +0000 (UTC) (envelope-from dan@langille.org) Received: from clavin2.langille.org (clavin2.langille.org [199.233.228.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 682081210; Wed, 18 Nov 2015 02:23:06 +0000 (UTC) (envelope-from dan@langille.org) Received: from (clavin2.int.langille.org (clavin2.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 97BCE82AD ; Wed, 18 Nov 2015 02:23:04 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: Measuring ZFS configuration differences From: Dan Langille In-Reply-To: Date: Tue, 17 Nov 2015 21:23:03 -0500 Cc: freebsd-performance@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> To: Marcelo Araujo X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 02:23:07 -0000 > On Nov 17, 2015, at 8:46 PM, Marcelo Araujo = wrote: >=20 > 2015-11-18 3:14 GMT+08:00 Dan Langille : >=20 >>=20 >> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo >> wrote: >>=20 >>=20 >>=20 >> 2015-11-12 6:34 GMT+08:00 Dan Langille : >>=20 >>> On Oct 12, 2015, at 1:00 PM, Dan Langille wrote: >>>>=20 >>>> Following up on the discussions during EuroBSDCon 2015 (Stockholm) >>> during the FreeBSD Developer >>>> Summit regarding various ZFS configuration settings, I write to = start >>> our implementation phase now that some >>>> usual suspects have joined the list. >>>>=20 >>>> re https://wiki.freebsd.org/201510DevSummit/Performance >>>>=20 >>>> I think the first order of business is granting access rights to = the >>> server (varm) in question: >>>>=20 >>>> http://dan.langille.org/2015/07/19/varm/ >>>>=20 >>>> During the workshop, mention was made of serial access. I can = arrange >>> that. >>>>=20 >>>> The server has IPMI, however, my first thought: >>>>=20 >>>> 1 - connect a USB-serial cable to varm & link that to another = server in >>> my rack. >>>=20 >>> Marcelo: At EuroBSDCon, was it you who mentioned a particular >>> configuration for the test machine which made >>> it easy to configure and run tests? Was it PXE booting or = something? >>>=20 >>>> 2 - create a jail in that server and give it access to that serial >>> connection >>>> 3 - redirect incoming port XYZ to that jail via a public-key-only = ssh >>> connection >>>> 4 - give people access >>>>=20 >>>> Any suggestions? >>>=20 >>> =E2=80=94 >>> Dan Langille >>> http://langille.org/ >>>=20 >>>=20 >> Hello Dan, >>=20 >> Yes, was me :) >>=20 >> I mention about zopkio test framework. >> I gave a presentation last weekend at PyCon Hong Kong about it. >>=20 >> Here is my slides: >> = http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests= -using-zopkio >>=20 >> The good of Zopkio is, we can write tests at once and run it as much = as we >> want in different machines. Also Zopkio depends of Naarad, that can = parse a >> CSV file and create metrics and SLA over those metrics, plot graphs = and so >> on. Pretty nice tool!!! >>=20 >> I'm wondering if we could start to test something and maybe show it = at >> AsiaBSDCon and BSDCon(Canada) next year? What do you think? >> What I need right now would be a list of tests that we want to = perform as >> well as what parameters we would like to take as metrics to compare.\ >>=20 >>=20 >> For tests, we can start with this list: >> https://github.com/dlangille/zfs_benchmarks/issues >>=20 >> We can start as soon as I figure out how to provide access to the >> testers. See above re serial connection. >>=20 >> I want to provide access, but I want to keep access restricted to = only >> this box and not to the rest of my home LAN. I plan to do this via a >> VLAN. >>=20 >> I could fire up a Rasperberry Pi and allow ssh into that. Will that = be >> enough >> power for what you need to do? >>=20 >>=20 > First of all, thanks to share the tests cases. >=20 > If I use zopkio, the best would be access SSH direct to the target = machine > where I need to run the tests. For zopkio, I need to have my SSH KEY = on the > target machine. I am OK with this. > As I don't know your network, maybe what you could do is: Via = RasperBerry, > forward the SSH to the target machine, I will pass-through via your > RasperBerry where you can control the access for the rest of your LAN. >=20 > Another approach could be, two different subnets and a firewall. Or as = you > said, VLANS. I will be doing VLANS, which have yet to be set up. The target system will have ZFS pools can be configured for different = tests (i.e. raidz2 vs raidz3). This will involve gpart etc because the drives & pools will need to be = 'wiped' between different test runs. I seem to recall someone suggesting PXE boot and configuring the system = remotely. Does anyone=20 recall that? That aspect of the discussion was not recorded: = https://wiki.freebsd.org/201510DevSummit/Performance =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-performance@freebsd.org Wed Nov 18 02:31:09 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6109A3003B for ; Wed, 18 Nov 2015 02:31:09 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 90D9A13CA for ; Wed, 18 Nov 2015 02:31:09 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by oiww189 with SMTP id w189so17103778oiw.3 for ; Tue, 17 Nov 2015 18:31:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=F8l+KfVKfzZFJ4/2nCI94X1DXhkA0Ust6bXYs8omQZ0=; b=UIR6q7D+/pilnoJAs5lv0RIeBX/EKGlkhqw1ZSHbn2RRHZn2cRvELFhfI1ufAr3zbP iOWVZvQN1K4PkvDN/4ib5AgsJX4yCcN3aNlIcV67uLGJbflTRMUA4dfjbZt/Gst6I93d Q1fcX3ANSQa+JPxsuPzgXNPjnVpoRJYCxHDWd445lrcmocMg9tKgjs1v8psNinM5FrHI UUwZRkDZ85qed5kdXAX5TyZDibOdKEQF6++XfFa9qV0lTJ4X/Ov8KizFgGo/Bdxeeqvv 7M0XH4UxG66jptY2Eb/V2r2jTegtawKIDiXWUe+FNaxcCJXKRYlW8OxSzf0jJoKm8Bxc ndcA== MIME-Version: 1.0 X-Received: by 10.202.192.67 with SMTP id q64mr25298308oif.60.1447813868755; Tue, 17 Nov 2015 18:31:08 -0800 (PST) Received: by 10.182.174.1 with HTTP; Tue, 17 Nov 2015 18:31:08 -0800 (PST) Reply-To: araujo@FreeBSD.org In-Reply-To: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> Date: Wed, 18 Nov 2015 10:31:08 +0800 Message-ID: Subject: Re: Measuring ZFS configuration differences From: Marcelo Araujo To: Dan Langille Cc: freebsd-performance@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 02:31:09 -0000 2015-11-18 10:23 GMT+08:00 Dan Langille : > > > On Nov 17, 2015, at 8:46 PM, Marcelo Araujo > wrote: > > > > 2015-11-18 3:14 GMT+08:00 Dan Langille : > > > >> > >> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo > >> wrote: > >> > >> > >> > >> 2015-11-12 6:34 GMT+08:00 Dan Langille : > >> > >>> On Oct 12, 2015, at 1:00 PM, Dan Langille wrote: > >>>> > >>>> Following up on the discussions during EuroBSDCon 2015 (Stockholm) > >>> during the FreeBSD Developer > >>>> Summit regarding various ZFS configuration settings, I write to star= t > >>> our implementation phase now that some > >>>> usual suspects have joined the list. > >>>> > >>>> re https://wiki.freebsd.org/201510DevSummit/Performance > >>>> > >>>> I think the first order of business is granting access rights to the > >>> server (varm) in question: > >>>> > >>>> http://dan.langille.org/2015/07/19/varm/ > >>>> > >>>> During the workshop, mention was made of serial access. I can arran= ge > >>> that. > >>>> > >>>> The server has IPMI, however, my first thought: > >>>> > >>>> 1 - connect a USB-serial cable to varm & link that to another server > in > >>> my rack. > >>> > >>> Marcelo: At EuroBSDCon, was it you who mentioned a particular > >>> configuration for the test machine which made > >>> it easy to configure and run tests? Was it PXE booting or something? > >>> > >>>> 2 - create a jail in that server and give it access to that serial > >>> connection > >>>> 3 - redirect incoming port XYZ to that jail via a public-key-only ss= h > >>> connection > >>>> 4 - give people access > >>>> > >>>> Any suggestions? > >>> > >>> =E2=80=94 > >>> Dan Langille > >>> http://langille.org/ > >>> > >>> > >> Hello Dan, > >> > >> Yes, was me :) > >> > >> I mention about zopkio test framework. > >> I gave a presentation last weekend at PyCon Hong Kong about it. > >> > >> Here is my slides: > >> > http://www.slideshare.net/araujobsd/functional-and-scale-performance-test= s-using-zopkio > >> > >> The good of Zopkio is, we can write tests at once and run it as much a= s > we > >> want in different machines. Also Zopkio depends of Naarad, that can > parse a > >> CSV file and create metrics and SLA over those metrics, plot graphs an= d > so > >> on. Pretty nice tool!!! > >> > >> I'm wondering if we could start to test something and maybe show it at > >> AsiaBSDCon and BSDCon(Canada) next year? What do you think? > >> What I need right now would be a list of tests that we want to perform > as > >> well as what parameters we would like to take as metrics to compare.\ > >> > >> > >> For tests, we can start with this list: > >> https://github.com/dlangille/zfs_benchmarks/issues > >> > >> We can start as soon as I figure out how to provide access to the > >> testers. See above re serial connection. > >> > >> I want to provide access, but I want to keep access restricted to only > >> this box and not to the rest of my home LAN. I plan to do this via a > >> VLAN. > >> > >> I could fire up a Rasperberry Pi and allow ssh into that. Will that b= e > >> enough > >> power for what you need to do? > >> > >> > > First of all, thanks to share the tests cases. > > > > If I use zopkio, the best would be access SSH direct to the target > machine > > where I need to run the tests. For zopkio, I need to have my SSH KEY on > the > > target machine. > > I am OK with this. > > > As I don't know your network, maybe what you could do is: Via > RasperBerry, > > forward the SSH to the target machine, I will pass-through via your > > RasperBerry where you can control the access for the rest of your LAN. > > > > Another approach could be, two different subnets and a firewall. Or as > you > > said, VLANS. > > I will be doing VLANS, which have yet to be set up. > > The target system will have ZFS pools can be configured for different > tests (i.e. raidz2 vs raidz3). > This will involve gpart etc because the drives & pools will need to be > 'wiped' between different test > runs. > > I seem to recall someone suggesting PXE boot and configuring the system > remotely. Does anyone > recall that? That aspect of the discussion was not recorded: > https://wiki.freebsd.org/201510DevSummit/Performance > > Bapt@ mentioned that, this is the way how we are doing in another project. But in my point of view, it is not a must for our case! The PXE wold be good if we try to test different of OS flavors, or build different images. Best, --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-performance@freebsd.org Wed Nov 18 21:13:15 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 132AFA32D8A for ; Wed, 18 Nov 2015 21:13:15 +0000 (UTC) (envelope-from dan@langille.org) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C22E71DDE; Wed, 18 Nov 2015 21:13:14 +0000 (UTC) (envelope-from dan@langille.org) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 7813F4BE ; Wed, 18 Nov 2015 21:13:00 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: Measuring ZFS configuration differences From: Dan Langille In-Reply-To: Date: Wed, 18 Nov 2015 16:12:55 -0500 Cc: freebsd-performance@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> To: Marcelo Araujo X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 21:13:15 -0000 > On Nov 17, 2015, at 9:31 PM, Marcelo Araujo = wrote: >=20 > 2015-11-18 10:23 GMT+08:00 Dan Langille : >=20 >>=20 >>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo = >> wrote: >>>=20 >>> 2015-11-18 3:14 GMT+08:00 Dan Langille : >>>=20 >>>>=20 >>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo = >>>> wrote: >>>>=20 >>>>=20 >>>>=20 >>>> 2015-11-12 6:34 GMT+08:00 Dan Langille : >>>>=20 >>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille = wrote: >>>>>>=20 >>>>>> Following up on the discussions during EuroBSDCon 2015 = (Stockholm) >>>>> during the FreeBSD Developer >>>>>> Summit regarding various ZFS configuration settings, I write to = start >>>>> our implementation phase now that some >>>>>> usual suspects have joined the list. >>>>>>=20 >>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance >>>>>>=20 >>>>>> I think the first order of business is granting access rights to = the >>>>> server (varm) in question: >>>>>>=20 >>>>>> http://dan.langille.org/2015/07/19/varm/ >>>>>>=20 >>>>>> During the workshop, mention was made of serial access. I can = arrange >>>>> that. >>>>>>=20 >>>>>> The server has IPMI, however, my first thought: >>>>>>=20 >>>>>> 1 - connect a USB-serial cable to varm & link that to another = server >> in >>>>> my rack. >>>>>=20 >>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular >>>>> configuration for the test machine which made >>>>> it easy to configure and run tests? Was it PXE booting or = something? >>>>>=20 >>>>>> 2 - create a jail in that server and give it access to that = serial >>>>> connection >>>>>> 3 - redirect incoming port XYZ to that jail via a public-key-only = ssh >>>>> connection >>>>>> 4 - give people access >>>>>>=20 >>>>>> Any suggestions? >>>>>=20 >>>>> =E2=80=94 >>>>> Dan Langille >>>>> http://langille.org/ >>>>>=20 >>>>>=20 >>>> Hello Dan, >>>>=20 >>>> Yes, was me :) >>>>=20 >>>> I mention about zopkio test framework. >>>> I gave a presentation last weekend at PyCon Hong Kong about it. >>>>=20 >>>> Here is my slides: >>>>=20 >> = http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests= -using-zopkio >>>>=20 >>>> The good of Zopkio is, we can write tests at once and run it as = much as >> we >>>> want in different machines. Also Zopkio depends of Naarad, that can >> parse a >>>> CSV file and create metrics and SLA over those metrics, plot graphs = and >> so >>>> on. Pretty nice tool!!! >>>>=20 >>>> I'm wondering if we could start to test something and maybe show it = at >>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think? >>>> What I need right now would be a list of tests that we want to = perform >> as >>>> well as what parameters we would like to take as metrics to = compare.\ >>>>=20 >>>>=20 >>>> For tests, we can start with this list: >>>> https://github.com/dlangille/zfs_benchmarks/issues >>>>=20 >>>> We can start as soon as I figure out how to provide access to the >>>> testers. See above re serial connection. >>>>=20 >>>> I want to provide access, but I want to keep access restricted to = only >>>> this box and not to the rest of my home LAN. I plan to do this via = a >>>> VLAN. >>>>=20 >>>> I could fire up a Rasperberry Pi and allow ssh into that. Will = that be >>>> enough >>>> power for what you need to do? >>>>=20 >>>>=20 >>> First of all, thanks to share the tests cases. >>>=20 >>> If I use zopkio, the best would be access SSH direct to the target >> machine >>> where I need to run the tests. For zopkio, I need to have my SSH KEY = on >> the >>> target machine. >>=20 >> I am OK with this. >>=20 >>> As I don't know your network, maybe what you could do is: Via >> RasperBerry, >>> forward the SSH to the target machine, I will pass-through via your >>> RasperBerry where you can control the access for the rest of your = LAN. >>>=20 >>> Another approach could be, two different subnets and a firewall. Or = as >> you >>> said, VLANS. >>=20 >> I will be doing VLANS, which have yet to be set up. >>=20 >> The target system will have ZFS pools can be configured for different >> tests (i.e. raidz2 vs raidz3). >> This will involve gpart etc because the drives & pools will need to = be >> 'wiped' between different test >> runs. >>=20 >> I seem to recall someone suggesting PXE boot and configuring the = system >> remotely. Does anyone >> recall that? That aspect of the discussion was not recorded: >> https://wiki.freebsd.org/201510DevSummit/Performance >>=20 >>=20 > Bapt@ mentioned that, this is the way how we are doing in another = project. > But in my point of view, it is not a must for our case! >=20 > The PXE wold be good if we try to test different of OS flavors, or = build > different images. OK. The only thing holding us back is: - adding the air filters to the case - moving to the new switch with the new VLANs It's now a matter of time. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-performance@freebsd.org Wed Nov 18 22:47:42 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44538A3239F for ; Wed, 18 Nov 2015 22:47:42 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 063591A09; Wed, 18 Nov 2015 22:47:42 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by igl9 with SMTP id 9so112804435igl.0; Wed, 18 Nov 2015 14:47:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=tsDR84x8BhjxqAbUlH/YcTt70SDgUfpXmDqh4wFKKKw=; b=ZMd0bQByqdaRrnvj0y+5tSdr7QZpfCKtJfW1/yEUFTSeQitwBZAT8sI9Ludl8xAXVq BZAEoKH7m/gPdgKN6lb6jzOgKUcixsE1xpcy0MyBOsk4JFFo8ArdEluDDGX5LDGBgJ/m S4FjI5KimsdE2acPmxLAOlVH4Aaoo7k1ncVs3lZbGRW32t/ZmR37ObxTVrwRy7gXBeAJ Yv2GNEjm8h088EaUZ/jVPrSFRezYMZQtds2jZvlLXWJbuf05w8rs5nZbEz1tRTiNvOK0 zzvYPGFadugoY/SEU0NWT+rXj1NgTT5L2vU6h6XuXGlnKcqrtuWB6z/wSlvXkdM5cHi/ lQLg== MIME-Version: 1.0 X-Received: by 10.50.27.9 with SMTP id p9mr10265849igg.28.1447886861407; Wed, 18 Nov 2015 14:47:41 -0800 (PST) Received: by 10.107.179.197 with HTTP; Wed, 18 Nov 2015 14:47:41 -0800 (PST) In-Reply-To: References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> Date: Wed, 18 Nov 2015 17:47:41 -0500 Message-ID: Subject: Re: Measuring ZFS configuration differences From: Tom Curry To: Dan Langille Cc: Marcelo Araujo , freebsd-performance@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 22:47:42 -0000 On Wed, Nov 18, 2015 at 4:12 PM, Dan Langille wrote: > > > On Nov 17, 2015, at 9:31 PM, Marcelo Araujo > wrote: > > > > 2015-11-18 10:23 GMT+08:00 Dan Langille : > > > >> > >>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo > >> wrote: > >>> > >>> 2015-11-18 3:14 GMT+08:00 Dan Langille : > >>> > >>>> > >>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo > >>>> wrote: > >>>> > >>>> > >>>> > >>>> 2015-11-12 6:34 GMT+08:00 Dan Langille : > >>>> > >>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille wrote: > >>>>>> > >>>>>> Following up on the discussions during EuroBSDCon 2015 (Stockholm) > >>>>> during the FreeBSD Developer > >>>>>> Summit regarding various ZFS configuration settings, I write to > start > >>>>> our implementation phase now that some > >>>>>> usual suspects have joined the list. > >>>>>> > >>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance > >>>>>> > >>>>>> I think the first order of business is granting access rights to t= he > >>>>> server (varm) in question: > >>>>>> > >>>>>> http://dan.langille.org/2015/07/19/varm/ > >>>>>> > >>>>>> During the workshop, mention was made of serial access. I can > arrange > >>>>> that. > >>>>>> > >>>>>> The server has IPMI, however, my first thought: > >>>>>> > >>>>>> 1 - connect a USB-serial cable to varm & link that to another serv= er > >> in > >>>>> my rack. > >>>>> > >>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular > >>>>> configuration for the test machine which made > >>>>> it easy to configure and run tests? Was it PXE booting or somethin= g? > >>>>> > >>>>>> 2 - create a jail in that server and give it access to that serial > >>>>> connection > >>>>>> 3 - redirect incoming port XYZ to that jail via a public-key-only > ssh > >>>>> connection > >>>>>> 4 - give people access > >>>>>> > >>>>>> Any suggestions? > >>>>> > >>>>> =E2=80=94 > >>>>> Dan Langille > >>>>> http://langille.org/ > >>>>> > >>>>> > >>>> Hello Dan, > >>>> > >>>> Yes, was me :) > >>>> > >>>> I mention about zopkio test framework. > >>>> I gave a presentation last weekend at PyCon Hong Kong about it. > >>>> > >>>> Here is my slides: > >>>> > >> > http://www.slideshare.net/araujobsd/functional-and-scale-performance-test= s-using-zopkio > >>>> > >>>> The good of Zopkio is, we can write tests at once and run it as much > as > >> we > >>>> want in different machines. Also Zopkio depends of Naarad, that can > >> parse a > >>>> CSV file and create metrics and SLA over those metrics, plot graphs > and > >> so > >>>> on. Pretty nice tool!!! > >>>> > >>>> I'm wondering if we could start to test something and maybe show it = at > >>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think? > >>>> What I need right now would be a list of tests that we want to perfo= rm > >> as > >>>> well as what parameters we would like to take as metrics to compare.= \ > >>>> > >>>> > >>>> For tests, we can start with this list: > >>>> https://github.com/dlangille/zfs_benchmarks/issues > >>>> > >>>> We can start as soon as I figure out how to provide access to the > >>>> testers. See above re serial connection. > >>>> > >>>> I want to provide access, but I want to keep access restricted to on= ly > >>>> this box and not to the rest of my home LAN. I plan to do this via = a > >>>> VLAN. > >>>> > >>>> I could fire up a Rasperberry Pi and allow ssh into that. Will that > be > >>>> enough > >>>> power for what you need to do? > >>>> > >>>> > >>> First of all, thanks to share the tests cases. > >>> > >>> If I use zopkio, the best would be access SSH direct to the target > >> machine > >>> where I need to run the tests. For zopkio, I need to have my SSH KEY = on > >> the > >>> target machine. > >> > >> I am OK with this. > >> > >>> As I don't know your network, maybe what you could do is: Via > >> RasperBerry, > >>> forward the SSH to the target machine, I will pass-through via your > >>> RasperBerry where you can control the access for the rest of your LAN= . > >>> > >>> Another approach could be, two different subnets and a firewall. Or a= s > >> you > >>> said, VLANS. > >> > >> I will be doing VLANS, which have yet to be set up. > >> > >> The target system will have ZFS pools can be configured for different > >> tests (i.e. raidz2 vs raidz3). > >> This will involve gpart etc because the drives & pools will need to be > >> 'wiped' between different test > >> runs. > >> > >> I seem to recall someone suggesting PXE boot and configuring the syste= m > >> remotely. Does anyone > >> recall that? That aspect of the discussion was not recorded: > >> https://wiki.freebsd.org/201510DevSummit/Performance > >> > >> > > Bapt@ mentioned that, this is the way how we are doing in another > project. > > But in my point of view, it is not a must for our case! > > > > The PXE wold be good if we try to test different of OS flavors, or buil= d > > different images. > > OK. The only thing holding us back is: > > - adding the air filters to the case > - moving to the new switch with the new VLANs > > It's now a matter of time. > > =E2=80=94 > Dan Langille > http://langille.org/ > > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" > Dan, I don't know if you have already cracked the serial access nut, but every supermicro motherboard I have owned has supported serial over lan. I personally have never used it, I have just noticed it from time to time. Here is a good resource on setting it up http://serverfault.com/questions/574351/serial-over-lan-on-freebsd-10-0-wit= h-supermicro-x9-scm-f (note: it's for an X9 motherboard but it should work for your X10, see the comment at the very end) Also, FreeBSD, ZFS and performance are three of my most favorite words. Is there any way I can participate in this adventure? From owner-freebsd-performance@freebsd.org Thu Nov 19 12:02:41 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79B47A332C8 for ; Thu, 19 Nov 2015 12:02:41 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 43ED31E6D for ; Thu, 19 Nov 2015 12:02:41 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [127.0.0.1] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id B8C0F24E7 for ; Thu, 19 Nov 2015 15:02:32 +0300 (MSK) Reply-To: lev@FreeBSD.org To: freebsd-performance@freebsd.org From: Lev Serebryakov Subject: Is it possible to mimic Linux' "perf" on FreeBSD? pmcstat -WHAT? X-Enigmail-Draft-Status: N0110 Organization: FreeBSD Message-ID: <564DBA59.8000908@FreeBSD.org> Date: Thu, 19 Nov 2015 15:02:33 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 12:02:41 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I need to mimic Linux perf output on FreeBSD. It should not be exact match (I could do any text transformation), but it should be semantically same. Now on Linux these arguments are used (to be honest, I'm not very good at Linux perf syntax): perf record --freq 1000 --event cycles,instructions ... And after that perf script --fields time,event,ip,sym,dso Looks like, "pmcstat -P " is close, but not exactly what I need, as I could not find a way to get fine-grained output (per-address, not per-symbol) from it. Also, it doesn't allow "cycles" event to be passed to "-P" option :( - -- // Lev Serebryakov -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJWTbpZXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePLyUP/1aj+4eeFpT75SI8WOeukJAG oIWuhui6yjpecsCKXhcJZ1N5qHZ5BI8H/DSKOySEvUd66cxpaZchFhICjuk0oxeh 87iqFVeHGuAoXUYw28MnXp7R+0mZKQbRy9uiAhA3iodVvZQl5oohiLjhqoNifwqE hAEKXBGaBcsxgcRndX97P44W8xw1cOgVEu96s7mbXAaPS3i5PkGif3bMdj4m0hiq zSEjvKs3kMlK7iUgTjf3r7VcFVlU3TxUoaLHe0wLWfH5pUA/OywyTH8opmmiiFJV gpjo5TRn9xghycnHKnqdBWTQ1DQtGvnJyWnSkccjBdS+GDwND3mO8X5+de0eZ+QX CRh3fzDOYv0Id/gHmlddrsXbr50BHyUS7GU3xEXiJNPX38gksTPZwIXZQew+YBg+ QXwUXxmuJWTGnVlVE0rqfOM/3OJ/OznAnYhGmXtobXO/dfRxaCpWayleQhPPQsvu 4vNX+1MiC8SC8jfxNu1WCZS7bXcD3auj6OmwoaBqFWzKNrEK62RhB9gXU4MhhgW3 szvpC5aA6gRpnsI0YEFTcWoLLK3qzU3Uf/l9eRJRN757gwIOH3WhcadIDBJ0c4rW /jEpg86reNP5/ReMQxGRf1pzVT+ROJgTPg6xfziFu8klWNksl7kfYE4jFPslA+hD Kh5aBHyZSxbbqvAjDayN =2n3C -----END PGP SIGNATURE----- From owner-freebsd-performance@freebsd.org Fri Nov 20 16:22:24 2015 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7DCFA318C9 for ; Fri, 20 Nov 2015 16:22:24 +0000 (UTC) (envelope-from dan@langille.org) Received: from clavin2.langille.org (clavin2.langille.org [199.233.228.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C28A1537; Fri, 20 Nov 2015 16:22:24 +0000 (UTC) (envelope-from dan@langille.org) Received: from (clavin2.int.langille.org (clavin2.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 56A3A444A ; Fri, 20 Nov 2015 16:22:22 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: Measuring ZFS configuration differences From: Dan Langille In-Reply-To: Date: Fri, 20 Nov 2015 11:12:33 -0500 Cc: freebsd-performance@freebsd.org, Marcelo Araujo Content-Transfer-Encoding: quoted-printable Message-Id: <81C1CC91-F9EC-43DC-A2D8-A8BDDE28E9A3@langille.org> References: <8B37FEDC-218A-4071-8CB7-48361BB72B1D@langille.org> <14A0EA61-6545-42BB-910E-62C752D4396C@langille.org> To: Tom Curry X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2015 16:22:24 -0000 > On Nov 18, 2015, at 5:47 PM, Tom Curry wrote: >=20 > On Wed, Nov 18, 2015 at 4:12 PM, Dan Langille = wrote: >=20 >>=20 >>> On Nov 17, 2015, at 9:31 PM, Marcelo Araujo = >> wrote: >>>=20 >>> 2015-11-18 10:23 GMT+08:00 Dan Langille : >>>=20 >>>>=20 >>>>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo = >>>> wrote: >>>>>=20 >>>>> 2015-11-18 3:14 GMT+08:00 Dan Langille : >>>>>=20 >>>>>>=20 >>>>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo = >>>>>> wrote: >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> 2015-11-12 6:34 GMT+08:00 Dan Langille : >>>>>>=20 >>>>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille = wrote: >>>>>>>>=20 >>>>>>>> Following up on the discussions during EuroBSDCon 2015 = (Stockholm) >>>>>>> during the FreeBSD Developer >>>>>>>> Summit regarding various ZFS configuration settings, I write to >> start >>>>>>> our implementation phase now that some >>>>>>>> usual suspects have joined the list. >>>>>>>>=20 >>>>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance >>>>>>>>=20 >>>>>>>> I think the first order of business is granting access rights = to the >>>>>>> server (varm) in question: >>>>>>>>=20 >>>>>>>> http://dan.langille.org/2015/07/19/varm/ >>>>>>>>=20 >>>>>>>> During the workshop, mention was made of serial access. I can >> arrange >>>>>>> that. >>>>>>>>=20 >>>>>>>> The server has IPMI, however, my first thought: >>>>>>>>=20 >>>>>>>> 1 - connect a USB-serial cable to varm & link that to another = server >>>> in >>>>>>> my rack. >>>>>>>=20 >>>>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular >>>>>>> configuration for the test machine which made >>>>>>> it easy to configure and run tests? Was it PXE booting or = something? >>>>>>>=20 >>>>>>>> 2 - create a jail in that server and give it access to that = serial >>>>>>> connection >>>>>>>> 3 - redirect incoming port XYZ to that jail via a = public-key-only >> ssh >>>>>>> connection >>>>>>>> 4 - give people access >>>>>>>>=20 >>>>>>>> Any suggestions? >>>>>>>=20 >>>>>>> =E2=80=94 >>>>>>> Dan Langille >>>>>>> http://langille.org/ >>>>>>>=20 >>>>>>>=20 >>>>>> Hello Dan, >>>>>>=20 >>>>>> Yes, was me :) >>>>>>=20 >>>>>> I mention about zopkio test framework. >>>>>> I gave a presentation last weekend at PyCon Hong Kong about it. >>>>>>=20 >>>>>> Here is my slides: >>>>>>=20 >>>>=20 >> = http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests= -using-zopkio >>>>>>=20 >>>>>> The good of Zopkio is, we can write tests at once and run it as = much >> as >>>> we >>>>>> want in different machines. Also Zopkio depends of Naarad, that = can >>>> parse a >>>>>> CSV file and create metrics and SLA over those metrics, plot = graphs >> and >>>> so >>>>>> on. Pretty nice tool!!! >>>>>>=20 >>>>>> I'm wondering if we could start to test something and maybe show = it at >>>>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think? >>>>>> What I need right now would be a list of tests that we want to = perform >>>> as >>>>>> well as what parameters we would like to take as metrics to = compare.\ >>>>>>=20 >>>>>>=20 >>>>>> For tests, we can start with this list: >>>>>> https://github.com/dlangille/zfs_benchmarks/issues >>>>>>=20 >>>>>> We can start as soon as I figure out how to provide access to the >>>>>> testers. See above re serial connection. >>>>>>=20 >>>>>> I want to provide access, but I want to keep access restricted to = only >>>>>> this box and not to the rest of my home LAN. I plan to do this = via a >>>>>> VLAN. >>>>>>=20 >>>>>> I could fire up a Rasperberry Pi and allow ssh into that. Will = that >> be >>>>>> enough >>>>>> power for what you need to do? >>>>>>=20 >>>>>>=20 >>>>> First of all, thanks to share the tests cases. >>>>>=20 >>>>> If I use zopkio, the best would be access SSH direct to the target >>>> machine >>>>> where I need to run the tests. For zopkio, I need to have my SSH = KEY on >>>> the >>>>> target machine. >>>>=20 >>>> I am OK with this. >>>>=20 >>>>> As I don't know your network, maybe what you could do is: Via >>>> RasperBerry, >>>>> forward the SSH to the target machine, I will pass-through via = your >>>>> RasperBerry where you can control the access for the rest of your = LAN. >>>>>=20 >>>>> Another approach could be, two different subnets and a firewall. = Or as >>>> you >>>>> said, VLANS. >>>>=20 >>>> I will be doing VLANS, which have yet to be set up. >>>>=20 >>>> The target system will have ZFS pools can be configured for = different >>>> tests (i.e. raidz2 vs raidz3). >>>> This will involve gpart etc because the drives & pools will need to = be >>>> 'wiped' between different test >>>> runs. >>>>=20 >>>> I seem to recall someone suggesting PXE boot and configuring the = system >>>> remotely. Does anyone >>>> recall that? That aspect of the discussion was not recorded: >>>> https://wiki.freebsd.org/201510DevSummit/Performance >>>>=20 >>>>=20 >>> Bapt@ mentioned that, this is the way how we are doing in another >> project. >>> But in my point of view, it is not a must for our case! >>>=20 >>> The PXE wold be good if we try to test different of OS flavors, or = build >>> different images. >>=20 >> OK. The only thing holding us back is: >>=20 >> - adding the air filters to the case >> - moving to the new switch with the new VLANs >>=20 >> It's now a matter of time. >>=20 >> =E2=80=94 >> Dan Langille >> http://langille.org/ >>=20 >>=20 >>=20 >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >>=20 >=20 >=20 > Dan, >=20 > I don't know if you have already cracked the serial access nut, but = every > supermicro motherboard I have owned has supported serial over lan. I > personally have never used it, I have just noticed it from time to = time. > Here is a good resource on setting it up > = http://serverfault.com/questions/574351/serial-over-lan-on-freebsd-10-0-wi= th-supermicro-x9-scm-f > (note: it's for an X9 motherboard but it should work for your X10, see = the > comment at the very end) If we go that way, I will keep that in mind, thank you. > Also, FreeBSD, ZFS and performance are three of my most favorite = words. Is > there any way I can participate in this adventure? Yes, there will be, I'm sure.=20 This comes to mind immediately: running the tests on your own server = will validate/invalidate our results. For now, we have no results, because it's all waiting on me and my = network at home. It's a time issue. This weekend is pretty much filled with other stuff. =E2=80=94=20 Dan Langille http://langille.org/