From owner-freebsd-performance@FreeBSD.ORG Mon Sep 22 21:22:11 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66A4E31E for ; Mon, 22 Sep 2014 21:22:11 +0000 (UTC) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33C8BABB for ; Mon, 22 Sep 2014 21:22:10 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.9/8.14.9) with ESMTP id s8MLM9QV010215 for ; Mon, 22 Sep 2014 14:22:09 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.9/8.14.9/Submit) id s8MLM9GO010214 for freebsd-performance@freebsd.org; Mon, 22 Sep 2014 14:22:09 -0700 (PDT) (envelope-from david) Date: Mon, 22 Sep 2014 14:22:09 -0700 From: David Wolfskill To: freebsd-performance@freebsd.org Subject: I like iostat, but... Message-ID: <20140922212209.GA9619@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="FL5UXtIhxfXey3p5" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2014 21:22:11 -0000 --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable =2E.. I rather wish I could get the same information via sysctl. (Well, something seems to be available via the "opaque" kern.devstat.all sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing it seems as if that would require knowledge about the internals of the system where the data were acquired.) If iostat(8) could be taught to (optionally) provide a timestamp, that might suffice. The problem I'm trying to solve is this: I need to be able to acquire various resource counters (along with timestamps), so I can post-process the acquired data (generally, on a system other than the one where the data were gathered) in order to be able to see how the resource-consumption changes over the duration of a moderately long-running task (typically. 0.5 - 8 hrs.). I have (with some success) cobbled up a shell script to do much of this. (And yes, I've measured the behavior of some typical workloads in this environment vs. merely running the workload under "/usr/bin/time -lpo", with 5 test iterations for each, there was no statistically significant difference with a 95% confidence interval.) The script: * Parses its arguments, and from that information constructs a command line (which invokes sysctl(8), and (optionally) netstat(1), and pipes that output through awk(1)). * Determines the current time-of-day ("date +%s") * Enters a loop, which: + "eval"s the constructed command line (causing a timestamped line of information to be spat out standard output); the timestamp is from the most-recently-acquired time-of-day. + Determines the current time-of-day ("date +%s"). + Based on the desired sampling interval, calculates the number of seconds to sleep. (If I could get strftime to format fractional seconds, that could be handy.) + Sleeps for the calculated interval. + Determines the current time-of-day ("date +%s"). And for most things I care about, it works fairly well. Further, I do this as a shell script precisely so I don't need to build a new version of the tool for a new target system: the script purposely only requires tools that are in base FreeBSD, and requires no special privilege to use. For some sample graphs I have generated from this kind of data, please see . I believe that having an ability to correlate the "iostat -x" information with the CPU, load average, and memory utilization would be useful -- but I don't see a reasonable way to go about it. If anyone has suggestions, I'm listening. :-} Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --FL5UXtIhxfXey3p5 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJUIJL+XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7ha0P/jLtEbOUlGC7te61kHAIAeaf T80IeCwkwZiSokelMP2/w/zTl7aSdlk67Mds+CdrONhIokbjD8Itga2kAsgAR+eW 3FNkx6Y9K89TeA+xFX0uqzvEHUXGplM1EF50VlNpgG2U5RP5cghCs1sigc0frtQ3 a9ykO0Mersbo04S0s51Z7iYCY7hQVDHfER0emHtUh1U1FDC7BRvJbqxErvlgN7ik BWa9ApHVGV3so1PF9GVP6CPchLjpJqScuupzISq6/vV4sGFOv/181ZdBo8er1QuO e5ABffLNg3dMp9RTHzIWffRwe5fH++woy4Pwacpgv8z8oGvB6ZiHE6B/rpS32erg rAiaORE8dqZRuSsYJujH/QASciED9P1iHlrt4dC9CdTzCcmR69Qp3nY5wejaZ7lc tdqVIMNaprai/U3qKdf5vbisNLtZgwx+ptuiDAl13f+AjwuM1ULgrkzFFSbcIobW Ck6HXRInUzNUguVgHIeg2YTyxO3sLt/IJnWbtEKqjF2CPy9l92XfJqfZ7dzMY9PH 7sPqFbB+65fO5SiL/mRfii1r7JaiqzXhBwk4kcWa+VazdPXa6TB2p2SJA7yWW5b8 I2Tj356YsaQ7czHivolCB7/KzuIztomTGhdKyNTAXir9yDzuLyrpAxd1WxNeHsBf vWnW9y9ug1d7gTnJafSU =wi7z -----END PGP SIGNATURE----- --FL5UXtIhxfXey3p5--