From owner-freebsd-performance@FreeBSD.ORG Mon Sep 22 21:30:52 2014 Return-Path: Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 218FB51D for ; Mon, 22 Sep 2014 21:30:52 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 53EB4BC9 for ; Mon, 22 Sep 2014 21:30:48 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA05780; Tue, 23 Sep 2014 00:30:44 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XWBC0-0006ZQ-38; Tue, 23 Sep 2014 00:30:44 +0300 Message-ID: <542094B2.5040302@FreeBSD.org> Date: Tue, 23 Sep 2014 00:29:22 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: David Wolfskill , freebsd-performance@FreeBSD.org Subject: Re: I like iostat, but... References: <20140922212209.GA9619@albert.catwhisker.org> In-Reply-To: <20140922212209.GA9619@albert.catwhisker.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 22 Sep 2014 23:03:59 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2014 21:30:52 -0000 On 23/09/2014 00:22, David Wolfskill wrote: > ... I rather wish I could get the same information via sysctl. (Well, > something seems to be available via the "opaque" kern.devstat.all > sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing > it seems as if that would require knowledge about the internals of the > system where the data were acquired.) Perhaps sysutils/devstat could be of help? > If iostat(8) could be taught to (optionally) provide a timestamp, that > might suffice. > > The problem I'm trying to solve is this: I need to be able to acquire > various resource counters (along with timestamps), so I can post-process > the acquired data (generally, on a system other than the one where the > data were gathered) in order to be able to see how the > resource-consumption changes over the duration of a moderately > long-running task (typically. 0.5 - 8 hrs.). > > I have (with some success) cobbled up a shell script to do much of > this. (And yes, I've measured the behavior of some typical workloads > in this environment vs. merely running the workload under "/usr/bin/time > -lpo", with 5 test iterations for each, there was no statistically > significant difference with a 95% confidence interval.) > > The script: > > * Parses its arguments, and from that information constructs a command > line (which invokes sysctl(8), and (optionally) netstat(1), and > pipes that output through awk(1)). > > * Determines the current time-of-day ("date +%s") > > * Enters a loop, which: > + "eval"s the constructed command line (causing a timestamped line of > information to be spat out standard output); the timestamp is from > the most-recently-acquired time-of-day. > + Determines the current time-of-day ("date +%s"). > + Based on the desired sampling interval, calculates the number of > seconds to sleep. (If I could get strftime to format fractional > seconds, that could be handy.) > + Sleeps for the calculated interval. > + Determines the current time-of-day ("date +%s"). > > And for most things I care about, it works fairly well. > > Further, I do this as a shell script precisely so I don't need to build > a new version of the tool for a new target system: the script purposely > only requires tools that are in base FreeBSD, and requires no special > privilege to use. > > For some sample graphs I have generated from this kind of data, please > see . > > I believe that having an ability to correlate the "iostat -x" > information with the CPU, load average, and memory utilization would be > useful -- but I don't see a reasonable way to go about it. If anyone > has suggestions, I'm listening. :-} -- Andriy Gapon