Date: Tue, 23 Sep 2014 16:54:53 +0800 From: Julian Elischer <julian@freebsd.org> To: Stefan Parvu <sparvu@systemdatarecorder.org>, freebsd-performance@freebsd.org Subject: Re: I like iostat, but... Message-ID: <5421355D.1020305@freebsd.org> In-Reply-To: <20140923113844.6f9e9584965dfd401f6943af@systemdatarecorder.org> References: <20140922212209.GA9619@albert.catwhisker.org> <20140923113844.6f9e9584965dfd401f6943af@systemdatarecorder.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/23/14, 4:38 PM, Stefan Parvu wrote: >> ... I rather wish I could get the same information via sysctl. (Well, >> something seems to be available via the "opaque" kern.devstat.all >> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing >> it seems as if that would require knowledge about the internals of the >> system where the data were acquired.) > I gave up parsing sysctl via Perl for disks and network devices. It would be > nice to have devstat properly working via sysctl for disk devices. Similar way > kern.cp_times does. Currently there is no simple way to extract per disk stats from > sysctl as a Perl or Sh consumer, unless we build a C module to do that. > >> If iostat(8) could be taught to (optionally) provide a timestamp, that >> might suffice. > In fact all performance userland tools should be able to nicely produce timestamp > CSV records which can be used for capacity planning and feed them to an > analytic product. Something like: > > 1411445589:4.01:16.03:383.97:1.93:0.29:1.68:0.11:95.99:0.00:0.00:0.00:0.00:123.00:229516.00:570992.00 > > or something like iostat: > > 1411445733:ada0:0.00:0.00:0.00:0.00:0.00:0.00:0.00 > > where the first field would be always the timestmp, unix time. It is not that complicated > but it does not exist. > >> The problem I'm trying to solve is this: I need to be able to acquire >> various resource counters (along with timestamps), so I can post-process >> the acquired data (generally, on a system other than the one where the >> data were gathered) in order to be able to see how the >> resource-consumption changes over the duration of a moderately >> long-running task (typically. 0.5 - 8 hrs.). > My idea of having standard data recorders: sysrec, cpurec, nicrec diskrec > which can extract: overall system consumption, per device statistics. > http://www.systemdatarecorder.org/recording/agents.html > > First place to start with will be sysrec, the main recorder which will report overall > system consumption: > > * cpu utilization across all CPUs > * memory utilization > * disk IO across all disk devices > * network IO across all NICs > * LA > http://www.systemdatarecorder.org/recording/sysrec_freebsd.html > > >> I believe that having an ability to correlate the "iostat -x" >> information with the CPU, load average, and memory utilization would be >> useful -- but I don't see a reasonable way to go about it. If anyone >> has suggestions, I'm listening. :-} > How about sysrec, like describe above. I dont like this version because it is using > iostat, netstat for disks and NICs but I dont have a better solution right now. > > There is another way for NIC stats to use libstatgrab and a Perl module for it. I got some > troubles in using it under FreeBSD 10 and having some considerable memory > usage over time, and I did not have time to test it further. But I will a bit later. > There are some limitations about libstatgrab: does not know per CPU stats for > example. > > > http://www.i-scream.org/libstatgrab/ > try this: http://lists.freebsd.org/pipermail/freebsd-current/2006-August/065003.html
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5421355D.1020305>