From owner-freebsd-arch@FreeBSD.ORG Wed Apr 3 17:45:14 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 04F9F1AA; Wed, 3 Apr 2013 17:45:14 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) by mx1.freebsd.org (Postfix) with ESMTP id 459CB27B; Wed, 3 Apr 2013 17:45:13 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id y10so4505789wgg.2 for ; Wed, 03 Apr 2013 10:45:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=VZiuGLxlTwXYfGaZSv20CU3FgUUzEqhUxF8JeK7df+0=; b=kTgVg99MAkoXXrK9J/kZz0CqAnO/KaF0pu0Eiqv4ORzBQYjPnq/qUGtWcbSeoj8YqC QutDKEOYY7CFRuDPFLpSQwEquDkQzWe2AT3nZ5Mnvr75JxzXEEBPNSofJOWGTAGptF8h Ibd0xM1NxYt6hKbWnPbui7F35tyxi5KmK/xFMOjR0BeZX+D3c+6ZOFK6aT7G2zFW4GGV t2IoZp0FeFXIkhWb9kAWYVaPlJX047qrdG2fR9VABvAkDltM/19ZxiQtw73H98RnYESt WaTw786M78EEUrjfqSMTdL8MD3fYBVwGtZqf300vb6Xay9cKPpHwdu9+T1NcfXZKrYBN HY8A== MIME-Version: 1.0 X-Received: by 10.194.119.33 with SMTP id kr1mr4314848wjb.36.1365011112389; Wed, 03 Apr 2013 10:45:12 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.243.7 with HTTP; Wed, 3 Apr 2013 10:45:12 -0700 (PDT) In-Reply-To: <515C68B5.2010006@ixsystems.com> References: <20130401115128.GZ76816@FreeBSD.org> <20130402232606.GC1810@garage.freebsd.pl> <20130403002846.GB15334@onelab2.iet.unipi.it> <20130403100401.GA1349@garage.freebsd.pl> <515C68B5.2010006@ixsystems.com> Date: Wed, 3 Apr 2013 10:45:12 -0700 X-Google-Sender-Auth: 2XL9mwIQrjTTagskhnTXwEtqJFA Message-ID: Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters From: Adrian Chadd To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org, Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 17:45:14 -0000 On 3 April 2013 10:36, Alfred Perlstein wrote: > Hey folks, sorry for the top post here, but I just came into this thread. > > Here at iXsystems we've just developed a set of scripts to scrape the > various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc, > etc) and put them into graphs based on time. Cool! > The goal is to be able to line up all these metrics with whatever benchmark > we are currently running and be able to see what may be causing issues. > > Potentially you should be able to scroll through the graphs and see things > like "ran out of mbufs @time", "vm system began paging at @time", "buffer > deaemon went nuts @time" > > Then we can take the information back and leverage it to make tuning > decisions, or potentially change kernel algorithms. > > The only problem we have is that every user land tool has its own format, so > along with my team we have written some shell to coerce the output from the > various programs into pseudo-CSV (key/value pair) which can then be post > processed by tools to convert to CSV which can then be put into something > like open office, or put through an R program to graph it. > > I'm hoping to have something shortly. > > What I was hoping to do over the next few days was discuss with people how > we can (or should we even) fix the user land statistics tools to output > machine readable output that can be easily parsed. > > Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to > parse. > > The idea of outputting xml is good, CSV is OK, however CSV is problematic as > in the case of sysctl, if new nodes appear, then we can't begin to emit > them, we must either ignore them, or abort, or log them to auxiliary files. > Anything that makes life easier is good. > > I should be able to share our scripts within the next couple of days. that's quite shiny. It'd be really nice if we could come up with a consistent statistics display, summary and export library. That way we could write tools that used a given data fetch/display API and then we could have optional "things" that implement various export methods. For example, I'm packing up Sam Leffler's "libstatfoo" for inclusion into -HEAD, primarily so the tools that use it (wlanstats, mwlstats, athstats and all the other ath stats programs I'm using) can use it. But once I've converted the stats tool over to that, I can do a few cute things: * the library has a generic way to list all of the supported statistics fields - you register the statistic names with the library, then you can create arbitrary format strings with the information; * the library handles "now" versus "time series" data display itself - you just need to populate the statistic structure with the relevant stats and then call the right function to display things; * I plan on extending it to spit out CSV output as a generic feature, so I can start doing things like importing the output of those tools direct into rrdtool/etc without any intermediary parsing scripts; * .. and if I then add new stats fields, the (requested) output of the script won't change, so tools won't break. adrian