From owner-freebsd-hackers@FreeBSD.ORG Tue May 20 20:43:04 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76553D56 for ; Tue, 20 May 2014 20:43:04 +0000 (UTC) Received: from csmtp7.one.com (csmtp7.one.com [195.47.247.107]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 376A920C1 for ; Tue, 20 May 2014 20:43:04 +0000 (UTC) Received: from bigmac.router9fbd7c.com (unknown [176.222.238.90]) by csmtp7.one.com (Postfix) with ESMTPA id 87CF8400000AF for ; Tue, 20 May 2014 20:33:02 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: [GSoC] Machine readable output from userland utilities From: Erik Cederstrand In-Reply-To: <49E9736E-AD14-4647-8B15-30603D01360C@mail.bg> Date: Tue, 20 May 2014 22:33:01 +0200 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <91FE2526-F21C-42AB-BECB-058DBA975A9E@cederstrand.dk> References: <49E9736E-AD14-4647-8B15-30603D01360C@mail.bg> X-Mailer: Apple Mail (2.1878.2) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 20:43:04 -0000 Den 20/05/2014 kl. 18.59 skrev Zaro Korchev : > I'm working on the project "Machine readable output from userland = utilities" and I want to share my ideas and thoughts. >=20 > [...] > Do you have any suggestions or ideas? Microsoft PowerShell does something similar. They print human-readable = output to the shell, but pass C# (I think) object instances along when = piping data to other utilities. The idea of separating formatting from = data is good, but it also leads to insanity because what you see is = never what is passed on to the next utility. The utilities need to = support the filtering and processing you would otherwise do with awk, = grep, sed etc., and you need to look up two manual pages - one for the = human output, and one for the C# models. Also, you can only combine = utilities that were meant to be combined. This is a huge loss of = flexibility compared to what I'm used to from UNIX. Has anyone given general thought to where the "Machine readable output = from userland utilities" idea is headed? In particular, what should = happen if I stick a pipe at the end of a command? Do we want utilities = to understand XML/JSON for input, too? So for example 'rm' understands = JSON from 'ls'. Unless this was really smart thought out, only tools = that were prepared to work together would be able to do so, and you = would need to look up which utilities work together. Even if we leave input alone and focus on output, we might want to let = sed, grep, awk and friends understand JSON/XML input so we can e.g. use = XPath syntax to grep for certain elements in XML output, or filter = certain JSON elements from the output. At least I think FreeBSD should = gain command-line tools that offer at a subset of the JSON and XML = functionality that e.g. Python offers when working with these data = formats. Using simple awk and grep with these formats would be regex = madness. I've had long hours fighting with grep, awk and others to get simple = scripts to behave properly due to the machine-unfriendly format of some = tools, so I really like the idea of machine-readable output. sysctl = offers some good examples of do's and dont's in this regard, and I feel = that the concept of machine readability needs to be thought through very = carefully to be exactly as awesome as when I first started grep'ing, = awk'ing, sed'ing and piping my way through the UNIX shell. Erik=