Date: Tue, 20 May 2014 21:58:04 -0700 (PDT) From: "Chris H" <bsd-lists@bsdforge.com> To: "Alfred Perlstein" <bright@mu.org> Cc: freebsd-hackers@freebsd.org, Erik Cederstrand <erik+lists@cederstrand.dk> Subject: Re: [GSoC] Machine readable output from userland utilities Message-ID: <f17ef374463361cc4d42009f7b418f67.authenticated@ultimatedns.net> In-Reply-To: <537C2993.1060206@mu.org> References: <49E9736E-AD14-4647-8B15-30603D01360C@mail.bg> <91FE2526-F21C-42AB-BECB-058DBA975A9E@cederstrand.dk> <537C2993.1060206@mu.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> > On 5/20/14, 1:33 PM, Erik Cederstrand wrote: >> Den 20/05/2014 kl. 18.59 skrev Zaro Korchev <zkorchev@mail.bg>: >> >>> I'm working on the project "Machine readable output from userland utilities" and I want >>> to share my ideas and thoughts. >>> >>> [...] >>> Do you have any suggestions or ideas? >> Microsoft PowerShell does something similar. They print human-readable output to the >> shell, but pass C# (I think) object instances along when piping data to other utilities. >> The idea of separating formatting from data is good, but it also leads to insanity because >> what you see is never what is passed on to the next utility. The utilities need to support >> the filtering and processing you would otherwise do with awk, grep, sed etc., and you need >> to look up two manual pages - one for the human output, and one for the C# models. Also, >> you can only combine utilities that were meant to be combined. This is a huge loss of >> flexibility compared to what I'm used to from UNIX. >> >> Has anyone given general thought to where the "Machine readable output from userland >> utilities" idea is headed? In particular, what should happen if I stick a pipe at the end >> of a command? Do we want utilities to understand XML/JSON for input, too? So for example >> 'rm' understands JSON from 'ls'. Unless this was really smart thought out, only tools that >> were prepared to work together would be able to do so, and you would need to look up which >> utilities work together. >> >> Even if we leave input alone and focus on output, we might want to let sed, grep, awk and >> friends understand JSON/XML input so we can e.g. use XPath syntax to grep for certain >> elements in XML output, or filter certain JSON elements from the output. At least I think >> FreeBSD should gain command-line tools that offer at a subset of the JSON and XML >> functionality that e.g. Python offers when working with these data formats. Using simple >> awk and grep with these formats would be regex madness. >> >> I've had long hours fighting with grep, awk and others to get simple scripts to behave >> properly due to the machine-unfriendly format of some tools, so I really like the idea of >> machine-readable output. sysctl offers some good examples of do's and dont's in this >> regard, and I feel that the concept of machine readability needs to be thought through >> very carefully to be exactly as awesome as when I first started grep'ing, awk'ing, sed'ing >> and piping my way through the UNIX shell. >> > > Basically the idea would be to write a simple tool that is able to > extract using an xpath or json selector. > > Example (very rough code): > > ifconfig --output xml | selector --format xml --path /name --path > /name/etheraddr | \ > while read name ether ; do > echo "Interface $name has hardware address $ether" ; > done > > In all seriousness though, the real target is people writing higher > level languages (than shell) on top of FreeBSD. Perhaps python or ruby > spawning a utility and then that utility making the output easy to read. > > One thing to note is that the output should not just be formatted but > normalized as well. The fact that "uptime" can emit 15 different > formats for the uptime string is terrible for people coding on top of > the base utils, the json/xml/other output should be decided on some form > of normalized data likely in seconds + microseconds or something, but > anything truly machine readable is better than the current output when > popen'd by a webapp. > > -Alfred Greetings, all. I may be getting into this thread a bit late in the game. But if I understand the gist of this correctly; isn't all this pretty much what Perl was intended for? All the best. --Chris > > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f17ef374463361cc4d42009f7b418f67.authenticated>