Date: Thu, 24 Jul 2014 21:49:21 -0700 From: Simon Gerraty <sjg@juniper.net> To: <arch@freebsd.org> Cc: sjg@freebsd.org, marcel@freebsd.org, phil@juniper.net Subject: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML Message-ID: <20140725044921.9F0D3580A2@chaos.jnpr.net>
next in thread | raw e-mail | index | archive | help
Hi,
At a vendor summit a few years ago I asked about whether anyone but us
(Juniper) would be interested in the ablity to have standard BSD apps
output XML.
I was actually surprised by the amount of interest expressed.
I've occasionally nagged our UI team ever since for a clean and simple
API that we could contribute to address this.
We now have a what I think is a viable candidate
and we'd like to take the next steps towards contributing it and
converting at least a few apps.
Not only does it handle TXT and XML output but JSON and HTML as well,
and very rich HTML at that.
With some slick javascript - you can do amazing things with the level of
detail you can get out of this sort of thing.
The API is of necessity a bit more complex than just printf(3).
Considering the level of functionality available though it is a good
tradeoff.
The main open issue (assuming this functionality is still desired) is
support of wide charachters.
We figure the worst case solution is a sed(1) script to generate the wide
version of the API from the normal one, but perhaps simply always using
UTF8 would be a better solution?
Thanks
--sjg
The following from Phil provides some idea of the functionality
available and the API.
The one shown here uses the default output handle (stdout), but there
are variants that allow multiple output handles.
>Here's some sample output (from my libxo-ified "w"):
>
>(This uses an environment variable to trigger the mode and options:
> T X J and H are the modes (text, xml, json, html);
> P means pretty print (indent, newlines)
> I means print help (datatype, description), if provided (there aren't for "w")
> x means print xpath to the data)
>
>% foreach i ( T XP JP HP HPIx )
> echo === $i ===
> env LIBXO_OPTIONS=$i ./xtest -n | head -10
> end
>=== T ===
> 6:47PM up 18 days, 2:01, 9 user%s, load averages: 0.00, 0.00, 0.00
>USER TTY FROM LOGIN@ IDLE WHAT
>phil pts/0 76.182.32.73 5:09PM 33 /bin/sh
>phil pts/1 76.182.32.73 05Jul14 2 /usr/bin/perl /u/phil/bin/plum (
>phil pts/2 76.182.32.73 05Jul14 1 /bin/tcsh
>phil pts/3 76.182.32.73 05Jul14 2days ssh dent
>phil pts/4 76.182.32.73 Tue02PM 2days ssh svl-junos-d026.juniper.net
>phil pts/5 76.182.32.73 Wed01AM 2days telnet man-o-war 2006
>phil pts/6 76.182.32.73 Fri10PM 2days ssh 198.85.229.65
>phil pts/7 76.182.32.73 Fri10PM 2days ssh zap
>=== XP ===
><uptime-information>
> <time-of-day> 6:47PM</time-of-day>
> <uptime seconds="1562436">18 days</uptime>
> <uptime> 2:01</uptime>
> <users>9</users>
> <load-average-1>0.00</load-average-1>
> <load-average-5>0.00</load-average-5>
> <load-average-15>0.00</load-average-15>
> <user-table>
> <user-entry>
>=== JP ===
>"uptime-information": {
> "time-of-day": " 6:47PM",
> "uptime": "18 days",
> "uptime": 2:01,
> "users": 9,
> "load-average-1": 0.00,
> "load-average-5": 0.00,
> "load-average-15": 0.00,
> "user-table": {
> "user-entry": [
>=== HP ===
><div class="line">
> <div class="data" data-tag="time-of-day"> 6:47PM</div>
> <div class="text"> </div>
> <div class="text"> up</div>
> <div class="text"> </div>
> <div class="data" data-tag="uptime">18 days</div>
> <div class="text">,</div>
> <div class="text"> </div>
> <div class="data" data-tag="uptime"> 2:01</div>
> <div class="text">,</div>
>=== HPIx ===
><div class="line">
> <div class="data" data-tag="time-of-day" data-xpath="/uptime-information/time-of-day"> 6:47PM</div>
> <div class="text"> </div>
> <div class="text"> up</div>
> <div class="text"> </div>
> <div class="data" data-tag="uptime" data-xpath="/uptime-information/uptime">18 days</div>
> <div class="text">,</div>
> <div class="text"> </div>
> <div class="data" data-tag="uptime" data-xpath="/uptime-information/uptime"> 2:01</div>
> <div class="text">,</div>
>
>Thanks,
> Phil
>
>-----
>
>FWIW: here's the diff for "w". I don't have "wchar_t" support
>yet, so I just undid it for now.
>
>
>diff -rbu /usr/src/usr.bin/w/pr_time.c ./pr_time.c
>--- /usr/src/usr.bin/w/pr_time.c 2010-12-21 12:09:25.000000000 -0500
>+++ ./pr_time.c 2014-07-21 17:12:19.000000000 -0400
>@@ -55,10 +55,10 @@
> int
> pr_attime(time_t *started, time_t *now)
> {
>- static wchar_t buf[256];
>+ static char buf[256];
> struct tm tp, tm;
> time_t diff;
>- wchar_t *fmt;
>+ char *fmt;
> int len, width, offset = 0;
>
> tp = *localtime(started);
>@@ -67,7 +67,7 @@
>
> /* If more than a week, use day-month-year. */
> if (diff > 86400 * 7)
>- fmt = L"%d%b%y";
>+ fmt = "%d%b%y";
>
> /* If not today, use day-hour-am/pm. */
> else if (tm.tm_mday != tp.tm_mday ||
>@@ -75,23 +75,23 @@
> tm.tm_year != tp.tm_year) {
> /* The line below does not take DST into consideration */
> /* else if (*now / 86400 != *started / 86400) { */
>- fmt = use_ampm ? L"%a%I%p" : L"%a%H";
>+ fmt = use_ampm ? "%a%I%p" : "%a%H";
> }
>
> /* Default is hh:mm{am,pm}. */
> else {
>- fmt = use_ampm ? L"%l:%M%p" : L"%k:%M";
>+ fmt = use_ampm ? "%l:%M%p" : "%k:%M";
> }
>
>- (void)wcsftime(buf, sizeof(buf), fmt, &tp);
>- len = wcslen(buf);
>- width = wcswidth(buf, len);
>+ (void)strftime(buf, sizeof(buf), fmt, &tp);
>+ len = strlen(buf);
>+ width = len;
> if (len == width)
>- (void)wprintf(L"%-7.7ls", buf);
>+ xo_emit("{:login-time/%-7.7s}", buf);
> else if (width < 7)
>- (void)wprintf(L"%ls%.*s", buf, 7 - width, " ");
>+ xo_emit("{:login-time/%s}%.*s", buf, 7 - width, " ");
> else {
>- (void)wprintf(L"%ls", buf);
>+ xo_emit("{:login-time/%s}", buf);
> offset = width - 7;
> }
> return (offset);
>@@ -108,7 +108,7 @@
> /* If idle more than 36 hours, print as a number of days. */
> if (idle >= 36 * 3600) {
> int days = idle / 86400;
>- (void)printf(" %dday%s ", days, days > 1 ? "s" : " " );
>+ xo_emit(" {:idle/%dday%s} ", days, days > 1 ? "s" : " " );
> if (days >= 100)
> return (2);
> if (days >= 10)
>@@ -117,15 +117,15 @@
>
> /* If idle more than an hour, print as HH:MM. */
> else if (idle >= 3600)
>- (void)printf(" %2d:%02d ",
>+ xo_emit(" {:idle/%2d:%02d/} ",
> (int)(idle / 3600), (int)((idle % 3600) / 60));
>
> else if (idle / 60 == 0)
>- (void)printf(" - ");
>+ xo_emit(" - ");
>
> /* Else print the minutes idle. */
> else
>- (void)printf(" %2d ", (int)(idle / 60));
>+ xo_emit(" {:idle/%2d} ", (int)(idle / 60));
>
> return (0); /* not idle longer than 9 days */
> }
>diff -rbu /usr/src/usr.bin/w/w.c ./w.c
>--- /usr/src/usr.bin/w/w.c 2010-12-21 12:09:25.000000000 -0500
>+++ ./w.c 2014-07-21 18:13:50.000000000 -0400
>@@ -86,6 +86,7 @@
> #include <unistd.h>
> #include <utmp.h>
> #include <vis.h>
>+#include <libxo/libxo.h>
>
> #include "extern.h"
>
>@@ -260,9 +261,12 @@
> }
> (void)fclose(ut);
>
>+ xo_open_container("uptime-information");
>+
> if (header || wcmd == 0) {
> pr_header(&now, nusers);
> if (wcmd == 0) {
>+ xo_close_container("uptime-information");
> (void)kvm_close(kd);
> exit(0);
> }
>@@ -274,11 +278,11 @@
> #define HEADER_WHAT "WHAT\n"
> #define WUSED (UT_NAMESIZE + UT_LINESIZE + W_DISPHOSTSIZE + \
> sizeof(HEADER_LOGIN_IDLE) + 3) /* header width incl. spaces */
>- (void)printf("%-*.*s %-*.*s %-*.*s %s",
>+ xo_emit("{T:/%-*.*s} {T:/%-*.*s} "
>+ "{T:/%-*.*s} {T:LOGIN@} {T:IDLE} {T:WHAT}\n",
> UT_NAMESIZE, UT_NAMESIZE, HEADER_USER,
> UT_LINESIZE, UT_LINESIZE, HEADER_TTY,
>- W_DISPHOSTSIZE, W_DISPHOSTSIZE, HEADER_FROM,
>- HEADER_LOGIN_IDLE HEADER_WHAT);
>+ W_DISPHOSTSIZE, W_DISPHOSTSIZE, HEADER_FROM);
> }
>
> if ((kp = kvm_getprocs(kd, KERN_PROC_ALL, 0, &nentries)) == NULL)
>@@ -347,6 +351,9 @@
> }
> }
>
>+ xo_open_container("user-table");
>+ xo_open_list("user-entry");
>+
> for (ep = ehead; ep != NULL; ep = ep->next) {
> char host_buf[UT_HOSTSIZE + 1];
> struct sockaddr_storage ss;
>@@ -356,6 +363,8 @@
> time_t t;
> int isaddr;
>
>+ xo_open_instance("user-entry");
>+
> host_buf[UT_HOSTSIZE] = '\0';
> strncpy(host_buf, ep->utmp.ut_host, UT_HOSTSIZE);
> p = *host_buf ? host_buf : "-";
>@@ -388,6 +397,9 @@
> p = buf;
> }
> if (dflag) {
>+ xo_open_container("process-table");
>+ xo_open_list("process-entry");
>+
> for (dkp = ep->dkp; dkp != NULL; dkp = debugproc(dkp)) {
> const char *ptr;
>
>@@ -395,23 +407,37 @@
> dkp->ki_comm, MAXCOMLEN);
> if (ptr == NULL)
> ptr = "-";
>- (void)printf("\t\t%-9d %s\n",
>+ xo_open_instance("process-entry");
>+ xo_emit("\t\t{:process-id/%-9d/%d} "
>+ "{:command/%s}\n",
> dkp->ki_pid, ptr);
>+ xo_close_instance("process-entry");
> }
>+ xo_close_list("process-entry");
>+ xo_close_container("process-table");
> }
>- (void)printf("%-*.*s %-*.*s %-*.*s ",
>+ xo_emit("{:user/%-*.*s/%@**@s} {:tty/%-*.*s/%@**@s} "
>+ "{:from/%-*.*s/%@**@s} ",
> UT_NAMESIZE, UT_NAMESIZE, ep->utmp.ut_name,
> UT_LINESIZE, UT_LINESIZE,
> strncmp(ep->utmp.ut_line, "tty", 3) &&
> strncmp(ep->utmp.ut_line, "cua", 3) ?
> ep->utmp.ut_line : ep->utmp.ut_line + 3,
> W_DISPHOSTSIZE, W_DISPHOSTSIZE, *p ? p : "-");
>+
> t = _time_to_time32(ep->utmp.ut_time);
> longattime = pr_attime(&t, &now);
> longidle = pr_idle(ep->idle);
>- (void)printf("%.*s\n", argwidth - longidle - longattime,
>- ep->args);
>+ xo_emit("{:command/%.*s/%@*@s}\n",
>+ argwidth - longidle - longattime, ep->args);
>+
>+ xo_close_instance("user-entry");
> }
>+
>+ xo_close_list("user-entry");
>+ xo_close_container("user-table");
>+ xo_close_container("uptime-information");
>+
> (void)kvm_close(kd);
> exit(0);
> }
>@@ -430,7 +456,7 @@
> */
> if (strftime(buf, sizeof(buf),
> use_ampm ? "%l:%M%p" : "%k:%M", localtime(nowp)) != 0)
>- (void)printf("%s ", buf);
>+ xo_emit("{:time-of-day/%s} ", buf);
> /*
> * Print how long system has been up.
> */
>@@ -444,35 +470,45 @@
> uptime %= 3600;
> mins = uptime / 60;
> secs = uptime % 60;
>- (void)printf(" up");
>+ xo_emit(" up");
>+ xo_attr("seconds", "%lu", (unsigned long) tp.tv_sec);
> if (days > 0)
>- (void)printf(" %d day%s,", days, days > 1 ? "s" : "");
>+ xo_emit(" {:uptime/%d day%s},",
>+ days, days > 1 ? "s" : "");
> if (hrs > 0 && mins > 0)
>- (void)printf(" %2d:%02d,", hrs, mins);
>+ xo_emit(" {:uptime/%2d:%02d},", hrs, mins);
> else if (hrs > 0)
>- (void)printf(" %d hr%s,", hrs, hrs > 1 ? "s" : "");
>+ xo_emit(" {:uptime/%d hr%s},",
>+ hrs, hrs > 1 ? "s" : "");
> else if (mins > 0)
>- (void)printf(" %d min%s,", mins, mins > 1 ? "s" : "");
>+ xo_emit(" {:uptime/%d min%s},",
>+ mins, mins > 1 ? "s" : "");
> else
>- (void)printf(" %d sec%s,", secs, secs > 1 ? "s" : "");
>+ xo_emit(" {:uptime/%d sec%s},",
>+ secs, secs > 1 ? "s" : "");
> }
>
> /* Print number of users logged in to system */
>- (void)printf(" %d user%s", nusers, nusers == 1 ? "" : "s");
>+ xo_emit(" {:users/%d} user%s", nusers, nusers == 1 ? "" : "s");
>
> /*
> * Print 1, 5, and 15 minute load averages.
> */
> if (getloadavg(avenrun, sizeof(avenrun) / sizeof(avenrun[0])) == -1)
>- (void)printf(", no load average information available\n");
>+ xo_emit(", no load average information available\n");
> else {
>- (void)printf(", load averages:");
>+ static const char *format[] = {
>+ " {:load-average-1/%.2f}",
>+ " {:load-average-5/%.2f}",
>+ " {:load-average-15/%.2f}",
>+ };
>+ xo_emit(", load averages:");
> for (i = 0; i < (int)(sizeof(avenrun) / sizeof(avenrun[0])); i++) {
> if (use_comma && i > 0)
>- (void)printf(",");
>- (void)printf(" %.2f", avenrun[i]);
>+ xo_emit(",");
>+ xo_emit(format[i], avenrun[i]);
> }
>- (void)printf("\n");
>+ xo_emit("\n");
> }
> }
>
>@@ -493,10 +529,9 @@
> usage(int wcmd)
> {
> if (wcmd)
>- (void)fprintf(stderr,
>- "usage: w [-dhin] [-M core] [-N system] [user ...]\n");
>+ xo_error("usage: w [-dhin] [-M core] [-N system] [user ...]\n");
> else
>- (void)fprintf(stderr, "usage: uptime\n");
>+ xo_error("usage: uptime\n");
> exit(1);
> }
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140725044921.9F0D3580A2>